Azure Service Bus dead letter queue monitoring best practices

What is a Service Bus dead-letter Queue?

Every Service Bus Queue or Topic will have a secondary sub-queue called Dead-letter Queues to hold the messages that cannot be processed further. If the receiver has not consumed, the messages with exceptions or errors will automatically be moved to the Dead-letter Queue.

So it's a great concept that helps to:

• Move the wrong messages to a separate Queue, avoiding the regular Service Bus Queue/Topic to be full of messages that will never be processed.
• Prevent message loss, as any Service Bus message that cannot be delivered will be collected and stored in the Dead-lettered Queues.

Why set up Monitoring for Service Bus dead-letter Queues?

Here are a few limitations that could sometimes be significant problems for enterprise applications that heavily rely on messaging and communication

• By default, neither the sender nor the receiver will be notified when an active Service Bus message gets dead lettered.
• It would be hard to constantly keep track of the messages pushed into Dead-letter Queues, leading to the bulk storage of wrong messages.
• The messages in the Dead-letter Queues will remain there until we manually pick them up for reprocessing or deletion.

This scenario is when Service Bus dead-letter Queue Monitoring and Alerting comes into play!

Before diving deep, I will list the primary root causes for message dead lettering, which might help you better monitor Dead-lettered Service Bus Messages and Queues.

Reasons for Service Bus Message Dead-lettering

• The receiver is unavailable for a specific duration due to unexpected downtimes
• There might be communication failure, being not able to identify the receiver
• Sometimes the receiver would forcefully dead letter a message
• Error in the message payload or some system issues
• Message not being picked for processing before the time to live is expired
• Message processing errors due to exceeding the Maximum delivery count, Header Size, Maximum transfer hop count, etc.
 
• Errors on processing Subscription Rules or when the Session ID is set to null

Different options available for Monitoring the Dead-letter Queues

Native-Azure Monitoring tools

As known, Azure Monitor provides both the monitoring and alerting capabilities for Service Bus dead-letter messages and queues. With this, you can indeed set up alert rules and action groups to get notified when your dead-letter message count reaches a particular threshold value.

Still, there are certain disadvantages you should be aware of, like, Azure Monitors does not offer features to

• Provide consolidated monitoring on state, properties, and metrics
• Monitor Queues and Topic Subscriptions dead-lettered count in a single report
• View the subscription-level metrics for dead-lettered messages
• Repair & resubmit dead-lettered messages
• Reprocess bulk dead-lettered messages on autopilot
• Automated corrective actions
• Restrict user access/audit user actions that are performed on dead-letter Queues

Azure Monitors might be a good choice for Service Bus Queues/Topics but lags a bit behind in monitoring the Service Bus dead-letter Queues and Messages.

Is building custom solutions considered the prime option?

No, not really! Few organizations, however, prefer to build their monitoring solutions because they can tailor them precisely to their needs. This way, your team can even develop exclusive features to overcome the above challenges while using Azure Monitors for Dead-letter Queues.

Of course, it is a time-consuming process that will require a separate team to maintain the solution and provide you with regular updates.

Advanced third-party tools

Currently, there are many tools available on the market, making it very important to choose the one that will fully meet all of your needs for monitoring dead letter queues and messages.

In that case, Serverless360 excels with its ability to provide out-of-box monitoring, alerting, intelligent automation, and reprocessing features for dead-lettered Service Bus messages.

Service Bus dead-letter Queue Monitoring: Serverless360

Configuring alerts and monitoring

• Serverless360 helps to stay on track of the total message count in the dead-letter queues
• Quickly get rid of messaging issues with proactive monitoring
• Instant alerts whenever an active message is moved to the dead-letter queue
• Monitor Service Bus Queues and Topics on multiple metrics or properties like dead-letter message count and notify by setting up threshold limits.

Consolidated Monitoring report

Business applications often involve many Azure services, and in some cases, there are multiple Service Buses involved. In that scenario, Serverless360 allows monitoring various Queues and Topic Subscriptions dead-lettered message count in a single report.

Monitor the dead-lettered messages on each Topic Subscriptions

The inability of Azure Monitors to provide Subscription-level monitoring can be quite a big drawback as the messages are not dead-lettered on the Topic itself but more on an individual subscription.

So with Serverless360, you can now effortlessly monitor the Topic Subscriptions on various properties, including the Dead Letter Message Count, etc., and get alerted when the message count reaches the configured threshold value.

Repair and resubmit the dead-lettered messages

Serverless360 doesn't just stop sending alerts when the messages are dead-lettered. Still, it helps identify the reason for dead-lettering and reprocess it to the same or different destination.

It offers support to get complete visibility on the message content and properties, modify the message and resubmit it, quickly find the dead-lettered messages with advanced search options, and more.

Perform operations on autopilot

These top features in Serverless360 eliminate the need for manual intervention whenever a Service Bus message is dead-lettered.

• Auto reprocess messages on a schedule that get dead lettered for known reasons
• Auto-delete the dead-lettered messages once they are successfully resubmitted
• Intelligent automation to resubmit the dead-lettered messages on bulk
• Get instantly notified when an automated task on your dead-letter queue is complete

Visualize using interactive dashboards

The tool offers dashboards for each Azure resource, including the Service Bus Queues and Topics Subscriptions. It allows us to track each resource separately and present the necessary data & metrics in an easy-to-understand way.

You can also provide granular access control and audit everything performed on your Azure service Bus using Serverless360's built-in features.

Conclusion

The focus of this blog is to help you know the concept of Service Bus dead-letter Queues and the various reasons behind message dead-lettering. Furthermore, we discussed the challenges in using native Azure monitoring tools and how Serverless360 stands out when dealing with Service Bus dead-lettered Messages and Queues.