Azure IoT Hub and message priority

Azure IoT is evolving. As I sad last year, each week that pass new features are added to Azure IoT Hub that consolidate the service and help us to have a better connection with our devices.

I often seen requirements that specifies that there are alerts that needs to be send to/from device with higher priority than the rest of the messages. There is no perfect workaround and each use case might required a different approach. We will also discover that a new functionality offered by Azure IoT Hub covers one of the flows.

There are two main flows where priority messages may be required:

• Device To Backend Communication (Device2Cloud -D2C)
• Backend To Device Communication (Cloud2Device - C2D)

We would discuss each flow separately. At the end end of the post we will map the workarounds and the current solutions on the flows.

Message priority on Device To Backend Communication

The use case is not complex. We need to be able to send messages to device to our backend with different priorities. For example when an alert is triggered. Even if Azure IoT is fast, we need to be able to consume messages as fast as possible. 

1. Routing Rule

The newest and most powerful feature that is available now for this scenario is to use Routing Rules. We can define a routing rule that looks at the message heather and redirect the message to a specific Service Bus Topic, Service Bus Queue or Event Hub. 

In this way we can redirect messages with different priorities to different consumers that can handle.

2. Stream Analytics Jobs

This classical solution involves an instance of Stream Analytics behind Azure IoT Hub that looks as messages and redirect them to different outputs, based on their priority. 

It is similar with Routing Rules, but a little different, because we need to scale the Stream Analytics based on load. Also, when we are using Stream Analytics we can look at message content, in contrast with Routing Rules that can routes messages only based on message heather.

From performance perspective, it is pretty clear that is much faster if you look only at message heather and not inside message body. 

The downside of both solutions is that in both cases the message are leaving the device with the same priority. There is no way to specify at device level the priority level. Even if from technical perspective this is acceptable, because in the backend we are doing near-time processing and routing of messages, there are clients that might have clear requirements that specify that priority level shall be specified at device level and a message with higher priority needs to be consumed before a message with lower priority on backend.

3. Device Twin 

For this kind of requirements Device Twin can be a solution. If we have only 2 or 3 level or priorities, than we can send higher priority inside Device Twin. This shall work fine as long as the high priority messages are not often and the size of the message itself is not big.

In this scenario we are using Device Twin as a secondary communication channel where we are sending messages with higher priority.

Until now we looked at solutions based on Azure IoT Hub. There also other solutions, that are using other services or multiple instances of Azure IoT Hub. This means that the complexity of the system will increase and running cost will be higher. Of course, everything is coming with a price.  

4. Multiple instances of Azure IoT Hub

This solution involves having 2 instances of Azure IoT Hub. One for messages with normal priority level and another one with higher priority. We already needs to manage two instances of Azure IoT Hub, register devices to each of them, pay for both of them and so on.

Even if the solution is ugly, don't think that the cost will double from IoT Hub perspective. In theory the number of messages with higher priority should not be the same as with normal messages. Meaning that the instance size of IoT Hub can be smaller. 

This workaround can become ugly if you need multiple priorities levels....

5. Secondary Channel

It involves having one or more channels that can be used to send message with higher priority. Like in the previous case, where we had a secondary instance of IoT Hub, in this case we can have an Event Hub, a Service Bus anythings that can receive messages. Even a Web App that expose a REST API should work. 

Solutions like force you to reinvent the weal again. Things like security, redundancy, authorization and authentication, registration needs to be defined one more time, even if in Azure IoT Hub they are coming out of the box. 

Be aware, even if the solution might sounds appealing, it is more complex than this and in one way or another you'll need to write features that are already defined in IoT Hub.   

Message priority on Backend to Device communication 

In this case, we need to send messages with different priority from Backend to Device. In this moment there is no support for such a requirement.

1. Device Twin
Device Twin can be used as for Device to Backend communication to push data with higher priority to device. The good part is that the device will receive the message almost in real time, but the size and the quantity of messages with higher priority is limited.

2. Multiple instances of Azure IoT Hub
This solution involves using multiple instances of Azure IoT Hub (at least 2). For each priority level we might have a different instance of IoT Hub. In this way we can send messages with different priority faster than the rest of the messages.
The biggest downside of this solution is not only that we need to register the device in two different instances of IoT Hub, but also at device level, there will be two 'agents' that will need to run at device level and listen new messages.

3. Secondary Channel
For message with higher priority we can use an external channel. Based on my past experience I highly recommend Azure Tables that are extremity fast and can be used with success for this kind of use cases. Each device can have his own table, where we can use Partition Keys to specify different priorities.
Yes, of course, more complexity at system level and more things to take care from security perspective, but this is life (smile). 

What I prefer?

Let's assume that it is impossible to convince the client to change the requirements. 

In this case for Device to Backend communication I would go with only two channels. All messages with normal priority or lower would go inside Azure IoT Hub, where I would define routing rules. For critical messages I would add them to device twin. I would fight to reduce the number of messages that are marked as critical, challenging each message that is marked as critical.

For Backend to Device communication I would go with an approach where I would use Azure Tables for cases when the number of messages with higher level is not negligible. When the number of messages with high priority is low I would go with Device Twin.

As you can see I didn't looks at Commands. Even if Azure IoT Hub supports commands, they are send on the same channel as the rest of the messages. It means, we cannot map different priorities on top of them, but can be used with success when we need to track different notifications related to a message (command).

Keep in mind that there is no perfect solution or service. Try to find the best solution that resolve your needs.