Skip to main content

Azure IoT Hub and message priority

Azure IoT is evolving. As I sad last year, each week that pass new features are added to Azure IoT Hub that consolidate the service and help us to have a better connection with our devices.

I often seen requirements that specifies that there are alerts that needs to be send to/from device with higher priority than the rest of the messages. There is no perfect workaround and each use case might required a different approach. We will also discover that a new functionality offered by Azure IoT Hub covers one of the flows.
There are two main flows where priority messages may be required:
  • Device To Backend Communication (Device2Cloud -D2C)
  • Backend To Device Communication (Cloud2Device - C2D)
We would discuss each flow separately. At the end end of the post we will map the workarounds and the current solutions on the flows.

Message priority on Device To Backend Communication
The use case is not complex. We need to be able to send messages to device to our backend with different priorities. For example when an alert is triggered. Even if Azure IoT is fast, we need to be able to consume messages as fast as possible. 

1. Routing Rule
The newest and most powerful feature that is available now for this scenario is to use Routing Rules. We can define a routing rule that looks at the message heather and redirect the message to a specific Service Bus Topic, Service Bus Queue or Event Hub. 
In this way we can redirect messages with different priorities to different consumers that can handle.

2. Stream Analytics Jobs
This classical solution involves an instance of Stream Analytics behind Azure IoT Hub that looks as messages and redirect them to different outputs, based on their priority. 
It is similar with Routing Rules, but a little different, because we need to scale the Stream Analytics based on load. Also, when we are using Stream Analytics we can look at message content, in contrast with Routing Rules that can routes messages only based on message heather.

From performance perspective, it is pretty clear that is much faster if you look only at message heather and not inside message body. 
The downside of both solutions is that in both cases the message are leaving the device with the same priority. There is no way to specify at device level the priority level. Even if from technical perspective this is acceptable, because in the backend we are doing near-time processing and routing of messages, there are clients that might have clear requirements that specify that priority level shall be specified at device level and a message with higher priority needs to be consumed before a message with lower priority on backend.

3. Device Twin 
For this kind of requirements Device Twin can be a solution. If we have only 2 or 3 level or priorities, than we can send higher priority inside Device Twin. This shall work fine as long as the high priority messages are not often and the size of the message itself is not big.
In this scenario we are using Device Twin as a secondary communication channel where we are sending messages with higher priority.

Until now we looked at solutions based on Azure IoT Hub. There also other solutions, that are using other services or multiple instances of Azure IoT Hub. This means that the complexity of the system will increase and running cost will be higher. Of course, everything is coming with a price.  

4. Multiple instances of Azure IoT Hub
This solution involves having 2 instances of Azure IoT Hub. One for messages with normal priority level and another one with higher priority. We already needs to manage two instances of Azure IoT Hub, register devices to each of them, pay for both of them and so on.
Even if the solution is ugly, don't think that the cost will double from IoT Hub perspective. In theory the number of messages with higher priority should not be the same as with normal messages. Meaning that the instance size of IoT Hub can be smaller. 
This workaround can become ugly if you need multiple priorities levels....

5. Secondary Channel
It involves having one or more channels that can be used to send message with higher priority. Like in the previous case, where we had a secondary instance of IoT Hub, in this case we can have an Event Hub, a Service Bus anythings that can receive messages. Even a Web App that expose a REST API should work. 
Solutions like force you to reinvent the weal again. Things like security, redundancy, authorization and authentication, registration needs to be defined one more time, even if in Azure IoT Hub they are coming out of the box. 
Be aware, even if the solution might sounds appealing, it is more complex than this and in one way or another you'll need to write features that are already defined in IoT Hub.   

Message priority on Backend to Device communication 
In this case, we need to send messages with different priority from Backend to Device. In this moment there is no support for such a requirement.

1. Device Twin
Device Twin can be used as for Device to Backend communication to push data with higher priority to device. The good part is that the device will receive the message almost in real time, but the size and the quantity of messages with higher priority is limited.

2. Multiple instances of Azure IoT Hub
This solution involves using multiple instances of Azure IoT Hub (at least 2). For each priority level we might have a different instance of IoT Hub. In this way we can send messages with different priority faster than the rest of the messages.
The biggest downside of this solution is not only that we need to register the device in two different instances of IoT Hub, but also at device level, there will be two 'agents' that will need to run at device level and listen new messages.

3. Secondary Channel
For message with higher priority we can use an external channel. Based on my past experience I highly recommend Azure Tables that are extremity fast and can be used with success for this kind of use cases. Each device can have his own table, where we can use Partition Keys to specify different priorities.
Yes, of course, more complexity at system level and more things to take care from security perspective, but this is life (smile). 

What I prefer?
Let's assume that it is impossible to convince the client to change the requirements. 
In this case for Device to Backend communication I would go with only two channels. All messages with normal priority or lower would go inside Azure IoT Hub, where I would define routing rules. For critical messages I would add them to device twin. I would fight to reduce the number of messages that are marked as critical, challenging each message that is marked as critical.

For Backend to Device communication I would go with an approach where I would use Azure Tables for cases when the number of messages with higher level is not negligible. When the number of messages with high priority is low I would go with Device Twin.

As you can see I didn't looks at Commands. Even if Azure IoT Hub supports commands, they are send on the same channel as the rest of the messages. It means, we cannot map different priorities on top of them, but can be used with success when we need to track different notifications related to a message (command).



Keep in mind that there is no perfect solution or service. Try to find the best solution that resolve your needs. 

Comments

Popular posts from this blog

How to check in AngularJS if a service was register or not

There are cases when you need to check in a service or a controller was register in AngularJS.
For example a valid use case is when you have the same implementation running on multiple application. In this case, you may want to intercept the HTTP provider and add a custom step there. This step don’t needs to run on all the application, only in the one where the service exist and register.
A solution for this case would be to have a flag in the configuration that specify this. In the core you would have an IF that would check the value of this flag.
Another solution is to check if a specific service was register in AngularJS or not. If the service was register that you would execute your own logic.
To check if a service was register or not in AngularJS container you need to call the ‘has’ method of ‘inhector’. It will return TRUE if the service was register.
if ($injector.has('httpInterceptorService')) { $httpProvider.interceptors.push('httpInterceptorService&#…

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Run native .NET application in Docker (.NET Framework 4.6.2)

Scope
The main scope of this post is to see how we can run a legacy application written in .NET Framework in Docker.

Context
First of all, let’s define what is a legacy application in our context. By a legacy application we understand an application that runs .NET Framework 3.5 or higher in a production environment where we don’t have any more the people or documentation that would help us to understand what is happening behind the scene.
In this scenarios, you might want to migrate the current solution from a standard environment to Docker. There are many advantages for such a migration, like:

Continuous DeploymentTestingIsolationSecurity at container levelVersioning ControlEnvironment Standardization
Until now, we didn’t had the possibility to run a .NET application in Docker. With .NET Core, there was support for .NET Core in Docker, but migration from a full .NET framework to .NET Core can be costly and even impossible. Not only because of lack of features, but also because once you…