Skip to main content

A glance in IoT protocols

We are living in a world where technology innovation is something usual. People are less impressed when they see a phone with better performance or a tablet with another great feature. Living in this world, which is interconnected makes us asking ourselves how all this devices communicate.
Worldwide, there are hundreds of protocols used by different providers. Some of them are worldwide known like MQTT (Multi-Client Publish/Subscribe Messaging) or CoAP that is a lightweight version of HTTP protocol. Each protocol has a special purpose and was designed for specific use cases.
  • DDS – design for Device To Device, used with success to communicate between machines over a very fast bus
  • MQTT - design for Device To Server, used to collect data (especially telemetric data) from devices and sending them to a server
  • XMPP – design for Device To Server, used when collected data needs to be interpreted by people (is based on XML)
  • AMQP – design for Server To Server, a protocol design to connect servers between each other, but now worldwide used to communicate between Devices and Servers

As we can see each protocol was designed for a specific use case. It is almost impossible for us to say which is the best protocol. Based on our needs and requirements we can select the one that fits best our needs.

DDS
DDS comes from Data Distribution Service and is a protocol design for Device To Device communication. It is used with success when we need to distribute data between devices, connected devices and sharing data between them.
This protocol allows us to distribute millions of messages per second over the wire with a very small latency to one or more devices. The power of this protocol stays in the simplicity, giving us the possibility to create a system that is formed from different devices that communicate in real time.
DDS is based on publish/subscribe model at Node level. In DDS, a Node can play the role of publisher creating so called Topics that can publish the Samples. Each Node will deliver the samples to the subscribers. This standard is created over UDP/IP. Last version of the protocol specification was published last year (September 2014) – version 2.2. 
All the communication is not only decoupled, but DDS handles all the actions that are usually made from application like recipient location, retry policy and fail-over mechanism. It allow users to specify the Quality Of Services (QoS) expected from the system. In this way, users has the ability to configure the system in the best and optimal way for their needs.

DDS it is implemented in a 100% distributed system, to avoid any point of failure. The components that form DDS are creating so called Global Data Space where new nodes and devices can be dynamically discovered. 
The most important characteristic of this protocol is real-time. Data needs to be delivered to the right devices in the right moment in time. Messages cannot be lost or delayed. 
DDS is worldwide used for power-grids, flying management system or financial trading. It is known as a reliable protocol that has the following 3 characteristics:
  • Real-time (atomicity at tens of microseconds) 
  • Dependable (99.999% availability)
  • High Performance (millions of messages per seconds with low latency)  


MQTT
MQTT comes from Message Queue Telemetry Transport. This protocol was created to telemetry and data collection. The main scope of this protocol is to collect data from devices and transport them in a secure and safe channel to the backends. 
It is used widely used when we need to collect telemetry data from many devices from the field. This protocol is simple, perfect to be used with small devices that have limited capabilities. 
This protocol is constructed to work around a central hub where data is delivered, stored and process. Because time is not a problem when working with telemetry data, this protocol is not a real-time protocol. In comparison with DDS we could say that this protocol is slow. This is not a problem because information related to telemetry doesn't need to reach the backend immediately. 
One of the most import characteristic of this protocol is reliability. Because you don't want to lose data, this protocol is constructed over TCP/IP protocol. Because of this it is widely used for monitoring purpose especially in oil, transportation and car industry, where monitoring is a very important and crucial part of the business.

As many other M2M protocols, MQTT is based on publish and subscribe pattern. At the backend endpoint is it very common to be connected with an ESB (Enterprise Service Bus) system or other messaging systems. It is important to know, that even if in the name of the protocol, the queue word appears, there is no kind of support for queueing – only for publish/subscribe.
Because it was design for low bandwidth with high latency network it is used with success over dial-up connections or satellite links. The footprint of the protocol is very small and can be used with success even in the most low bandwidth networks. 
This protocol is widely used because of the following 3 characteristics:
  • Low bandwidth (protocol footprint is very small)
  • Simple API (5 methods that define the protocol)
  • Pushing messaging (telemetry)


XMPP
XMPP comes from Extensible Messaging and Presence Protocol. Initially called Jabber, it is a protocol design for IM and used by messaging systems like Pandion, Jitsi, iChat, Pidgin or Xabber. XMPP can be implemented easily on different platforms and can be interpreted easily by people, unlike other protocols that are not human readable.
It is a text based protocol, based on XML standard. Because of this debugging and integration of this protocol is simple. It runs over TCP/IP and has support for addressing schema in email format (username@customDomain.com). This characteristic is useful when you need to create an IM system that allow person to person communication. Many implementation of this protocol are over HTTP, only a few of them are over TCP/IP. This happened because the protocol is based on XML standards. 
XML is also a weakness of this protocol, because the size of the messages is high. To be able to support this protocol, you need to have a network that doesn’t has bandwidth problems.
  
The reliability of this protocol is offered by TCP/IP implementation. From a speed perspective, this protocol is slow (even very slow in comparison with DDS), but perfect for person to person communication. In XMPP, the time is measured in seconds not milliseconds. The main cause of the latency is the polling mechanism that is used to check if new messages are available. There are a few implementations that are not based on polling (exception from the rule).
One of the most important characteristics of this protocol is security. From this perspective is one of the most secure protocol that are available. In combination with the addressing schema it can be used with success not only for IM. Also there are multiple different scenarios where sensors and actuators information can be aggregated and displayed on a UI using XMPP.
Because there is full support for domains and subdomains in XMPP protocol, it is not complicated to create a decentralized architecture over XMPP that contains different service providers. Because this protocol is based on XML/HTML it can run with success over 80 and 8080 ports.
The most important characteristics that define XMPP are:
  • Secure (because is over TCP/IP and can be easily integrated with HTTPs it is a very secure protocol)
  • Scalability (supporting a high number of devices)
  • Addressing Schema (simple  and efficient, having support for domains)


AMQP
AMQP comes from Advanced Messaging Queuing Protocol. In comparison with previous protocols that were presented until now, this protocol is made around queuing. It is one of the most reliable protocols that are not on the market and is the most used protocol for communication between devices – IoT protocol. 
It is like a super protocol in comparison with the other 3 and has the best characteristics of each of one presented above. All the communication takes place over TCP/IP and a very important characteristic of it is not losing data. Each message that is sent over AMQP will arrive to the final destination. Because of this is one of the most reliable protocols.
 We can image AMQP protocol like a jungle of queues and messaging that flows between them. In this way, this protocol offers a good point-to-point connection with acknowledge confirmation of each message that is received. This allow AMQP to run over networks that are not reliable and connection is poor. Having a queuing mechanism behind a protocol, the system don’t need to be up all the time. 

AMQP tracks all messages that are sent over the wire and ensure that each message will be delivered. Even in the case of a server failure, a connection error, the protocol and system is powerful enough to recover the messages and send them again. 
On top of this, AMQP supports transactions, given us the possibility to execute transactions and commits. Based on this feature, AMQP can be used in multiple scenarios and situations. In combination with queuing, multiple communication scenarios can be implemented. 
The most important characteristics of AMQP are:
  • Reliable (messages are not lost)
  • Queuing (full support for queuing)
  • Transaction (support for commits)


Conclusion
Each protocol has his own characteristics that are useful in different scenarios. Because of this it is impossible to say what is the best protocol. DDS is for real time communication between devices, MQTT is great when we need to collect telemetry data, XMPP is very simple and perfect for IM scenarios and AMQP is the perfect protocol for connecting devices to backend.
Beside this protocols there are hundreds of other protocols available on the market. We should keep in mind that there is no perfect protocol and each protocol has his own unique characteristics. When we select the protocol we should keep in mind all the features that we need, from functional to non-functional one. 
Keep in mind that for each business, real-time has a different meaning – real-time can be in milliseconds, in seconds or even hours in some situations. Based on all business requirements we will be able to select the protocol that satisfy all our needs.          

Comments

Popular posts from this blog

Windows Docker Containers can make WIN32 API calls, use COM and ASP.NET WebForms

After the last post , I received two interesting questions related to Docker and Windows. People were interested if we do Win32 API calls from a Docker container and if there is support for COM. WIN32 Support To test calls to WIN32 API, let’s try to populate SYSTEM_INFO class. [StructLayout(LayoutKind.Sequential)] public struct SYSTEM_INFO { public uint dwOemId; public uint dwPageSize; public uint lpMinimumApplicationAddress; public uint lpMaximumApplicationAddress; public uint dwActiveProcessorMask; public uint dwNumberOfProcessors; public uint dwProcessorType; public uint dwAllocationGranularity; public uint dwProcessorLevel; public uint dwProcessorRevision; } ... [DllImport("kernel32")] static extern void GetSystemInfo(ref SYSTEM_INFO pSI); ... SYSTEM_INFO pSI = new SYSTEM_INFO(

Azure AD and AWS Cognito side-by-side

In the last few weeks, I was involved in multiple opportunities on Microsoft Azure and Amazon, where we had to analyse AWS Cognito, Azure AD and other solutions that are available on the market. I decided to consolidate in one post all features and differences that I identified for both of them that we should need to take into account. Take into account that Azure AD is an identity and access management services well integrated with Microsoft stack. In comparison, AWS Cognito is just a user sign-up, sign-in and access control and nothing more. The focus is not on the main features, is more on small things that can make a difference when you want to decide where we want to store and manage our users.  This information might be useful in the future when we need to decide where we want to keep and manage our users.  Feature Azure AD (B2C, B2C) AWS Cognito Access token lifetime Default 1h – the value is configurable 1h – cannot be modified

What to do when you hit the throughput limits of Azure Storage (Blobs)

In this post we will talk about how we can detect when we hit a throughput limit of Azure Storage and what we can do in that moment. Context If we take a look on Scalability Targets of Azure Storage ( https://azure.microsoft.com/en-us/documentation/articles/storage-scalability-targets/ ) we will observe that the limits are prety high. But, based on our business logic we can end up at this limits. If you create a system that is hitted by a high number of device, you can hit easily the total number of requests rate that can be done on a Storage Account. This limits on Azure is 20.000 IOPS (entities or messages per second) where (and this is very important) the size of the request is 1KB. Normally, if you make a load tests where 20.000 clients will hit different blobs storages from the same Azure Storage Account, this limits can be reached. How we can detect this problem? From client, we can detect that this limits was reached based on the HTTP error code that is returned by HTTP