Skip to main content

A glance in IoT protocols

We are living in a world where technology innovation is something usual. People are less impressed when they see a phone with better performance or a tablet with another great feature. Living in this world, which is interconnected makes us asking ourselves how all this devices communicate.
Worldwide, there are hundreds of protocols used by different providers. Some of them are worldwide known like MQTT (Multi-Client Publish/Subscribe Messaging) or CoAP that is a lightweight version of HTTP protocol. Each protocol has a special purpose and was designed for specific use cases.
  • DDS – design for Device To Device, used with success to communicate between machines over a very fast bus
  • MQTT - design for Device To Server, used to collect data (especially telemetric data) from devices and sending them to a server
  • XMPP – design for Device To Server, used when collected data needs to be interpreted by people (is based on XML)
  • AMQP – design for Server To Server, a protocol design to connect servers between each other, but now worldwide used to communicate between Devices and Servers

As we can see each protocol was designed for a specific use case. It is almost impossible for us to say which is the best protocol. Based on our needs and requirements we can select the one that fits best our needs.

DDS comes from Data Distribution Service and is a protocol design for Device To Device communication. It is used with success when we need to distribute data between devices, connected devices and sharing data between them.
This protocol allows us to distribute millions of messages per second over the wire with a very small latency to one or more devices. The power of this protocol stays in the simplicity, giving us the possibility to create a system that is formed from different devices that communicate in real time.
DDS is based on publish/subscribe model at Node level. In DDS, a Node can play the role of publisher creating so called Topics that can publish the Samples. Each Node will deliver the samples to the subscribers. This standard is created over UDP/IP. Last version of the protocol specification was published last year (September 2014) – version 2.2. 
All the communication is not only decoupled, but DDS handles all the actions that are usually made from application like recipient location, retry policy and fail-over mechanism. It allow users to specify the Quality Of Services (QoS) expected from the system. In this way, users has the ability to configure the system in the best and optimal way for their needs.

DDS it is implemented in a 100% distributed system, to avoid any point of failure. The components that form DDS are creating so called Global Data Space where new nodes and devices can be dynamically discovered. 
The most important characteristic of this protocol is real-time. Data needs to be delivered to the right devices in the right moment in time. Messages cannot be lost or delayed. 
DDS is worldwide used for power-grids, flying management system or financial trading. It is known as a reliable protocol that has the following 3 characteristics:
  • Real-time (atomicity at tens of microseconds) 
  • Dependable (99.999% availability)
  • High Performance (millions of messages per seconds with low latency)  

MQTT comes from Message Queue Telemetry Transport. This protocol was created to telemetry and data collection. The main scope of this protocol is to collect data from devices and transport them in a secure and safe channel to the backends. 
It is used widely used when we need to collect telemetry data from many devices from the field. This protocol is simple, perfect to be used with small devices that have limited capabilities. 
This protocol is constructed to work around a central hub where data is delivered, stored and process. Because time is not a problem when working with telemetry data, this protocol is not a real-time protocol. In comparison with DDS we could say that this protocol is slow. This is not a problem because information related to telemetry doesn't need to reach the backend immediately. 
One of the most import characteristic of this protocol is reliability. Because you don't want to lose data, this protocol is constructed over TCP/IP protocol. Because of this it is widely used for monitoring purpose especially in oil, transportation and car industry, where monitoring is a very important and crucial part of the business.

As many other M2M protocols, MQTT is based on publish and subscribe pattern. At the backend endpoint is it very common to be connected with an ESB (Enterprise Service Bus) system or other messaging systems. It is important to know, that even if in the name of the protocol, the queue word appears, there is no kind of support for queueing – only for publish/subscribe.
Because it was design for low bandwidth with high latency network it is used with success over dial-up connections or satellite links. The footprint of the protocol is very small and can be used with success even in the most low bandwidth networks. 
This protocol is widely used because of the following 3 characteristics:
  • Low bandwidth (protocol footprint is very small)
  • Simple API (5 methods that define the protocol)
  • Pushing messaging (telemetry)

XMPP comes from Extensible Messaging and Presence Protocol. Initially called Jabber, it is a protocol design for IM and used by messaging systems like Pandion, Jitsi, iChat, Pidgin or Xabber. XMPP can be implemented easily on different platforms and can be interpreted easily by people, unlike other protocols that are not human readable.
It is a text based protocol, based on XML standard. Because of this debugging and integration of this protocol is simple. It runs over TCP/IP and has support for addressing schema in email format ( This characteristic is useful when you need to create an IM system that allow person to person communication. Many implementation of this protocol are over HTTP, only a few of them are over TCP/IP. This happened because the protocol is based on XML standards. 
XML is also a weakness of this protocol, because the size of the messages is high. To be able to support this protocol, you need to have a network that doesn’t has bandwidth problems.
The reliability of this protocol is offered by TCP/IP implementation. From a speed perspective, this protocol is slow (even very slow in comparison with DDS), but perfect for person to person communication. In XMPP, the time is measured in seconds not milliseconds. The main cause of the latency is the polling mechanism that is used to check if new messages are available. There are a few implementations that are not based on polling (exception from the rule).
One of the most important characteristics of this protocol is security. From this perspective is one of the most secure protocol that are available. In combination with the addressing schema it can be used with success not only for IM. Also there are multiple different scenarios where sensors and actuators information can be aggregated and displayed on a UI using XMPP.
Because there is full support for domains and subdomains in XMPP protocol, it is not complicated to create a decentralized architecture over XMPP that contains different service providers. Because this protocol is based on XML/HTML it can run with success over 80 and 8080 ports.
The most important characteristics that define XMPP are:
  • Secure (because is over TCP/IP and can be easily integrated with HTTPs it is a very secure protocol)
  • Scalability (supporting a high number of devices)
  • Addressing Schema (simple  and efficient, having support for domains)

AMQP comes from Advanced Messaging Queuing Protocol. In comparison with previous protocols that were presented until now, this protocol is made around queuing. It is one of the most reliable protocols that are not on the market and is the most used protocol for communication between devices – IoT protocol. 
It is like a super protocol in comparison with the other 3 and has the best characteristics of each of one presented above. All the communication takes place over TCP/IP and a very important characteristic of it is not losing data. Each message that is sent over AMQP will arrive to the final destination. Because of this is one of the most reliable protocols.
 We can image AMQP protocol like a jungle of queues and messaging that flows between them. In this way, this protocol offers a good point-to-point connection with acknowledge confirmation of each message that is received. This allow AMQP to run over networks that are not reliable and connection is poor. Having a queuing mechanism behind a protocol, the system don’t need to be up all the time. 

AMQP tracks all messages that are sent over the wire and ensure that each message will be delivered. Even in the case of a server failure, a connection error, the protocol and system is powerful enough to recover the messages and send them again. 
On top of this, AMQP supports transactions, given us the possibility to execute transactions and commits. Based on this feature, AMQP can be used in multiple scenarios and situations. In combination with queuing, multiple communication scenarios can be implemented. 
The most important characteristics of AMQP are:
  • Reliable (messages are not lost)
  • Queuing (full support for queuing)
  • Transaction (support for commits)

Each protocol has his own characteristics that are useful in different scenarios. Because of this it is impossible to say what is the best protocol. DDS is for real time communication between devices, MQTT is great when we need to collect telemetry data, XMPP is very simple and perfect for IM scenarios and AMQP is the perfect protocol for connecting devices to backend.
Beside this protocols there are hundreds of other protocols available on the market. We should keep in mind that there is no perfect protocol and each protocol has his own unique characteristics. When we select the protocol we should keep in mind all the features that we need, from functional to non-functional one. 
Keep in mind that for each business, real-time has a different meaning – real-time can be in milliseconds, in seconds or even hours in some situations. Based on all business requirements we will be able to select the protocol that satisfy all our needs.          


Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Fundamental Books of a Software Engineer (version 2018)

More then six years ago I wrote a blog post about fundamental books that any software engineer (developer) should read. Now it is an excellent time to update this list with new entries.

There are 5 different categories of books, that represent the recommended path. For example, you start with Coding books, after that, you read books about Programming, Design and so on.
There are some books about C++ that I recommend not because you shall know C++, only because the concepts that you can learn from it.


Writing solid codeCode completeProgramming Pearls, more programming pearls(recommended)[NEW] Introduction to Algorithms


Refactoring (M. Fowler)Pragmatic ProgrammerClean code[NEW] Software Engineering: A Practitioner's Approach[NEW] The Mythical Man-Month[NEW] The Art of Computer Programming


Applying UML and Patterns (GRASP patterns)C++ coding standards (Sutter, Alexandrescu)The C++ programming language (Stroustrup, Part IV)Object-oriented programming (Peter Coad)P…

Entity Framework (EF) TransactionScope vs Database.BeginTransaction

In today blog post we will talk a little about a new feature that is available on EF6+ related to Transactions.
Until now, when we had to use transaction we used ‘TransactionScope’. It works great and I would say that is something that is now in our blood.
using (var scope = new TransactionScope(TransactionScopeOption.Required)) { using (SqlConnection conn = new SqlConnection("...")) { conn.Open(); SqlCommand sqlCommand = new SqlCommand(); sqlCommand.Connection = conn; sqlCommand.CommandText = ... sqlCommand.ExecuteNonQuery(); ... } scope.Complete(); } Starting with EF6.0 we have a new way to work with transactions. The new approach is based on Database.BeginTransaction(), Database.Rollback(), Database.Commit(). Yes, no more TransactionScope.
In the followi…