I heard very often discussion related to messaging and event systems. People are trying to compare them and decide on one solution that suits all their needs on a platform. Comparison of an Enterprise Service Bus (ESB) and an Event base messaging system it’s wrong from the beginning.
Even if both solutions are great, they resolve different problems that we might have on a platform. ESB and Events are able to transport data between two or multiple systems, but this is where the similarities end. From the way how they are design, core features and what we can achieve with each of them, we will notify that their capabilities are limited and they don’t compete between each other.
As we will discover, later on, a complex solution might need to use both approaches to offer a consistent and competitive solution.
To be able to give hands-on examples, we will use for comparison two services that Microsoft Azure is offering.
One of the causes why we see events and messages solutions similar it causes by how both look like. Each of them has a header and a payload and their main mission is to transport content from one location to one or multiple destinations. This concept is similar to the one provided by a Datagram that is the basic transfer unit associated with a packet-switched network and well documented in the TCP/IP protocol - RFC 1594.
I like the analogy with a datagram because of how it is defined: self-contained, no dependencies of previous content exchanged and connectionless (without support/requiring of fixed connection). This can be translated in our case of supporting data exchange between two parties if fixed block sizes without requiring a stable and open connection.
Messages
When we are working on a communication channel where we need to ensure that the content is received with success a message based solution is recommended. A solution like the Azure Service Bus Queue or Topic provides us with a reliable communication channel that ensures that the message is not lost. There are times when we expect an acknowledge from the receiver(s), the case when a message based system is the best option.
In general, we need a message base solution when we are working with a command like an approach, where we have a publisher that publishes content on the wire. The expectation is that at least one system will handle the command and a job is executed. For these situations, we shall rely on messages.
A characteristic of message base systems like Azure Service Bus is the contract that exists between producer and consumer. Both sides know exactly how the message looks like, the data format and expect information in a specific way. In general, even the producer expect a response in a specific format.
Events
From the previous example with datagrams, events are more close to datagrams that messages. The purpose of data that is transported by events is more to inform than to expect a specific action to happen. They capture activities or changes on a system and publish them on different channels.
One of the biggest difference between events and messages is the expectation of the producer. For messages, the producer expects activity to happen, in comparison with an event producer that does not have any expectation.
In comparison with messages, events don’t contain any information related to who triggered the data and context information. They just report a state change or a specific value.
In general, events are used as data points generated by different systems or store information in the change of state of different actions. A sensor that reads the car speed does not expect a reaction from the consumer each time when the speed is reported. Beside this, it generates a stream of data that can be consumed and viewed in different ways – data points. In comparison with this, a device that starts the car remotely is sending a command to the car and expect action to be executed together with some acknowledgments. In this case, the device needs to generate a message that is sent to the car.
Another characteristic of events stream is the ability to consume events from specific time periods multiple times, or even peek data at a specific time interval. For message base system, like Azure Service Bus a message can be consumed only once by all subscribers and cannot be retrieved later on once it was consumed.
You might say that there are cases when a consumer can react to different events. Yes, this is true, but the logic and the way a consumer reacts it is detailed in the consumer, not in the event itself. Losing a few events in an IoT solution is accepted and will not impact the solution stability. In comparison losing a message that contains a SELL, THE SHARES can make a difference between bankruptcy or winning 10M $.
Events are seen as streams of events, where the most valuable information is obtaining by comparing events between each other and tracking the trend of data. The value of a single event is ZERO, but analyzing the events that contained the shares value for the last 12 months is priceless.
The previous example is related to statistical information or data points. Beside this, we can have events that inform us about different activities that are happening inside the system. The purpose of these events is just to inform and losing such an event would not affect our system.
A data points stream of events that are widely used is data logs. One of multiple systems or components can produce data logs events that can be later on processed. Azure Event Hubs allow us to automatically store datalog events directly to storage for later on analyze. In most of the cases you might never analyze the log events or to extract just some report from it.
Another difference between a message VS event base system is the filtering/routing capability. An event-based system like Azure Event Hub is able to retrieve us messages from the specific time period, but it does not have the capability to filter messages for consumer groups. In comparison, an ESB system like Azure Service Bus is fully capable to filter messages on each subscription based on predefined rules.
Azure Service Bus or any other messaging system can guarantee the order delivery. In an event-based system, the order can be different. When we are working with transactions, the data order is very important, making us look at a message based solution. Additional to this an event-based system is not capable to detect duplication. One of the main reasons is because is designed for high throughput, where any data processing would increase latency.
From a security perspective, at transport and storage level both approaches are similar offering encrypted storage and a secure connection. At authentication and authorization level, the message-based system is offering more options that can be seen exotic from an event-based system. This is happening because when you need to support high thoughts, complex authentication and authorization solutions that might rely on an external system could increase latency.
Event / Messages comparison
Conclusion
In a complex system, you will need both message and event-based solution. They don’t exclude one another and you can develop a solid system that combines messages and events together. A short personal for both of them can be found below.
Even if both solutions are great, they resolve different problems that we might have on a platform. ESB and Events are able to transport data between two or multiple systems, but this is where the similarities end. From the way how they are design, core features and what we can achieve with each of them, we will notify that their capabilities are limited and they don’t compete between each other.
As we will discover, later on, a complex solution might need to use both approaches to offer a consistent and competitive solution.
To be able to give hands-on examples, we will use for comparison two services that Microsoft Azure is offering.
- Azure Service Bus – an ESB solution hosted inside Microsoft Azure, that gives us the ability to work with Queues and Topics/Subscribers.
- Azure Event Hub – an event-based solution hosted inside Microsoft Azure, that gives us the ability to push and consumer millions of events in just a few seconds.
One of the causes why we see events and messages solutions similar it causes by how both look like. Each of them has a header and a payload and their main mission is to transport content from one location to one or multiple destinations. This concept is similar to the one provided by a Datagram that is the basic transfer unit associated with a packet-switched network and well documented in the TCP/IP protocol - RFC 1594.
I like the analogy with a datagram because of how it is defined: self-contained, no dependencies of previous content exchanged and connectionless (without support/requiring of fixed connection). This can be translated in our case of supporting data exchange between two parties if fixed block sizes without requiring a stable and open connection.
Messages
When we are working on a communication channel where we need to ensure that the content is received with success a message based solution is recommended. A solution like the Azure Service Bus Queue or Topic provides us with a reliable communication channel that ensures that the message is not lost. There are times when we expect an acknowledge from the receiver(s), the case when a message based system is the best option.
In general, we need a message base solution when we are working with a command like an approach, where we have a publisher that publishes content on the wire. The expectation is that at least one system will handle the command and a job is executed. For these situations, we shall rely on messages.
A characteristic of message base systems like Azure Service Bus is the contract that exists between producer and consumer. Both sides know exactly how the message looks like, the data format and expect information in a specific way. In general, even the producer expect a response in a specific format.
Events
From the previous example with datagrams, events are more close to datagrams that messages. The purpose of data that is transported by events is more to inform than to expect a specific action to happen. They capture activities or changes on a system and publish them on different channels.
One of the biggest difference between events and messages is the expectation of the producer. For messages, the producer expects activity to happen, in comparison with an event producer that does not have any expectation.
In general, events are used as data points generated by different systems or store information in the change of state of different actions. A sensor that reads the car speed does not expect a reaction from the consumer each time when the speed is reported. Beside this, it generates a stream of data that can be consumed and viewed in different ways – data points. In comparison with this, a device that starts the car remotely is sending a command to the car and expect action to be executed together with some acknowledgments. In this case, the device needs to generate a message that is sent to the car.
Another characteristic of events stream is the ability to consume events from specific time periods multiple times, or even peek data at a specific time interval. For message base system, like Azure Service Bus a message can be consumed only once by all subscribers and cannot be retrieved later on once it was consumed.
You might say that there are cases when a consumer can react to different events. Yes, this is true, but the logic and the way a consumer reacts it is detailed in the consumer, not in the event itself. Losing a few events in an IoT solution is accepted and will not impact the solution stability. In comparison losing a message that contains a SELL, THE SHARES can make a difference between bankruptcy or winning 10M $.
Events are seen as streams of events, where the most valuable information is obtaining by comparing events between each other and tracking the trend of data. The value of a single event is ZERO, but analyzing the events that contained the shares value for the last 12 months is priceless.
The previous example is related to statistical information or data points. Beside this, we can have events that inform us about different activities that are happening inside the system. The purpose of these events is just to inform and losing such an event would not affect our system.
A data points stream of events that are widely used is data logs. One of multiple systems or components can produce data logs events that can be later on processed. Azure Event Hubs allow us to automatically store datalog events directly to storage for later on analyze. In most of the cases you might never analyze the log events or to extract just some report from it.
Another difference between a message VS event base system is the filtering/routing capability. An event-based system like Azure Event Hub is able to retrieve us messages from the specific time period, but it does not have the capability to filter messages for consumer groups. In comparison, an ESB system like Azure Service Bus is fully capable to filter messages on each subscription based on predefined rules.
Azure Service Bus or any other messaging system can guarantee the order delivery. In an event-based system, the order can be different. When we are working with transactions, the data order is very important, making us look at a message based solution. Additional to this an event-based system is not capable to detect duplication. One of the main reasons is because is designed for high throughput, where any data processing would increase latency.
From a security perspective, at transport and storage level both approaches are similar offering encrypted storage and a secure connection. At authentication and authorization level, the message-based system is offering more options that can be seen exotic from an event-based system. This is happening because when you need to support high thoughts, complex authentication and authorization solutions that might rely on an external system could increase latency.
Event / Messages comparison
|
Feature / Characteristic
|
Azure Service Bus
|
Azure Event Hub
|
|
Command base
|
Yes
|
No
|
|
Reliable async delivery
|
Yes
|
Partial
|
|
Receiver confirmation
|
Yes
|
No
|
|
Data Stream
|
No
|
Yes
|
|
Content Routing/Filtering
|
YES
|
No
|
|
Fault tolerance
|
YES
|
No
|
|
Transaction handling
|
Yes
|
No
|
|
High volume of data
|
No
|
Yes
|
|
Multiple consumers
|
Yes
|
Yes
|
|
Telemetry use case
|
No
|
Yes
|
|
Order processing
|
Yes
|
No
|
|
Low latency
|
No
|
Yes
|
|
At least once delivery
|
Yes
|
Yes
|
|
One time delivery
|
Yes
|
No
|
|
Data ordering
|
Yes
|
No
|
|
Duplicate detection
|
Yes
|
No
|
|
Data consistency
|
Yes
|
No
|
|
FIFO like
|
Yes
|
No
|
|
Batching
|
Yes
|
Yes
|
|
Session
|
Yes
|
No
|
|
Dead-letter
|
Yes
|
No
|
|
Durability
|
Yes
|
Yes
|
|
TTL
|
Yes
|
Yes
|
|
Token base access (SAS)
|
Yes
|
Yes
|
|
Auto-forwarding
|
Yes
|
No
|
|
Scheduled delivery
|
Yes
|
No
|
|
Message deferral
|
Yes
|
No
|
|
Auto-delete
|
Yes
|
Yes
|
|
Retrieve old data
|
No
|
Yes
|
Conclusion
In a complex system, you will need both message and event-based solution. They don’t exclude one another and you can develop a solid system that combines messages and events together. A short personal for both of them can be found below.
- Events – <inform> notifications of a condition of state change
- Messages – <commands> raw data produced by a system to be consumed
Comments
Post a Comment