Skip to main content

Service Bus Queues from Windows Azure - Introduction

In the following post we will talk about Service Bus Queues. Basically, this type of service bus can help us to communicate between two separate types of modules. We can have one or more producers (modules that create messages) that send messages to service bus asynchronously. A receiver can handle each message that is send to the service bus.
Using Service Bus Queues we are ensure that each message that is send to the service bus will be available to the consumer. We don’t need to wait until the consumer process the message or to check if the message was lost in the process of sending the message to the consumer.
Another nice feature is the isolation that Service Bus Queues creates between the consumer and the producers. The consumer don’t need to know who send the message to the service bus, this is not relevant and important to him. Of course we can mark messages with the producer information. The same thing happens on the message senders. They don’t know about how will consume the message. They only know about the service bus. This type of communication is named brokered messaging communication.
As the name is saying, Service Bus Queues uses a collection similar to a queue to store the messages. Because of this the consumer will be able to receive the messages in the order that were sending to the queue (FIFO – first in, first out).
When you send or get a message to the queue, the operation has a transactional behavior and atomicity. What does means? When we are sending a message or we get one or more messages from the service bus, the infrastructure guaranty to us that no message will be lost during this process.
The maxim size of messages that are in Service Bus Queues can be 5GB. From what I saw, when you try to add messages to the service bus, when the maxim size was exceeded an error will be throw that can be handled in code. In this moment each message that is added to the queue not only need to be serializable, but also the size of the message need to be under than 256 KB. In the near future the size of a message will be 1 MB, but is nothing official.
If you use this kind of mechanism, you will have a mechanism that detects automatic duplicates. What does that duplicate messages can be removed automatically, without needing to check manually. Each message that is added to the service bus has a lifetime. We can send how long a message can exist in the service bus and when he will expire. This flag is called TTL (Time To Live) and the maxim value accepted is unlimited. When a message expired it is automatically removed from service bus. The default value is 60 seconds but can be changed every easily.
A producer or a consumer can add/get more than one message. Service Bus Queues support messages batches. For example a consumer can add 10 messages with only one request.
An interesting feature is the ability to change the delivery mechanism from a safe one to receive and delete pattern. This means that the message will be automatically delete from the Service Bus Queues without waiting the consumer to confirm that he received the message and process it. This can some costs.
Each message can have a delivery time property. In this way a message can be deliver to the consumer later on. It offers us the ability to schedule the messages – when the consumer will be able to receive them. On the consumer side, we have the long poling mechanism implemented. So we can pool for a message for a long time without any problems.
At the begging of the post I sad that we can have only one consumer. Basically we can have as many consumer we want, but only one of them we will receive a message. The same message will not be broadcast to all (for this functionality we can use Service Bus Topic. We can make a small hack to be able to specify on that consumer we want a message to be sent. By setting the SessionID property to a message only the specific consumers that listen to the given sessions will be able to receive that message. But be aware, broadcasting messages to more than one consumer is not possible using Service Bus Queues.
This kind of Service Bus can be very easily integrated with WCF. The maxim number “queues” in a Service Bus unit (service namespace) is 10.000, but if you want more, this limit can be increased. Theoretically, we can have an unlimited number of clients to a service bus. The only limit that exists is for TCP communication protocol that is set to 100.
In this moment we can communicate with Service Bus Queues in different ways. We have a lot of programing languages that are supported. The base communication is REST over HTTPS or TCP with TLS. Because of this we can find libraries that help us to use Service Bus Queues written in: .NET, PHP, Node.js and Java.
Let’s see what limitations we have on Service Bus Queues. First of all the latency is around 100ms and the maxim number of messages that can be sent are 2000. A queue name from service bus can have maximum 256 characters and of course we cannot have specials characters like @ or *. We can have only letters, numbers and ‘.’ ‘_’ ‘ ‘-’.
In this moment we don’t have any type of Shared Access Signature or public Service Bus Namespace. The authentication is made using:
  • Identify provider federation
  • Role based access control
  • ACS claims
We have three types of roles that are defined. A receiver that can consume a message. A sender that can only add a message but without consume any message from it and an admin. Than can add, get and iterate throw the queue (and delete of course).
In this post I presented some base concept of Service Bus Queues. In the next post I will show you how we can work with it.

Comments

Popular posts from this blog

Windows Docker Containers can make WIN32 API calls, use COM and ASP.NET WebForms

After the last post , I received two interesting questions related to Docker and Windows. People were interested if we do Win32 API calls from a Docker container and if there is support for COM. WIN32 Support To test calls to WIN32 API, let’s try to populate SYSTEM_INFO class. [StructLayout(LayoutKind.Sequential)] public struct SYSTEM_INFO { public uint dwOemId; public uint dwPageSize; public uint lpMinimumApplicationAddress; public uint lpMaximumApplicationAddress; public uint dwActiveProcessorMask; public uint dwNumberOfProcessors; public uint dwProcessorType; public uint dwAllocationGranularity; public uint dwProcessorLevel; public uint dwProcessorRevision; } ... [DllImport("kernel32")] static extern void GetSystemInfo(ref SYSTEM_INFO pSI); ... SYSTEM_INFO pSI = new SYSTEM_INFO(

Azure AD and AWS Cognito side-by-side

In the last few weeks, I was involved in multiple opportunities on Microsoft Azure and Amazon, where we had to analyse AWS Cognito, Azure AD and other solutions that are available on the market. I decided to consolidate in one post all features and differences that I identified for both of them that we should need to take into account. Take into account that Azure AD is an identity and access management services well integrated with Microsoft stack. In comparison, AWS Cognito is just a user sign-up, sign-in and access control and nothing more. The focus is not on the main features, is more on small things that can make a difference when you want to decide where we want to store and manage our users.  This information might be useful in the future when we need to decide where we want to keep and manage our users.  Feature Azure AD (B2C, B2C) AWS Cognito Access token lifetime Default 1h – the value is configurable 1h – cannot be modified

What to do when you hit the throughput limits of Azure Storage (Blobs)

In this post we will talk about how we can detect when we hit a throughput limit of Azure Storage and what we can do in that moment. Context If we take a look on Scalability Targets of Azure Storage ( https://azure.microsoft.com/en-us/documentation/articles/storage-scalability-targets/ ) we will observe that the limits are prety high. But, based on our business logic we can end up at this limits. If you create a system that is hitted by a high number of device, you can hit easily the total number of requests rate that can be done on a Storage Account. This limits on Azure is 20.000 IOPS (entities or messages per second) where (and this is very important) the size of the request is 1KB. Normally, if you make a load tests where 20.000 clients will hit different blobs storages from the same Azure Storage Account, this limits can be reached. How we can detect this problem? From client, we can detect that this limits was reached based on the HTTP error code that is returned by HTTP