At the end of the last year I had the opportunity to write an application that had to handle millions of messages every day. To distribute the messages between different components of the application we used Windows Azure Service Bus Service.
In the past I talked a lot about this distribution messaging system. In this post we will see what are the magic numbers of a Topic from Windows Azure Service Bus Service.
Scope:
To see how many messages can be process in as short period of time. What is the maximum number of messages that can be processed using Service Bus using only one Topic. What is the optimal number of consumers for a subscriber or topic.
Environment:
Each message that was added to Service Bus was pretty small. We had around 100 characters in UTF7 and 3 properties added to each BrokeredMessage.
We run the tests with one; two and three subscribers with different filter rules and the result were similar.
Action:
We created 5 to 10 worker roles that started to send hundreds of messages to our Service Bus Topic. In a very short period of time we could send millions of messages.
Results:
We tried pushing on the topic around 5 million messages. We observe that after a part of the messages are send to the Topic, the server starts to be slower. By slower I mean that the response latency increased drastically. Not only on the send operation, but also on the subscribers – the messages started to be received by consumer slower and slower.
One interesting part was on the portal. Even if the Topic was almost full, the number of messages that were reported on the Topic was only around 200.000-300.000.
After different experiment, we observe that with the current version of Service Bus and with the current configuration, a Topic can handle around 1.000.000 messages every 30 minutes.
Note: This value was obtain based on our experiment with our topic and messages configuration. This value is not a generic value, for any kind of project and configuration.
Cost:
From the cost perspective, we end up with a cost around 1$ per day if we use Windows Azure Service Bus for processing 1.000.000 messages. The size of each message was calculated by us around 24kB. This means that for processing 10.000.000 messages every day for a month would cost us around 300$. Before thinking that this are a lot of money, you should take into account that Service Bus will not lose any messages and in the end we process 300.000.000 (300M) messages per month.
I would conclude that the obtained value is pretty good. It is more that we expect to process a 30 minutes job, using one topic.
In the next post I will present the number of consumers that we used to process the messages in the best way possible.
Comments
Post a Comment