Skip to main content

(Part 1) Testing the limits of Windows Azure Service Bus

At the end of the last year I had the opportunity to write an application that had to handle millions of messages every day. To distribute the messages between different components of the application we used Windows Azure Service Bus Service.
In the past I talked a lot about this distribution messaging system. In this post we will see what are the magic numbers of a Topic from Windows Azure Service Bus Service.
To see how many messages can be process in as short period of time. What is the maximum number of messages that can be processed using Service Bus using only one Topic. What is the optimal number of consumers for a subscriber or topic.
Each message that was added to Service Bus was pretty small. We had around 100 characters in UTF7 and 3 properties added to each BrokeredMessage.
We run the tests with one; two and three subscribers with different filter rules and the result were similar.
We created 5 to 10 worker roles that started to send hundreds of messages to our Service Bus Topic.  In a very short period of time we could send millions of messages.
We tried pushing on the topic around 5 million messages. We observe that after a part of the messages are send to the Topic, the server starts to be slower. By slower I mean that the response latency increased drastically. Not only on the send operation, but also on the subscribers – the messages started to be received by consumer slower and slower.
One interesting part was on the portal. Even if the Topic was almost full, the number of messages that were reported on the Topic was only around 200.000-300.000.
After different experiment, we observe that with the current version of Service Bus and with the current configuration, a Topic can handle around 1.000.000 messages every 30 minutes.
Note: This value was obtain based on our experiment with our topic and messages configuration. This value is not a generic value, for any kind of project and configuration.
From the cost perspective, we end up with a cost around 1$ per day if we use Windows Azure Service Bus for processing 1.000.000 messages. The size of each message was calculated by us around 24kB. This means that for processing 10.000.000 messages every day for a month would cost us around 300$. Before thinking that this are a lot of money, you should take into account that Service Bus will not lose any messages and in the end we process 300.000.000 (300M) messages per month.

I would conclude that the obtained value is pretty good. It is more that we expect to process a 30 minutes job, using one topic.
In the next post I will present the number of consumers that we used to process the messages in the best way possible.


Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

How to check in AngularJS if a service was register or not

There are cases when you need to check in a service or a controller was register in AngularJS.
For example a valid use case is when you have the same implementation running on multiple application. In this case, you may want to intercept the HTTP provider and add a custom step there. This step don’t needs to run on all the application, only in the one where the service exist and register.
A solution for this case would be to have a flag in the configuration that specify this. In the core you would have an IF that would check the value of this flag.
Another solution is to check if a specific service was register in AngularJS or not. If the service was register that you would execute your own logic.
To check if a service was register or not in AngularJS container you need to call the ‘has’ method of ‘inhector’. It will return TRUE if the service was register.
if ($injector.has('httpInterceptorService')) { $httpProvider.interceptors.push('httpInterceptorService&#…

Fundamental Books of a Software Engineer (version 2018)

More then six years ago I wrote a blog post about fundamental books that any software engineer (developer) should read. Now it is an excellent time to update this list with new entries.

There are 5 different categories of books, that represent the recommended path. For example, you start with Coding books, after that, you read books about Programming, Design and so on.
There are some books about C++ that I recommend not because you shall know C++, only because the concepts that you can learn from it.


Writing solid codeCode completeProgramming Pearls, more programming pearls(recommended)[NEW] Introduction to Algorithms


Refactoring (M. Fowler)Pragmatic ProgrammerClean code[NEW] Software Engineering: A Practitioner's Approach[NEW] The Mythical Man-Month[NEW] The Art of Computer Programming


Applying UML and Patterns (GRASP patterns)C++ coding standards (Sutter, Alexandrescu)The C++ programming language (Stroustrup, Part IV)Object-oriented programming (Peter Coad)P…