(Part 5) Testing the limits of Windows Azure Service Bus

In the last series of post about testing the limits of Windows Azure Service Bus we saw what is the best configuration for our problem, the best configuration is to have 4 medium instances.
The next question for us is: How we can decrease the time processing?
In our case we saw that from the cost and benefits perspective we reached the limitation on Service Bus. This is around 4 medium instances that are able to process around  1.000.000 messages in 34 minutes. But what we can do if we have 10.000.000 messages that needs to be processed in 1 hour.
We could increase the number of instances, but this will not improve our performance too much. In this moment we use only one topic. This topic, as Service Bus, has its own limitations – we cannot make an undefined number of request per second and expect to have a low latency.
A solution to decrease the number of request is to use batches – we already use them.
Another solution is to scale the topics. In this moment we have only one topic. If we would use 5 topics, than we could have 4 instances that will consume messages for each topic. In this way we could consume 5.000.000 messages in 34 minutes. The downside is from the cost perspective. The costs will increase with a 5X factor.
From the cost perceptive, this would be acceptable because all the instances work at maximum capacity. In the moment when we don’t need any more this instances we could stop them.
The only problem is how we can distribute the messages between different topics.
One solution would be to identify different attributes of our messages and try to group messages based on this attributes. In this way we could distributed the messages on more than one topic. In theory this could be a good solution, but how many times you could group items in an equal way. In real life this cannot be accomplish in normal cases.
What we could do is to create a mechanism that can select what topic to use. For example each producer, at a specific time interval or after he send a specific number of messages, check what is the load on each topic and decide what topic should be used.
In this way we could have a flat distribution of messages over our topics.
In this post we saw how we can scale on horizontal Windows Azure Service Bus. With a good design we should be able to support scaling in all the locations where we expect to have a bottleneck.