Skip to main content

(Part 3) Testing the limits of Windows Azure Service Bus

In the latest post series about the limits of Windows Azure Service Bus, we saw that is the maximum number of messages that we can process through a single topic (1.000.000 messages every 30 minutes) and what kind of worker role we should use to process this messages in the optimum way (Medium size).
In this post I will try to respond to another question that is normal to appear when we are using a cloud solution.
Can I scale the number of instances that consume messages from the topic?
Environment:
Each message that was added to Service Bus was pretty small. We had around 100 characters in UTF7 and 3 properties added to each BrokeredMessage.
We run the tests with one; two and three subscribers with different filter rules and the result were similar.
Each message that is received from Service Bus required a custom action to be executed. This action is pretty complicated and consumes CPU power. Also the logic requires to access remote services (that are stored in the same data-center).
For all this tests we used a medium size worker role.
Action:
In the first phase we pushed on the topic 100.000 messages and measure how long it takes to process all the available messages. We tried to identify what is the best number of instances that we can have in parallel from the perspective of costs and time.
The next phase was to see detect what is the optimum number of messages that can be processed using the given number of instances that we found in the first phase.
Results:
Phase 1: We tried to process 100.000 of messages with different number of worker roles instances. The size of all instances was Medium. The results of these tests were:
  • 1 instance – 18 minutes and 10 seconds
  • 2 instance – 11 minutes and 40 seconds
  • 3 instance – 7 minutes and 15 seconds
  • 4 instance – 4 minutes and 15 seconds
  • 5 instance – 3 minutes and 20 seconds
  • 6 instance – 2 minutes and 59 seconds
  • 7 instance – 2 minutes and 27 seconds
  • 8 instance – 2 minutes and 5 seconds
We can observe that at while scaling up, the time decrease with around 30% percent until we reach 4-5 instances, when the time start to decrease more slowly. This happens because when a topic is intensive used, the response time increase. Don’t expect to have the same latency when the topic is hit 10 times per seconds or 1000 per seconds.
When calculating the price, we should take into account to different prices. The first price represents the cost of running N instances for the period of time when messages from topic are processed. The second price is the obsolete price, which represents the cost of running N instances for the minimum period of time – in the case of Windows Azure the smallest time unit is hour.
Taking all this things into account, we observed that 4 instance of medium size can process our messages in the shortest period of time with the best costs.
Having this “magic” number of instances and the size of them we made the next step. We measure how long it takes for our application that is deployed on 4 worker roles with medium size to process different number of messages. The results of this test were:
  • 100.000 messages – 4 minutes
  • 200.000 messages – 8 minutes
  • 300.000 messages – 11 minutes
  • 500.000 messages – 19 minutes
  • 1.000.000 messages – 34 minutes
  • 3.000.000 messages – 2 hours and 32 minutes
The results were pretty interesting. The performance is also the same until we reach a critical point. Do you remember the first post when I mentioned that we observe that 1.000.000 messages per 30 minutes seems to be the maximum number of messages that we can process in an optimum way?
This is the point when the performance starts to go down. We can observe that the difference between processing 1.000.000 messages and 3.000.000 messages is pretty big. From the time perspective the last test requested almost 4.5x more time.
At the end of the tests we decided to use only one worker role of medium size to see how long it takes to process different number of messages. The time results were:
  • 100.000 messages – 18 minutes
  • 200.000 messages – 32 minutes
  • 300.000 messages – 51 minutes
  • 500.000 messages – 1 hour and 31 minutes
  • 1.000.000 messages – 2 hour and 58 minutes
What I liked at this result it was how the duration increased. The duration time increases almost like the number of messages.
Conclusion
For our business problems we observed that having 4 worker roles of medium size of the same topic is the best configuration that we can have from time and costs perspective. This result is extremely important because we know where is the point when we need to scale in a different way.
In the next post related to this topic we will talked about costs.
Remarks: Usually all the duration values are rounded to minutes (without seconds and milliseconds). This does not mean that we don’t have these values.

Part 4

Comments

Popular posts from this blog

How to check in AngularJS if a service was register or not

There are cases when you need to check in a service or a controller was register in AngularJS.
For example a valid use case is when you have the same implementation running on multiple application. In this case, you may want to intercept the HTTP provider and add a custom step there. This step don’t needs to run on all the application, only in the one where the service exist and register.
A solution for this case would be to have a flag in the configuration that specify this. In the core you would have an IF that would check the value of this flag.
Another solution is to check if a specific service was register in AngularJS or not. If the service was register that you would execute your own logic.
To check if a service was register or not in AngularJS container you need to call the ‘has’ method of ‘inhector’. It will return TRUE if the service was register.
if ($injector.has('httpInterceptorService')) { $httpProvider.interceptors.push('httpInterceptorService&#…

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Run native .NET application in Docker (.NET Framework 4.6.2)

Scope
The main scope of this post is to see how we can run a legacy application written in .NET Framework in Docker.

Context
First of all, let’s define what is a legacy application in our context. By a legacy application we understand an application that runs .NET Framework 3.5 or higher in a production environment where we don’t have any more the people or documentation that would help us to understand what is happening behind the scene.
In this scenarios, you might want to migrate the current solution from a standard environment to Docker. There are many advantages for such a migration, like:

Continuous DeploymentTestingIsolationSecurity at container levelVersioning ControlEnvironment Standardization
Until now, we didn’t had the possibility to run a .NET application in Docker. With .NET Core, there was support for .NET Core in Docker, but migration from a full .NET framework to .NET Core can be costly and even impossible. Not only because of lack of features, but also because once you…