Skip to main content

Design a state machine mechanism using Windows Azure Service Bus (part 1)

In this post we try to see a possible solution to implement a simple state machine (with a simple workflow) over Windows Azure Service Bus.
Let’s imagine a system that needs to process over 1 million requests per day. Each request that enter in the system will pass through different statutes until the process will be finished. If we have simple rules, it will be very easily for us to implement a system that will process each request and change the state of our request.
This can be imagined as a simple workflow, where a request flows from one state to another. At each step, we are making executing a small action. The big question here is how we can design a solution in a way where we will don’t have any kind of scalability problems.
The biggest problem that can appear at this step is related to scalability and distribution the logic over machines. This can become a nightmare when death-locks appears or when the system will reach his limits. The costs of hardware’s that will help us to survey another day can be enormous. 
We will see a simple solution that uses Windows Azure Service Bus to resolve our problem. In the next post we see how easily we can scale this solution.
Let’s assume that each request will go through 4 states before it will process completely. Each action that need to be done implicate an external system or an internal system that was already implemented.
For this case we can use Windows Azure Service Bus with success to implement a mechanism that will send a request from one state to another. In this way we will have reliability on our site. Not only this, we will not have any kind of problem with losing the requests that has a specific state if something happens with the system.
In this first solution, we will use Windows Azure Service Bus Topic. As we already know, Windows Azure Service Bus Topic permit give us the possibility to send the same message to more than one listener. From some perspective it is very similar to message broadcast. Each listener need to listen the topic through a subscription.
What is very useful at this is what we can with a subscription. We have the possibility to have not only more than one client to our subscription, but we can define custom rules. Using these rules we will be able to define what type of messages will be accepted by our subscription.
If a subscription will accept only messages that has a property with a specific value, than we will be able to process only messages with a specific state by our subscription. After the message is processes, we can change create a new message with the new state and send it back to the Windows Azure Service Bus Topic.
The system would look something like this:

As we can see, we use the same topic to add messages that are in a new state. Each subscription will process messages on the given state. In this current implementation, we are using Windows Azure Service Bus Topics because we don’t want to use a different place to store each message that has a different state. In the same time, for each subscriber we can have more than one listener. For the states where we know that computing power is more expensive, than we can allocate more listeners.
In the next post we will see how we can make this system more scalable using Windows Azure Service Bus features.

Comments

Post a Comment

Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Entity Framework (EF) TransactionScope vs Database.BeginTransaction

In today blog post we will talk a little about a new feature that is available on EF6+ related to Transactions.
Until now, when we had to use transaction we used ‘TransactionScope’. It works great and I would say that is something that is now in our blood.
using (var scope = new TransactionScope(TransactionScopeOption.Required)) { using (SqlConnection conn = new SqlConnection("...")) { conn.Open(); SqlCommand sqlCommand = new SqlCommand(); sqlCommand.Connection = conn; sqlCommand.CommandText = ... sqlCommand.ExecuteNonQuery(); ... } scope.Complete(); } Starting with EF6.0 we have a new way to work with transactions. The new approach is based on Database.BeginTransaction(), Database.Rollback(), Database.Commit(). Yes, no more TransactionScope.
In the followi…

GET call of REST API that contains '/'-slash character in the value of a parameter

Let’s assume that we have the following scenario: I have a public HTTP endpoint and I need to post some content using GET command. One of the parameters contains special characters like “\” and “/”. If the endpoint is an ApiController than you may have problems if you encode the parameter using the http encoder.
using (var httpClient = new HttpClient()) { httpClient.BaseAddress = baseUrl; Task<HttpResponseMessage> response = httpClient.GetAsync(string.Format("api/foo/{0}", "qwert/qwerqwer"))); response.Wait(); response.Result.EnsureSuccessStatusCode(); } One possible solution would be to encode the query parameter using UrlTokenEncode method of HttpServerUtility class and GetBytes method ofUTF8. In this way you would get the array of bytes of the parameter and encode them as a url token.
The following code show to you how you could write the encode and decode methods.
publ…