Cloud as a Story - Vunvulea Radu

In one of the current project we need to support jobs on SQL Azure. If you already tried to create a job in SQL Azure maybe you noticed that there is no support for that. I mean that the job concept don’t exist on SQL Azure – yet, there are some rumors that in the near future we will have something like this. Our client needs this functionality now and we cannot wait one more week. Because of this we had to find a solution to make our client happy. On the store of Windows Azure we can find an application called Scheduler that can help us to implement something like this (CRON jobs). You can find more about this app on the following link:  http://vunvulearadu.blogspot.ro/2013/02/cron-job-in-windows-azure-scheduler.html In our case we cannot use this application because we have custom security configuration on the Azure machines. Because of this we cannot expose an endpoint to a 3rd part. The solution that we end up is a temporary solution that will be used until we will be able to ru

These days I participated at ITCamp 2013 . This year I had the opportunity to participate at a premium event with almost 400 attendees. For Romania, this is big number, it is hard to gather 400 people at a premium conference. What we found this year? COOL stuff. We discover what is the future of tablets, what is happening on the cloud and the voodoo magic that is happening inside the team management... and so on. I really enjoyed the conference, the sessions and especially the location – you could see all the city from above. Each year one of the most important thing at ITCamp is socialization – meeting people from different parts of the world, discover what they are doing and what are the trends of the IT industry. This year I had the opportunity to be invited as a speaker at ITCamp. Last year I talked about background task on Windows 8, but this year I decided to go on the cloud and talk about messaging pattern and how we can implement those patterns using Windows Azure Service

Today I will write a short post about JavaScript and unit testing. I heard a lot of .NET developers that they didn’t wrote code for JavaScript because is not supported by Visual Studio, is complicated to run it on the build machine or they don’t have time. Guys, I have news for you, Visual Studio 2012 supports unit tests for JavaScript, even Visual Studio 2010. You can run them almost like a normal unit test for C# without needing to install anything. The JavaScript unit tests are so smart that are integrated in a way that you don’t need to change/install anything on your build machine – you even receive the standard message notification when a unit test fail. You don’t have time for them – I will not comment this, definition of DONE is wrong for those developers. When I need to write unit tests for JavaScript code I usually prefer qunit. Why? Because in combination with a small NuGet package called NQunit you can make magic. qunit give you the possibility to write and run Java

This was one of the greatest weekends of this year. Me and another 60 developers from Cluj-Napoca had the opportunity to participate to a master class about asynchronous programming. The master class was organized by Codecamp and took place for two days and was 100% free for all the attendees (a BIG THANK YOU for iQuest – they were the sponsors of this event). At this event we had the best trainers ever - Andrew Clymerand & Richard Blewett . This was the first master class, when in both days the training room was full 110%. At the end of the master class (Sunday afternoon) all the attendees had learn new stuff. I hope that they are prepared now for the asynchronous world. Special thanks for Andrew and Richard. They flew special from UK to Cluj-Napoca for this master class.

Part 1 – What is the maximum number of messages that can be processed in 30 minutes? Part 2 – What is the optimum size of worker roles that process messages from Service Bus? Part 3 – What is the maximum number of worker role that can process messages from Service Bus? Part 4 – What are cost of Service Bus when we need to process millions of messages every hour? Part 5 – What are the scalabilities points of Service Bus? Part 6 - Topic isolation

We all heard about trends. We have trends in music, fashion and of course in IT. For 2013 there were announced some trends that are already part of our lives. Who didn’t hear about cloud, machine to machine (M2M) or NoSQL. All these trends have entered in our lives as part of our ordinary day. Big Data is a trend that existed last year and it remained one of the strongest one. In the next part of this article I would like to talk about Hadoop. Why? Big Data doesn’t exist without Hadoop. Big Data could be an amount of bytes that the client wouldn’t even know how to process them. Clients started a long time ago to ask for a scalable way of processing data. On ordinary systems, processing 50T would be a problem. Computer systems for indexing and processing this amount of data are extremely costly, not just financial but also in terms of time. At this point Hadoop is one (maybe the best) of the best processing solution for a big amount of data. First of all let’s see how it appeared and

NoSQL – one of 2013’s trends. If three or four years ago we rarely heard about a project to use NoSQL, nowadays the number of projects using non-relational databases is extremely high. In this article we will see the advantages In NoSQL taxonomy, a document is and challenges we could have when we use seen as a record from relational databases NoSQL. In the second part of the article we will analyze and emphasize several nonrelational solutions and their benefits. What is NoSQL? The easiest definition would be: NoSQL is a database that doesn’t respect the rules of a non-relational database (DBMS). A non-relational database is not based on a relational model. Data is not groups in tables; therefore there is no mathematical relationship between them. These databases are built in order to run on a large cluster. Data from such storage does not have a predefined schema. For this reasons, any new field can be added without any problem. NoSQL has appeared and developed around web applica

The new version of Windows Azure Service Bus came with a lot of new feature. One of the new feature that was added in Windows Azure SDK 2.0 is the support of message browsing. What is message browsing? This is a functionality that gives the ability to a client to iterate and access messages from a Service Bus Queue without locking or removing a message from the actual queue. There are pretty interesting thing that we can do using this feature. For example we can peek a message from a specific index in the queue: BrokenMessage message = queueClient.Peek(3); Or we can peek the first message that is available: BrokenMessage message = queueClient.Peek(); Another option is to peek a specific number of message from the queue: IEnumerable<BrokenMessage> messages = queueClient.PeekBatch(20); The PeekBatch support also two parameters, where the first one is used to specify the starting index (similar with Peek(3)): IEnumerable<BrokenMessage> messages = queueClient.PeekBatch

This week the 11th number of Today Software Magazine was lunched. In this number of the magazine I wrote about how Hadoop manage to store hundreds of terabytes without any kind of problem. I tried to explain what the secret that make Hadoop a system that supports more than 40.000 nodes. The article in Romanian can be found here:  www.todaysoftmag.com/tsm/ro/11 I hope until next week to have the article in English also. Until than I will share the slides from the presentation. HDFS and Hadoop from Radu Vunvulea Video:

What is the problem of the following code? Task.Factory.StartNew(() => { while (true) { Console.WriteLine("@ " + DateTime.Now.ToString()); Thread.Sleep(TimeSpan.FromSeconds(2)); } }); var task = Task.Factory.StartNew(() => { while (true) { Console.WriteLine("! " + DateTime.Now.ToString()); Thread.Sleep(TimeSpan.FromSeconds(4)); } }); Task.WaitAll(task); The main problem is related to Thread.Sleep. Using this method in combination with tasks we can gen odd behavior and in the end the performance of the tasks can be affected. One of the behavior that we could get is related to how the sleep period – that will not be the same as we expect ("random" sleep interval). This occurs because of the TPL behavior, which will not guarantee that each task will have a dedicated thread where he can run. What we should do to resolve this problem? We have two possible solutio

In the last series of post about testing the limits of Windows Azure Service Bus we saw what is the best configuration for our problem, the best configuration is to have 4 medium instances. The next question for us is: How we can decrease the time processing? In our case we saw that from the cost and benefits perspective we reached the limitation on Service Bus. This is around 4 medium instances that are able to process around  1.000.000 messages in 34 minutes. But what we can do if we have 10.000.000 messages that needs to be processed in 1 hour. We could increase the number of instances, but this will not improve our performance too much. In this moment we use only one topic. This topic, as Service Bus, has its own limitations – we cannot make an undefined number of request per second and expect to have a low latency. A solution to decrease the number of request is to use batches – we already use them. Another solution is to scale the topics. In this moment we have only one top

Did you ever need or want to disable the send functionality from a queue or from a topic? All the people start to send messages to you even if you say STOP. How nice would be to be able to say STOP. We have good news for people that use Windows Azure Service Bus (Queue, Topics and Subscriptions). With the new version of SDK (2.0), we can control the operation that can be made on Service Bus. How nice is that. People would ask – why we would want to do something like this. Well, there are times, especially when we make tests, when we want to control this. I see this value in two situations: When we want to make performance tests, to see how many messages we can consume from a queue in a specific time interval When there a lot of messages in the queue or topic and we don’t want to accept new messages anymore. This feature can be controlled from the description of the queue, topic or subscription.  Based on what kind of service we use from Service Bus we can have different status. It

The new version of Windows Azure SDK was launched. I’m glad to see that new feature of Service Bus are available. For me, one of the most important features that was added to Service Bus is the one that give us the possibility to consume messages from a topic or queue using an event-driven model – “pump model”. Until now, we had to have an infinite loop which check if a new message is available. This simple solution was good, but it was very primitive. In a real production application, a new message would trigger an event that would execute a specific code. To be able to support something like this, the user would need to create an infinite loop, check if a new message is available and trigger a specific event. This infinite pooling creates a lot of problems, especially when something would go wrong – detecting an error and managing it. while (true) { try { if (isNotStopped) { break; } BrokeredMessage message = null; message = queueClient.Receive(); if (mesage == n