Skip to main content

Managing cloud training environments inside Azure and AWS


Providing learning and training cloud environments where engineers can play around it’s not so cheap as you might think. One of the most expensive resources are the computation ones likes VMs that are used not only to play with the cloud itself but also to host training resources like a specific CMS.
In general, the training environments are not required to run 24/7. In most of the cases they are used only during the working hours 8/5. Additional to this it is not required for them to run the full working hours. They can be spin-up only when there are persons that want to use them, otherwise, there is no need to run them.
By reducing the running of VMs from 24/7 to 8/5 you automatically save around 60% of VMs cost during the year. For example, for a training environment that initially was estimated to around $4000/year, the cost was reduced to $2500 by only reducing the running time from 24/7 to 8/5.

Startup/Shutdown automatically

Microsoft Azure - Runbooks can be used inside Azure to do automation. You can define a runbook that starts a VMs under a resource group automatically at a specific hour and shut down in the evening. If you can find more check the official Azure documentation - https://docs.microsoft.com/en-us/azure/automation/automation-solution-vm-management

AWS – The official solution provides by Amazon it is based on AWS Functions, Amazon DynamoDB and CloudWatch. Inside AWS DynamoDB the schedule is stored and the action of startup and shut down of VMs is done by Azure Function that it is triggered by AWS CloudWatch. The CloudFormation template to implement this solution can be found here - https://aws.amazon.com/solutions/instance-scheduler/

Start Manually / Shutdown automatically
In the above solutions, the biggest problem is that your VMs are not used all the time. Because of this, you might have days when the VMs are available, but nobody is using them. For these situations, you can define using the automation solution presented above only the shutdown implementation. The start of the VMs is done manually by each user when he needs the resources.
You might want to have some VMs start automatically in the morning and the rest of them would start only when needed.

Dev/Test Labs Environments
When things are becoming more complicated, and you need to do define access policies, have higher control of resources and want kind of actions can be done by each user you can start to use solutions like Azure DevTest  Labs that enables developers to manage their VMs by themselves. At the same time at company level, you can define policies related to what kind of VMs can be spin-up, the no. of them, the time when all resources are shut down in the afternoon and many more.

Conclusion
I prefer the option no. 1 or 2 where you have automatical systems that can spin-up or shut-down the computation resources. There is enough flexibility to let the engineers defining their schedule. At the same time, you have the policy layer on top of resources that enable you to control the costs.
When you are in these situations take into account the following aspects:
  1. -          Maximum no. of VMs that can be created
  2. -          Who has the rights to create new VMs
  3. -          Who has the rights to start a VMs
  4. -          Who can change the tier side of the VMs
  5. -          Is a shut-down automatic mechanism in place?
  6. -          Who can configure the shut-down mechanism?


Comments

Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Entity Framework (EF) TransactionScope vs Database.BeginTransaction

In today blog post we will talk a little about a new feature that is available on EF6+ related to Transactions.
Until now, when we had to use transaction we used ‘TransactionScope’. It works great and I would say that is something that is now in our blood.
using (var scope = new TransactionScope(TransactionScopeOption.Required)) { using (SqlConnection conn = new SqlConnection("...")) { conn.Open(); SqlCommand sqlCommand = new SqlCommand(); sqlCommand.Connection = conn; sqlCommand.CommandText = ... sqlCommand.ExecuteNonQuery(); ... } scope.Complete(); } Starting with EF6.0 we have a new way to work with transactions. The new approach is based on Database.BeginTransaction(), Database.Rollback(), Database.Commit(). Yes, no more TransactionScope.
In the followi…

GET call of REST API that contains '/'-slash character in the value of a parameter

Let’s assume that we have the following scenario: I have a public HTTP endpoint and I need to post some content using GET command. One of the parameters contains special characters like “\” and “/”. If the endpoint is an ApiController than you may have problems if you encode the parameter using the http encoder.
using (var httpClient = new HttpClient()) { httpClient.BaseAddress = baseUrl; Task<HttpResponseMessage> response = httpClient.GetAsync(string.Format("api/foo/{0}", "qwert/qwerqwer"))); response.Wait(); response.Result.EnsureSuccessStatusCode(); } One possible solution would be to encode the query parameter using UrlTokenEncode method of HttpServerUtility class and GetBytes method ofUTF8. In this way you would get the array of bytes of the parameter and encode them as a url token.
The following code show to you how you could write the encode and decode methods.
publ…