Skip to main content

Cloud Modernization for AI: Scalability and Performance Jams (Pill 1 of 5 / Cloud Pills)

Cloud! When we think of the cloud, our mind goes to unlimited scalability and power, using an efficient system that can automatically respond to demand. The reality is not like this, mainly because most of the systems were migrated as is to the cloud or built by teams where the cloud skills were limited. Cloud modernization is not part of the lifecycle of systems that run inside the cloud, meaning that the same cloud services are used by a system after 5-10 years. Performance bottlenecks emerge when AI is added to systems because of inefficient resource allocations, architectural decisions, and a lack of understanding of the demand of the AI model. 

The truth is that most of the systems that run in the cloud nowadays are not ready for the demand for an AI service. Microsoft, like other vendors, provides paths to navigate in the AI journey that requires customers to adopt new cloud technologies or change the way how the system works. One of the mistakes made by organizations is to underestimate the complexity of an AI system. Cloud platforms like Microsoft Azure provide scalable services, powerful VMs, and an easy way to provide more resources. However, none of this offers a system that would work with the data, bandwidth, and security demands of an AI system. At the same time, dedicated GPU VMs like Azure ND-series that are designed for deep learning do not distribute the workload efficiently, which increases the cloud spend. Customers must use AKS (Azure Kubernetes Service) or similar solutions to implement smart workload balancing. The data pipeline is a common challenge for AI. 

Organizations need to be able to streamline data ingestion and processing, but many times, they fail because the existing solution relies on an older solution. This generates a bottleneck at the data level, not allowing them to streamline the ingestion and processing of data. Tools like ADF (Azure Data Factory) and Microsoft Fabric provide orchestration capabilities for data workflows, but this requires storing the data in the same location to avoid latency. To ensure fast queries, services with hierarchical namespace support, like Azure Data Lake Storage (v2) , must be combined with AI to provide high-speed access to structured data. The risk of not using such services is that AI models will spend more time waiting for data than processing them. Peeks don’t happen to classical cloud applications. When we deploy an AI solution, peaks can also occur, requiring us to stress test our AI systems. Services like Azure Machine Learning can automatically scale up or down but require configuring an auto-scaling threshold to avoid downtime or overprovisioning. For this purpose, it is important to include monitoring solutions like Azure Monitor and Application Insights that can track the AI system performance, identify possible performance degradation, and trigger a scaling activity before impacting our business. 

Organizations expect that moving and running AI payloads in the cloud will automatically reduce costs. The reality is that the cloud costs will be higher because of inefficient scaling strategies, improper cloud services used and lack of cost control. Cost management strategies such as reserved instances for predictable AI payloads and spot VMs for non-critical jobs are crucial to balancing the cost and performance of the end-to-end system. Organizations that fail to integrate FinOps into their cloud modernization journey for AI will end up with exceeding budget consumption and a lack of cost control. 

A successful AI strategy must include cloud modernization, which covers strategy, design, monitoring, and proactive scaling of AI services and the cloud ecosystem. Cloud modernization for AI is not just about adding AI services and scaling the existing ones. It’s about making cloud applications and AI services run efficiently, scalable, and cost-efficient. This can be achieved by having a cloud modernization strategy and a strong FinOpns function.

Comments

Popular posts from this blog

How to audit an Azure Cosmos DB

In this post, we will talk about how we can audit an Azure Cosmos DB database. Before jumping into the problem let us define the business requirement: As an Administrator I want to be able to audit all changes that were done to specific collection inside my Azure Cosmos DB. The requirement is simple, but can be a little tricky to implement fully. First of all when you are using Azure Cosmos DB or any other storage solution there are 99% odds that you’ll have more than one system that writes data to it. This means that you have or not have control on the systems that are doing any create/update/delete operations. Solution 1: Diagnostic Logs Cosmos DB allows us activate diagnostics logs and stream the output a storage account for achieving to other systems like Event Hub or Log Analytics. This would allow us to have information related to who, when, what, response code and how the access operation to our Cosmos DB was done. Beside this there is a field that specifies what was th...

Can User Technical Profile Influence AI Architecture Decisions?

In the last few days, I ran a small experiment on Devin/Windsurf/Cascade, BMAD, and LLM-based architecture decisions. The initial concern was simple: if an AI coding assistant knows something about the user, for example, that the user is a Java developer or a .NET developer, can this information influence the architecture and technology stack that the AI will propose? This question is important because many companies are starting to use AI tools not only for coding but also for product discovery, architecture, documentation, and technical decision-making. If the AI is influenced by the developer's personal profile, the result may not be fully neutral. It may look like a business or architecture decision, but in reality, it may be partially shaped by the context of the person using the tool. The test was done using BMAD with Devin/Windsurf/Cascade. The scenario was a unified field service operations platform. The task was to let BMAD refine the product, define the architecture and s...

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine: threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration: TeamCity .NET 4.51 EF 6.0.2 VS2013 It see...