Why AI ROI is more volatile than classical IT projects

 Traditional IT solutions typically remain stable for years, requiring only occasional patches, scaling, or feature additions. In contrast, AI systems operate in a changing environment, so ROI must be monitored and recalculated regularly.

Here are the three main reasons for the volatility of AI in Azure and other cloud projects:

• Models are frequently replaced or retired. In the Microsoft and Azure ecosystem, model families evolve rapidly. A model used last year may become unavailable or inferior. Even without application changes, costs, speed, and quality can fluctuate.
• Quality can change over time. New documents in SharePoint, evolving policies, and new user questions can impact performance. Without regular updates to your RAG pipeline, prompts, and evaluation sets, accuracy and trust decline, reducing system adoption.
• Costs can increase with usage. Token cost is only one factor; vector search, storage, observability, and human review become crucial as adoption grows. What appears cost-effective in a proof of concept can become expensive at scale.

Imagine you build a Policy Copilot for employees in Microsoft Teams, using Azure OpenAI Service and Azure AI Search over SharePoint that plays the role of RAG. In the first month, it works well. People ask questions like “Can I expense a taxi?” or “What’s the travel policy?” and get correct answers with citations.

However, over time, several changes occur:

1. Your policy documents change, like new HR rules or finance thresholds. If SharePoint is updated but your indexing or chunking isn’t adjusted, the system starts missing the right sections. Accuracy can drop from 85% to 70%. Users lose trust, and the value drops faster than the costs.
2. A model version change. Your initial proof-of-concept might use one model, but later you switch to a newer version for better results or faster responses. However, prompts may behave differently, providing longer, more cautious, or less accurate answers. Testing and updating prompts can be costly. As usage increases, costs increase. With 200 daily users, expenses are minimal, but at 5,000 daily users, new issues appear as search queries increase significantly during office hours, stronger guardrails are required, and a process for reviewing sensitive topics must be established.

ROI remains achievable, but only if the system is managed as an ongoing product rather than a one-time project.

To sustain ROI through changes, incorporate these three items:

• Lifecycle plan: Plan to review model selection quarterly, and ensure model changes are managed in a controlled manner.
• Evaluation: Keep a set of key test questions, check answer quality every week, and connect it to how much people use the
• Cost guardrails: Monitor per-conversation costs, apply rate limits and caching, and optimize retrieval before addressing token costs.

The value is real, but the system is constantly evolving. With effective lifecycle management, ROI can be measured and sustained. Without it, ROI becomes unpredictable.