Skip to main content

The tokens run out

By

 Last week, I was working around seven in the evening when I ran out of tokens. I was working on a system built entirely with AI. For a system build with AI, it is impossible to work on this system without AI, so when the tokens were gone, I had to stop.

It reminded me of ten or eleven years ago, when the electricity went off at the office. The computers stopped, and everyone went outside because there was not much else to do inside. Back then, our work depended on electricity. Now, more and more, it depends on AI too.

Tokens can seem like a small technical detail, just a limit you check from time to time. But when they are gone, you feel what they mean. They can be the difference between continuing your work and having to stop.

I wonder if we are starting to depend on AI in the same way. We use it to write, analyse things, find solutions, and move projects forward. Maybe, after some time, working without it will become much harder. Not because we have forgotten how to do the work, but because we have built our way of working around AI.

This makes me think about having a backup option. For internet, some people have a second connection in case the main one stops working. Maybe we should think in a similar way about AI. An on-premise model, or even a smaller and less capable model, could help us continue when the main service is unavailable or we run out of tokens. It may not do the work as well, but sometimes being able to continue is more valuable than having the best model.

 Maybe laptops or other external devices will also become part of this backup. Some can already run smaller AI models locally, but this is just the beginning of the journey. They may not be powerful enough for every task, but they could help us keep working when the main AI service is not available.

Comments

Post a Comment

Popular posts from this blog

How to audit an Azure Cosmos DB

By
In this post, we will talk about how we can audit an Azure Cosmos DB database. Before jumping into the problem let us define the business requirement: As an Administrator I want to be able to audit all changes that were done to specific collection inside my Azure Cosmos DB. The requirement is simple, but can be a little tricky to implement fully. First of all when you are using Azure Cosmos DB or any other storage solution there are 99% odds that you’ll have more than one system that writes data to it. This means that you have or not have control on the systems that are doing any create/update/delete operations. Solution 1: Diagnostic Logs Cosmos DB allows us activate diagnostics logs and stream the output a storage account for achieving to other systems like Event Hub or Log Analytics. This would allow us to have information related to who, when, what, response code and how the access operation to our Cosmos DB was done. Beside this there is a field that specifies what was th...
Post a Comment
Read more

Why Database Modernization Matters for AI

By
  When companies transition to the cloud, they typically begin with applications and virtual machines, which is often the easier part of the process. The actual complexity arises later when databases are moved. To save time and effort, cloud adoption is more of a cloud migration in an IaaS manner, fulfilling current, but not future needs. Even organisations that are already in the cloud find that their databases, although “migrated,” are not genuinely modernised. This disparity becomes particularly evident when they begin to explore AI technologies. Understanding Modernisation Beyond Migration Database modernisation is distinct from merely relocating an outdated database to Azure. It's about making your data layer ready for future needs, like automation, real-time analytics, and AI capabilities. AI needs high throughput, which can be achieved using native DB cloud capabilities. When your database runs in a traditional setup (even hosted in the cloud), in that case, you will enc...
Post a Comment
Read more

Azure Well-Architected AI workload Assessment

By
  AI is everywhere, part of the IT solutions we build and run today. Having an AI service, a good model, and data is not enough. As for cloud, the real difference is how we build, manage and run the whole solution. Microsoft created the Azure Well-Architected Framework for AI Workloads exactly for this reason — to help teams design AI systems that are reliable, secure, and cost-efficient. The assessment has six main categories that we cover in the next section. Based on the results, we can gain a good understanding of the current AI workload estate and a list of actions to improve how you run and manage your AI workloads . Designing the AI Application The first step in building your AI application is to consider how you will structure it. Using containers for tasks like data processing or model inference helps maintain consistency across the system. This approach makes it easier to update, move, and manage different components. When you have multiple steps in your workflow, such as...
Post a Comment
Read more