In this post, we talk
about how we can migrate an Azure Cosmos DB database from one tenant to another.
Why?
There are cases
when you need to migrate your database, especially when your systems evolve. In
this situation moving the database from one Azure Subscription to another one,
it is not enough.
Context
The migration was
required because of the business context and the way how the product evolved. When you need to do such a migration,
there a few questions that you need to ask yourself.
1.
What
kind of Azure Cosmos DB API are you using? We used SQL API
2.
What
is the maintenance window? We had 24 hours for the entire system, with a 3-hour
window for Azure Cosmos DB.
3.
What
is the database size? 20GB
4.
What
is the index size? 2.2GB
5.
What are
the average request and average throughput per second? The throughput is around
200 RU/s with around 25 requests/s
Current options
At this moment in
time, Azure Cosmos DB does not have a self-service feature that allows us to
create and restore backups of our databases. There is a feature request on
Azure Feedback for it, but we still need to wait for it.
[1] Azure Ticket –
Backup & Restore
This involves opening a ticket to Azure Support Team to create a backup. Once the backup was created, another ticket should
be open to restoring the backup of the database. This approach should work without
any incidents as long as you don't have a fixed time window. There are
no SLA related to how fast the backup and restoration is executed by the support team - we don’t
have the insurance that the two actions are done in 3 hours time window.
You need to keep
in mind that backups are done automatically every 4 hours, so in theory, you
should create the restore ticket only – as long as you can keep the database out
of changes for 4 hours before opening the ticket.
If you want to
find more about it you can check the following two resources:
- - https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-backup-and-restore
- - https://docs.microsoft.com/en-us/azure/cosmos-db/online-backup-and-restore
[2] Data migration
tool
The second option
that we have available is based on the Data Migration tool that is available
from the Azure team. It enables us to migrate content from one Azure Cosmos DB
container to another one or from other resources like JSON, Hbase, AWS DynamoDB
and so on.
There are two
different interfaces that can be used. One is a CLI interface, great when we
want to do automation and the other one has a friendly UI interface, similar to
the one from SQL Import/Export. The tool is well documented, but there were questions
marks related to how fast the import can be done and what happens in the case an
error occurs.
The tool has the ability to specify the no. of retries
in case of a failure. This combine with the option to update (or not) existing
documents, would enable to restore an import job. When you use this tool, it is
necessary to overprovision the Azure Cosmos DB to ensure that there are enough
resources that can manage the import.
During an import
test, the full migration took around 2.5 hours. Even so, there were some
concerns from the team that wanted to see what other options do they have.
[3] Restore + Change Feed
This hybrid approach between using a backup of Azure Cosmos
DB by open a ticket to the support team combined with change feed works well
for cases when the update window is small and the number of changes that you
have on the database is not very high (under 20% of the documents).
The steps that you should do are the following:
(1)
Enable
a Change Feed to your Azure Cosmos DB database, that listen to all changes that
are happening to your documents and stream it to Event Hub or Blob Storage. For
these situations, Azure Event Hub would work much better.
(2)
The first step that needs to be done 3 or 4 days
before the migration is to create a ticket to and request a backup of Azure
Cosmos DB to be restored on a specific tenant.
(3)
Day 0 - Migration:
Current Azure Cosmos DB database goes in read-only mode.
(4)
Define and run an Azure Function that consumes
the change feed stored inside Event Hub and push the changes to the new the instance of Azure Cosmos DB (be careful with parallel processing of events and
data consistency - you need to take into account that you can run in parallel the functions for multiple partitions on a container). You can consume the feed starting from the time when the backup was done.
(5)
Once the Azure Function finishes the consumption
of all the events, you have the database fully restored.
Conclusion
Taking into account the context (20GB or data and 3 hours
time windows), the best options are no. 2 and 3 - Data migration tool OR Restore + Change
Feed. From the cost point of view, I would recommend going with option no. 2
because it is the safest one. But I would run the migration tool from an Azure VM that it is in the same Azure Regions as the destination or source database instance.
Option no. 3 can introduce bugs inside the system because
the function needs to be implemented. At the same time for bigger databases,
option no. 3 is the best one, as long as we have a database where changes don’t
occur to often.
Comments
Post a Comment