(Part 2) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Part 1 - (Part 1) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Part 2 - (Part 2) Things that we should consider when the content stored in Azure shouldn't leave a specific country
In the previews post we talked about what is happening with the content than is tranfered from Azure Regions to your endpoints that reside outside Azure Regions.
We discover that even if Azure Regions and your endpoint are in the same country, you and Microsoft doesn't have any control on what is happening with the content from the moment when content is leaving their datacenter and arrives to your endpoint. Based on ISP (Internet Service Providers) routing rules and load, an ISP can decide to redirect your traffic to routes from another country.
In this article we will talk about different the replication fetures of different Azure Services. The main focus is to understand what replication features might not be complient with our requirments - Data and Payload to not leave the country where Azure Regions also is.
We will take a look the most common and important Azure Services.
But before going forward, we should talk a little about Azure Paired Regions.
If an Azure Region is an area from a specific geograficaly region that can contain one or more datacenters, an Azure Paired Region is another region from the same geogrically region that is used for Business Continuity and Disaster Recovery (BCDR). Basically paired region is used for replication and backups.
From my perspective, the most important thing about Azure Paired Region is that there are in the same geogrically region. For large countries like USA, China, Australia, India this means that the geogrically region will be also the same country. But for small countries these will mean that the paired region resides in another country.
For example, in Europe we have an Azure Region in Ireland and another one in Netherlands. This means that if we configure our Azure Service to make a replica in the paired region (Ireland in this case), the data wil leave Netherlands and we will not be complient with local laws for specific industry (Healthcare).
Of course, there is tradeoff all the time between cost and the risks that we want to cover. Many times, it is acceptable if one Azure Region goes down to have the system down because it means that a disaster happen or you cound invest to replicate and backup to another local suplier.
There are some industries that this might not be acceptable. For example in healtcare, if a disaster happens you need the system to be up and running because in that moment your sysem will be used by hospitals and might make a different between life and death.
If you want to find out what is the paired region for different Azure Regions you should check the Microsoft page, that is updated as soon as a change is done related to Azure Paired Regions.
Azure Storage
When we talk about Azure Storage, please take into consideration, that this service is very often used by other services also, beeing a core service. In this context you should remember that for other Azure Services, like Azure VMs that requires you to have an Azure Storage for disk and storage, you need to use the right configuration of core services also.
There are four times of storage replication supported in this moment by Azure Store:
You should be very careful when you what to use ZRS and you have country constrains. ZRS replicate your content in multiple facilities, that can be in the same Azure Region or in another Azure Region. In this context you should be aware if the other facilities where your content is replicated are in the same Azure Region or in other regions (Paired Region).
Azure SQL Database
There multiple features that support Azure SQL Database clients to not lose their data and to have a minimum Recovery Time Objective (RTO).
The first feature is Point in Time Restore. This is an automatically backup that is done to your Azure SQL Database and can be used to restore your database. These backups are stored from 7 to 35 days (depending of your tier). The backup is done in the same Azure Region as your Azure SQL Database
The Geo-Restore is similar with Point in Time Restore, but relies on Geo-redundant storage (GRS) of Azure Storage. This means that the backup of your database is replicated in another Azure Region that might be in a different country.
Active Geo-Replication is one of the most powerful replication features that are out of the box on the market. You can have up to 4 replicas in different Azure Regions that will be sync with your main database. Of course, if you have any country restrictions, we should be aware of what region we are using when we configure our replicas.
DocumentDB
If you are using this Azure Service, then you should know from the beginning that the backup is done automatically by Microsoft by a build-in service. This back-up service is using geo-redundancy backups in the paired Azure Regions. It means that if you have any kind of country constraints and you are from Europe or Brazil, then you should double check what data you are storing there.
In combination with Azure Data Factory we are allowed to create custom backups and store them in our own Azure Storage, that can be in the same Azure Region with our DocumentDB instance.
HDInsight
I putted this service on the radar, because very often people are using a Hadoop system not only for processing data, but also for storing data.
The good part with HDInsight related to backups is that all of them relies on standard Hadoop (HBase,...) 'ways' where we have full control of our backups.
The Azure Storages that are used by this kind of clusters are specified during the cluster creation flow. These storages are Locally redundant storage (LRS).
Geo-Replication is supported in this kind of clusters allowing us to have replicas in other Azure Regions. If we have country restriction, then we should be aware of Geo-Replication if the Azure Paired Region is not in the same country (similar with geo-replication for Azure SQL Databases).
Azure IoT Hub
This new service that connects the devices that are on the field with our backend becomes more and more powerful and new features are added at every new release.
From the disaster recovery perspective, Azure IoT Hub is not an out of the box solution in this moment and we will need to take some actions if we want to do a failure from one instance of Azure IoT Hub to another.
The good part is that from this perspective, if we have any country restriction we are perfect fine. All content stays in the same Azure Region as our primary instance. There are no backups or replicas of our data done in another region.
Azure Service Fabric
In this moment in time, Azure Service Fabric can run on only one Azure Region. Similar with Azure IoT Hub, there are no replicas or backups done automatically in other Azure Regions.
Both full and incremental backups that are happening inside Azure Service Fabric are done using storage from the same Azure Region as our cluster. When we are doing backups, we can specify the Azure Storage where the backups to be persisted. Because of this, in the context of country restrictions, it is our responsibility to use Azure Storages that have only ZRS activated.
SQL Data Warehouse
This new service is still in preview and allows us to store and process petabytes of data. This warehouse creates backups of your data in Azure Storage and persist them in case of a failure.
The data is stored in Local Redundant Storage (LRS), meaning that your data doesn’t leave the current Azure Region.
In one of the previews version of SQL Data Warehouse, there was support for Geo-Restore, by using RA-GRS of Azure Storage. It seems that this feature is not available anymore.
Especially for services that are in preview, we should be aware all the time of changes that are happening. Not very often this changes might affect us.
Azure Media Services
For this service the story is pretty simple. Behind the scene we are specifying the Azure Storage that we are using for "Play on Demand". In this case we are free to use any kind of Azure Storage - even LRS that is not replicated in other Azure Regions.
Azure Event Hub
All events that send to an instance of Azure Event Hub are stored only in the region where our instance was created. For this service we don't have any kind of problem related to country restrictions.
Azure Key Vault
In this moment in time there is no feature for this service that would allow us to back-up/replicate our secrets in another Azure Region. From this point of view the scenarios where people would need such a replica or backups are not very common.
There is no manual replication or backup done in another Azure Region.
Replication are done out of the box automatically in the same geograficall region. In this context we should check if it is in the same country or not.
Azure Service Bus
For Service Bus, the story is simple, like for Azure Event Hub. All messages are persisted only in the original Azure Region and there are no backups done in different regions - there is no support for a namespace to span multiple Azure Regions.
Geo-Replication can be done only manually, by having two different namespaces (endpoints) in two different regions.
Azure Redis Cache
This services allows us to create backups using Redis Persistence Model (RPM). It means that a backup can be created in Azure Storage and restored later on in time.
The Azure Storage needs to be from the same Azure Region, but we are not restricted to use only Local Redundancy Storage (LRD). This means that we need to be aware of what kind of storage we are using when we have any country restrictions for our data.
Azure Data Lake
This is the kind of service that I dream all the time. Without any kind of storage restriction, you can use as much storage you need. The perfect ocean (smile) for your data to store and analyze.
When you create an Azure Data Lake you need to specify a location, an Azure Region, where all the content will be store. Basically that is the place where your ocean of data is created.
Based on the current information, it is not clear (for me at least), if there are any kind of features for replication across other regions. Because we are talking about large amount of data, I assume that it is not intended to replicate the content across Azure Regions (and this is a good think). In this context, my assumption that Azure Data Lake it is safe to use if there are any kind of country restriction, but a check with Microsoft representative would be a good idea before jumping into the ocean.
Conclusion
There are a lot of services in Azure that offer different replication and backup mechanism in the same Azure Region or across other regions. It is important to remember that we should check each service when we have special restrictions or requirements.
This kind of requirements, like country regulation for data are not common and very often we can find other ways to be compliant with them like encryption or splitting data based on the type of the information). I will talk about other ways how we can resolve this problem in future articles.
Part 1 - (Part 1) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Part 2 - (Part 2) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Part 2 - (Part 2) Things that we should consider when the content stored in Azure shouldn't leave a specific country
In the previews post we talked about what is happening with the content than is tranfered from Azure Regions to your endpoints that reside outside Azure Regions.
We discover that even if Azure Regions and your endpoint are in the same country, you and Microsoft doesn't have any control on what is happening with the content from the moment when content is leaving their datacenter and arrives to your endpoint. Based on ISP (Internet Service Providers) routing rules and load, an ISP can decide to redirect your traffic to routes from another country.
In this article we will talk about different the replication fetures of different Azure Services. The main focus is to understand what replication features might not be complient with our requirments - Data and Payload to not leave the country where Azure Regions also is.
We will take a look the most common and important Azure Services.
But before going forward, we should talk a little about Azure Paired Regions.
If an Azure Region is an area from a specific geograficaly region that can contain one or more datacenters, an Azure Paired Region is another region from the same geogrically region that is used for Business Continuity and Disaster Recovery (BCDR). Basically paired region is used for replication and backups.
From my perspective, the most important thing about Azure Paired Region is that there are in the same geogrically region. For large countries like USA, China, Australia, India this means that the geogrically region will be also the same country. But for small countries these will mean that the paired region resides in another country.
For example, in Europe we have an Azure Region in Ireland and another one in Netherlands. This means that if we configure our Azure Service to make a replica in the paired region (Ireland in this case), the data wil leave Netherlands and we will not be complient with local laws for specific industry (Healthcare).
Of course, there is tradeoff all the time between cost and the risks that we want to cover. Many times, it is acceptable if one Azure Region goes down to have the system down because it means that a disaster happen or you cound invest to replicate and backup to another local suplier.
There are some industries that this might not be acceptable. For example in healtcare, if a disaster happens you need the system to be up and running because in that moment your sysem will be used by hospitals and might make a different between life and death.
If you want to find out what is the paired region for different Azure Regions you should check the Microsoft page, that is updated as soon as a change is done related to Azure Paired Regions.
Azure Storage
When we talk about Azure Storage, please take into consideration, that this service is very often used by other services also, beeing a core service. In this context you should remember that for other Azure Services, like Azure VMs that requires you to have an Azure Storage for disk and storage, you need to use the right configuration of core services also.
There are four times of storage replication supported in this moment by Azure Store:
- Locally redundant storage (LRS)
- Zone-redundant storage (ZRS)
- Geo-redundant storage (GRS)
- Read-access geo-redundant storage (RA-GRS)
You should be very careful when you what to use ZRS and you have country constrains. ZRS replicate your content in multiple facilities, that can be in the same Azure Region or in another Azure Region. In this context you should be aware if the other facilities where your content is replicated are in the same Azure Region or in other regions (Paired Region).
Azure SQL Database
There multiple features that support Azure SQL Database clients to not lose their data and to have a minimum Recovery Time Objective (RTO).
The first feature is Point in Time Restore. This is an automatically backup that is done to your Azure SQL Database and can be used to restore your database. These backups are stored from 7 to 35 days (depending of your tier). The backup is done in the same Azure Region as your Azure SQL Database
The Geo-Restore is similar with Point in Time Restore, but relies on Geo-redundant storage (GRS) of Azure Storage. This means that the backup of your database is replicated in another Azure Region that might be in a different country.
Active Geo-Replication is one of the most powerful replication features that are out of the box on the market. You can have up to 4 replicas in different Azure Regions that will be sync with your main database. Of course, if you have any country restrictions, we should be aware of what region we are using when we configure our replicas.
DocumentDB
If you are using this Azure Service, then you should know from the beginning that the backup is done automatically by Microsoft by a build-in service. This back-up service is using geo-redundancy backups in the paired Azure Regions. It means that if you have any kind of country constraints and you are from Europe or Brazil, then you should double check what data you are storing there.
In combination with Azure Data Factory we are allowed to create custom backups and store them in our own Azure Storage, that can be in the same Azure Region with our DocumentDB instance.
HDInsight
I putted this service on the radar, because very often people are using a Hadoop system not only for processing data, but also for storing data.
The good part with HDInsight related to backups is that all of them relies on standard Hadoop (HBase,...) 'ways' where we have full control of our backups.
The Azure Storages that are used by this kind of clusters are specified during the cluster creation flow. These storages are Locally redundant storage (LRS).
Geo-Replication is supported in this kind of clusters allowing us to have replicas in other Azure Regions. If we have country restriction, then we should be aware of Geo-Replication if the Azure Paired Region is not in the same country (similar with geo-replication for Azure SQL Databases).
Azure IoT Hub
This new service that connects the devices that are on the field with our backend becomes more and more powerful and new features are added at every new release.
From the disaster recovery perspective, Azure IoT Hub is not an out of the box solution in this moment and we will need to take some actions if we want to do a failure from one instance of Azure IoT Hub to another.
The good part is that from this perspective, if we have any country restriction we are perfect fine. All content stays in the same Azure Region as our primary instance. There are no backups or replicas of our data done in another region.
Azure Service Fabric
In this moment in time, Azure Service Fabric can run on only one Azure Region. Similar with Azure IoT Hub, there are no replicas or backups done automatically in other Azure Regions.
Both full and incremental backups that are happening inside Azure Service Fabric are done using storage from the same Azure Region as our cluster. When we are doing backups, we can specify the Azure Storage where the backups to be persisted. Because of this, in the context of country restrictions, it is our responsibility to use Azure Storages that have only ZRS activated.
private async Task<bool> BackupCallbackAsync(BackupInfo backupInfo, CancellationToken cancellationToken)
{
Guid backupId = GetBackupId();
await externalBackupStore
.UploadBackupFolderAsync(
backupInfo.Directory,
backupId,
cancellationToken);
return true;
}
SQL Data Warehouse
This new service is still in preview and allows us to store and process petabytes of data. This warehouse creates backups of your data in Azure Storage and persist them in case of a failure.
The data is stored in Local Redundant Storage (LRS), meaning that your data doesn’t leave the current Azure Region.
In one of the previews version of SQL Data Warehouse, there was support for Geo-Restore, by using RA-GRS of Azure Storage. It seems that this feature is not available anymore.
Especially for services that are in preview, we should be aware all the time of changes that are happening. Not very often this changes might affect us.
Azure Media Services
For this service the story is pretty simple. Behind the scene we are specifying the Azure Storage that we are using for "Play on Demand". In this case we are free to use any kind of Azure Storage - even LRS that is not replicated in other Azure Regions.
Azure Event Hub
All events that send to an instance of Azure Event Hub are stored only in the region where our instance was created. For this service we don't have any kind of problem related to country restrictions.
Azure Key Vault
In this moment in time there is no feature for this service that would allow us to back-up/replicate our secrets in another Azure Region. From this point of view the scenarios where people would need such a replica or backups are not very common.
There is no manual replication or backup done in another Azure Region.
Replication are done out of the box automatically in the same geograficall region. In this context we should check if it is in the same country or not.
Azure Service Bus
For Service Bus, the story is simple, like for Azure Event Hub. All messages are persisted only in the original Azure Region and there are no backups done in different regions - there is no support for a namespace to span multiple Azure Regions.
Geo-Replication can be done only manually, by having two different namespaces (endpoints) in two different regions.
Azure Redis Cache
This services allows us to create backups using Redis Persistence Model (RPM). It means that a backup can be created in Azure Storage and restored later on in time.
The Azure Storage needs to be from the same Azure Region, but we are not restricted to use only Local Redundancy Storage (LRD). This means that we need to be aware of what kind of storage we are using when we have any country restrictions for our data.
Azure Data Lake
This is the kind of service that I dream all the time. Without any kind of storage restriction, you can use as much storage you need. The perfect ocean (smile) for your data to store and analyze.
When you create an Azure Data Lake you need to specify a location, an Azure Region, where all the content will be store. Basically that is the place where your ocean of data is created.
Based on the current information, it is not clear (for me at least), if there are any kind of features for replication across other regions. Because we are talking about large amount of data, I assume that it is not intended to replicate the content across Azure Regions (and this is a good think). In this context, my assumption that Azure Data Lake it is safe to use if there are any kind of country restriction, but a check with Microsoft representative would be a good idea before jumping into the ocean.
Conclusion
There are a lot of services in Azure that offer different replication and backup mechanism in the same Azure Region or across other regions. It is important to remember that we should check each service when we have special restrictions or requirements.
This kind of requirements, like country regulation for data are not common and very often we can find other ways to be compliant with them like encryption or splitting data based on the type of the information). I will talk about other ways how we can resolve this problem in future articles.
Part 1 - (Part 1) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Part 2 - (Part 2) Things that we should consider when the content stored in Azure shouldn't leave a specific country
Comments
Post a Comment