The high level of cloud adoption brings closer and closer to us the day to day problems that were easily solved using on-premises solutions. The data regulations related to data sovereignty and specific region and country regulations like GDPR add an extra complexity layer to the application we are building inside the cloud.
We have an imaginary organization called Osotnoc that is active in Australia, the US, UK and the EU. Each country has strict data regulations that require Osotnoc to store customers data inside each region.
Microsoft Azure has a strong presence in each region, so the team is building a solution on Azure SQL and App Service. The solution is deployed in all 4 regions, fully compliant with all local requirements but pretty expensive. Except for the database and computation layer, 4 different instances of Azure Application Gateways are deployed with WAF on top of them.
Besides this, more than 30% of active users requires access data from other regions. The current solution requires them to specify the tenant/region they want to access during the login step. To have the ability to compare data across regions, they need to log into two separate browsers. It's not a nice UX.
But what if we could make the solution more simple? The local data requirements are mainly referring to the data storage layer and not to the computation. Meaning that nobody stops us from getting content from repositories that are from different regions.
One way to do this is to use the sharding capability of Azure SQL Database and the Shard Map Manager. This would enable the application layer to have the ability to manage from which shard (DB) to connect. It allows users to run the same query across multiple regions (shards) and display them on the UI.
The solution works pretty well, but some logic needs to be handled at the application layer. It is not complicated, but an extra effort is required to support this. The good thing is that we have support for .NET and Java, the ShartMapManager, part of the Elastic Database package that can be used with success for this purpose.
There is an impact on latency when you need to load content from databases that are in a different region, but you can provide a better experience for users in this way. It's a good tradeof that you can do.
Depending on the regulations and the user patterns, you could even add a cache layer in each region for data that is often accessed. A cache is not seen as long term storage, and you are allowed to use it in specific situations, depending on the content type and the level of encryption that you use. The downside with a cache solution is that they are one of the weakest parts of a system. The security layer exists but is not their main straight. In the end in most of the cases a CDN or a remote cache it is used for non-sensitive content.
In both cases (CDN or cache) you need to have content full encrypted, the encryption key stored in EU and clear procedures related to how you manage data and what kind of actions you take.
What you need to take into account from data compliance and GDPR point of view:
- Processing EU citizen data outside the EU region is allowed as long as the encryption key is stored in EU. Special conditions are required to be allowed to do something like this
- A encrypted cache outside EU can be used, but content needs to be encrypted and additional technical and organizational measures needs to be taken.
Comments
Post a Comment