Skip to main content

Address data processing outside the data sovereignty region (GDPR)

 The high level of cloud adoption brings closer and closer to us the day to day problems that were easily solved using on-premises solutions. The data regulations related to data sovereignty and specific region and country regulations like GDPR add an extra complexity layer to the application we are building inside the cloud.

We have an imaginary organization called Osotnoc that is active in Australia, the US, UK and the EU. Each country has strict data regulations that require Osotnoc to store customers data inside each region. 

Microsoft Azure has a strong presence in each region, so the team is building a solution on Azure SQL and App Service. The solution is deployed in all 4 regions, fully compliant with all local requirements but pretty expensive. Except for the database and computation layer, 4 different instances of Azure Application Gateways are deployed with WAF on top of them. 

Besides this, more than 30% of active users requires access data from other regions. The current solution requires them to specify the tenant/region they want to access during the login step. To have the ability to compare data across regions, they need to log into two separate browsers. It's not a nice UX.

But what if we could make the solution more simple? The local data requirements are mainly referring to the data storage layer and not to the computation. Meaning that nobody stops us from getting content from repositories that are from different regions.

One way to do this is to use the sharding capability of Azure SQL Database and the Shard Map Manager. This would enable the application layer to have the ability to manage from which shard (DB) to connect. It allows users to run the same query across multiple regions (shards) and display them on the UI.

The solution works pretty well, but some logic needs to be handled at the application layer. It is not complicated, but an extra effort is required to support this. The good thing is that we have support for .NET and Java, the ShartMapManager, part of the Elastic Database package that can be used with success for this purpose. 

There is an impact on latency when you need to load content from databases that are in a different region, but you can provide a better experience for users in this way. It's a good tradeof that you can do.

Depending on the regulations and the user patterns, you could even add a cache layer in each region for data that is often accessed. A cache is not seen as long term storage, and you are allowed to use it in specific situations, depending on the content type and the level of encryption that you use. The downside with a cache solution is that they are one of the weakest parts of a system. The security layer exists but is not their main straight. In the end in most of the cases a CDN or a remote cache it is used for non-sensitive content.

In both cases (CDN or cache) you need to have content full encrypted, the encryption key stored in EU and clear procedures related to how you manage data and what kind of actions you take.

What you need to take into account from data compliance and GDPR point of view:

  • Processing EU citizen data outside the EU region is allowed as long as the encryption key is stored in EU. Special conditions are required to be allowed to do something like this
  • A encrypted cache outside EU can be used, but content needs to be encrypted and additional technical and organizational measures needs to be taken. 


Comments

Popular posts from this blog

Windows Docker Containers can make WIN32 API calls, use COM and ASP.NET WebForms

After the last post , I received two interesting questions related to Docker and Windows. People were interested if we do Win32 API calls from a Docker container and if there is support for COM. WIN32 Support To test calls to WIN32 API, let’s try to populate SYSTEM_INFO class. [StructLayout(LayoutKind.Sequential)] public struct SYSTEM_INFO { public uint dwOemId; public uint dwPageSize; public uint lpMinimumApplicationAddress; public uint lpMaximumApplicationAddress; public uint dwActiveProcessorMask; public uint dwNumberOfProcessors; public uint dwProcessorType; public uint dwAllocationGranularity; public uint dwProcessorLevel; public uint dwProcessorRevision; } ... [DllImport("kernel32")] static extern void GetSystemInfo(ref SYSTEM_INFO pSI); ... SYSTEM_INFO pSI = new SYSTEM_INFO(...

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine: threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration: TeamCity .NET 4.51 EF 6.0.2 VS2013 It see...

Navigating Cloud Strategy after Azure Central US Region Outage

 Looking back, July 19, 2024, was challenging for customers using Microsoft Azure or Windows machines. Two major outages affected customers using CrowdStrike Falcon or Microsoft Azure computation resources in the Central US. These two outages affected many people and put many businesses on pause for a few hours or even days. The overlap of these two issues was a nightmare for travellers. In addition to blue screens in the airport terminals, they could not get additional information from the airport website, airline personnel, or the support line because they were affected by the outage in the Central US region or the CrowdStrike outage.   But what happened in reality? A faulty CrowdStrike update affected Windows computers globally, from airports and healthcare to small businesses, affecting over 8.5m computers. Even if the Falson Sensor software defect was identified and a fix deployed shortly after, the recovery took longer. In parallel with CrowdStrike, Microsoft provi...