Skip to main content

In-memory data protection (encryption)

In this post we'll talk about memory encryption and why we need to care about it.

With EU regulations, IT solutions needs to secure data content about citizens End to End (E2E). This means that we need to offer not only a secure storage, a secure connection between different nodes, encryption at payload level, but also we need to offer a secure mechanism to store sensitive information in memory.

You might say that this is not important if the servers where data is processed is secured. Yes in an ideal world. There is no bug free software.
This means that M&S (Maintenance and Support) team that has direct access to that server might need to do a memory dump. The memory dump shall be encrypted, shared only with a limited number of people and many more. They could even share the memory dump with external 3rd parties that offer support for other systems or software that are used on that machine.
For this we need to ensure that data privacy is respected.

If the server is not configured properly, the system might create automatically a memory dump when an error occurred and upload it to different providers. On top of this, M&S teams usually prefer to configure the production system in a such a way that a memory dump is created automatically when a crush is detected.

How to encrypt data in memory
There are different libraries that can help s to achieve this goal, one of them is coming with .NET library - Memory Protection. If give us the possibility to encrypt an array of bytes that is stored in memory.
ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);
ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
As we can see, the encryption is very easy to use. The output of encrypted content is an array that is stored in memory. This array will be encrypted/decrypted. Having an array, it's allowing us to protect any kind of data.

When you are working with encryption, we don't need to forget about block size. As other encryption mechanism this library is using block size of 16 bytes for encryption. It means that if we encrypt a string or other type of data we need to ensure that the byte array block size is multiple of 16.
Unfortunately there is no out of the box support for this.
Below you can find also an extension method for string that add padding with default character. Of course this is not perfect solution, a padding standard like PKCS7 shall be used. When content is unencrypted, the default character used for padding is removed.
public static class StringPaddingExtensions
    public static byte[] ToByteArrayWithPadding(this String str)
        const int BlockingSize = 16;
        int byteLength = ((str.Length / BlockingSize) + 1) * BlockingSize;
        byte[] toEncrypt = new byte[byteLength];
        ASCII.GetBytes(str).CopyTo(toEncrypt, 0);
        return toEncrypt;

    public static string RemovePadding(this String str)
        char paddingChar = '\0';
        int indexOfFirstPadding = str.IndexOf(paddingChar);
        string cleanString = str.Remove(indexOfFirstPadding);
        return cleanString;
 string contentToEncrypt = "Hello Word!";

 byte[] toEncrypt = contentToEncrypt.ToByteArrayWithPadding();
 ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);

 ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
 string decryptedContent = ASCII.GetString(toEncrypt).RemovePadding();
I added on GitHub a full sample, with a unit tests that shows how to encrypt/decrypt content:

It is not bulletproof
Even if the content is encrypted while it is in memory, this doesn't means that we can do anything with it. There are moment during transition and processing when content will be in clear text. In this moment, a memory dump could catch easily data in clear.
Protected memory encrypts data that is stored in memory, but it doesn't allow us to process it. To be able to process it or do any kind of transformation we will need it in clear text.
The below diagrams shows when data is in a unprotected state.
As we can see, Protected Memory offer us protection only at one step. For the other steps we need to ensure that content is protected using other mechanism. With RED are marked steps when data is in memory and a memory dump could catch data in clear text.
In top of this, we need to ensure that GC clears that memory locations where data is stored in clear text.

Why there is all time a risk
Unfortunately, most of the times we need to process data that is often strings. Strings are one of the weakest data types, especially in .NET and Java. Using pinning you could have data that is saved all the time, but this is more common in C++ then .NET. Wring in this way for .NET will be a challenge and testing a nightmare.

Data protection is not only a complex discussion, but also cannot be bulletproof. Because of this we need to find a balance between the level of protection we need and how much we want to invest in it. In the end it is a risk mitigation.


Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

Entity Framework (EF) TransactionScope vs Database.BeginTransaction

In today blog post we will talk a little about a new feature that is available on EF6+ related to Transactions.
Until now, when we had to use transaction we used ‘TransactionScope’. It works great and I would say that is something that is now in our blood.
using (var scope = new TransactionScope(TransactionScopeOption.Required)) { using (SqlConnection conn = new SqlConnection("...")) { conn.Open(); SqlCommand sqlCommand = new SqlCommand(); sqlCommand.Connection = conn; sqlCommand.CommandText = ... sqlCommand.ExecuteNonQuery(); ... } scope.Complete(); } Starting with EF6.0 we have a new way to work with transactions. The new approach is based on Database.BeginTransaction(), Database.Rollback(), Database.Commit(). Yes, no more TransactionScope.
In the followi…

GET call of REST API that contains '/'-slash character in the value of a parameter

Let’s assume that we have the following scenario: I have a public HTTP endpoint and I need to post some content using GET command. One of the parameters contains special characters like “\” and “/”. If the endpoint is an ApiController than you may have problems if you encode the parameter using the http encoder.
using (var httpClient = new HttpClient()) { httpClient.BaseAddress = baseUrl; Task<HttpResponseMessage> response = httpClient.GetAsync(string.Format("api/foo/{0}", "qwert/qwerqwer"))); response.Wait(); response.Result.EnsureSuccessStatusCode(); } One possible solution would be to encode the query parameter using UrlTokenEncode method of HttpServerUtility class and GetBytes method ofUTF8. In this way you would get the array of bytes of the parameter and encode them as a url token.
The following code show to you how you could write the encode and decode methods.