Skip to main content

In-memory data protection (encryption)

In this post we'll talk about memory encryption and why we need to care about it.

Context
With EU regulations, IT solutions needs to secure data content about citizens End to End (E2E). This means that we need to offer not only a secure storage, a secure connection between different nodes, encryption at payload level, but also we need to offer a secure mechanism to store sensitive information in memory.

Why
You might say that this is not important if the servers where data is processed is secured. Yes in an ideal world. There is no bug free software.
This means that M&S (Maintenance and Support) team that has direct access to that server might need to do a memory dump. The memory dump shall be encrypted, shared only with a limited number of people and many more. They could even share the memory dump with external 3rd parties that offer support for other systems or software that are used on that machine.
For this we need to ensure that data privacy is respected.

If the server is not configured properly, the system might create automatically a memory dump when an error occurred and upload it to different providers. On top of this, M&S teams usually prefer to configure the production system in a such a way that a memory dump is created automatically when a crush is detected.

How to encrypt data in memory
There are different libraries that can help s to achieve this goal, one of them is coming with .NET library - Memory Protection. If give us the possibility to encrypt an array of bytes that is stored in memory.
ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);
ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
As we can see, the encryption is very easy to use. The output of encrypted content is an array that is stored in memory. This array will be encrypted/decrypted. Having an array, it's allowing us to protect any kind of data.

When you are working with encryption, we don't need to forget about block size. As other encryption mechanism this library is using block size of 16 bytes for encryption. It means that if we encrypt a string or other type of data we need to ensure that the byte array block size is multiple of 16.
Unfortunately there is no out of the box support for this.
Below you can find also an extension method for string that add padding with default character. Of course this is not perfect solution, a padding standard like PKCS7 shall be used. When content is unencrypted, the default character used for padding is removed.
public static class StringPaddingExtensions
{
    public static byte[] ToByteArrayWithPadding(this String str)
    {
        const int BlockingSize = 16;
        int byteLength = ((str.Length / BlockingSize) + 1) * BlockingSize;
        byte[] toEncrypt = new byte[byteLength];
        ASCII.GetBytes(str).CopyTo(toEncrypt, 0);
        return toEncrypt;
    }

    public static string RemovePadding(this String str)
    {
        char paddingChar = '\0';
        int indexOfFirstPadding = str.IndexOf(paddingChar);
        string cleanString = str.Remove(indexOfFirstPadding);
        return cleanString;
    }
}
 string contentToEncrypt = "Hello Word!";

 byte[] toEncrypt = contentToEncrypt.ToByteArrayWithPadding();
 ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);

 ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
 string decryptedContent = ASCII.GetString(toEncrypt).RemovePadding();
I added on GitHub a full sample, with a unit tests that shows how to encrypt/decrypt content: https://github.com/vunvulear/Stuff/blob/master/MemoryEncryption/MemoryEncryption.cs

It is not bulletproof
Even if the content is encrypted while it is in memory, this doesn't means that we can do anything with it. There are moment during transition and processing when content will be in clear text. In this moment, a memory dump could catch easily data in clear.
Protected memory encrypts data that is stored in memory, but it doesn't allow us to process it. To be able to process it or do any kind of transformation we will need it in clear text.
The below diagrams shows when data is in a unprotected state.
As we can see, Protected Memory offer us protection only at one step. For the other steps we need to ensure that content is protected using other mechanism. With RED are marked steps when data is in memory and a memory dump could catch data in clear text.
In top of this, we need to ensure that GC clears that memory locations where data is stored in clear text.

Why there is all time a risk
Unfortunately, most of the times we need to process data that is often strings. Strings are one of the weakest data types, especially in .NET and Java. Using pinning you could have data that is saved all the time, but this is more common in C++ then .NET. Wring in this way for .NET will be a challenge and testing a nightmare.

Conclusion
Data protection is not only a complex discussion, but also cannot be bulletproof. Because of this we need to find a balance between the level of protection we need and how much we want to invest in it. In the end it is a risk mitigation.

Comments

Popular posts from this blog

How to check in AngularJS if a service was register or not

There are cases when you need to check in a service or a controller was register in AngularJS.
For example a valid use case is when you have the same implementation running on multiple application. In this case, you may want to intercept the HTTP provider and add a custom step there. This step don’t needs to run on all the application, only in the one where the service exist and register.
A solution for this case would be to have a flag in the configuration that specify this. In the core you would have an IF that would check the value of this flag.
Another solution is to check if a specific service was register in AngularJS or not. If the service was register that you would execute your own logic.
To check if a service was register or not in AngularJS container you need to call the ‘has’ method of ‘inhector’. It will return TRUE if the service was register.
if ($injector.has('httpInterceptorService')) { $httpProvider.interceptors.push('httpInterceptorService&#…

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

[Post-Event] Codecamp Conference Cluj-Napoca - Nov 19, 2016

Last day I was invited to another Codecamp Conference, that took place in Cluj-Napoca. Like other Codecamp Conferences, the event was very big, with more than 1.000 participants and 70 sessions. There were 10 tracks in parallel, so it was pretty hard to decide at  what session you want to join.
It was great to join this conference and I hope that you discovered something new during the conference.
At this event I talked about Azure IoT Hub and how we can use it to connect devices from the field. I had a lot of demos using Raspberry PI 3 and Simplelink SensorTag. Most of the samples were written in C++ and Node.JS and people were impressed that even if we are using Microsoft technologies, we are not limited to C# and .NET. World and Microsoft are changing so fast. Just looking and Azure IoT Hub and new features that were launched and I'm pressed (Jobs, Methods, Device Twin).
On backend my demos covered Stream Analytics, Event Hub, Azure Object Storage and DocumentDB.

Title:
What abo…