Skip to main content

In-memory data protection (encryption)

In this post we'll talk about memory encryption and why we need to care about it.

Context
With EU regulations, IT solutions needs to secure data content about citizens End to End (E2E). This means that we need to offer not only a secure storage, a secure connection between different nodes, encryption at payload level, but also we need to offer a secure mechanism to store sensitive information in memory.

Why
You might say that this is not important if the servers where data is processed is secured. Yes in an ideal world. There is no bug free software.
This means that M&S (Maintenance and Support) team that has direct access to that server might need to do a memory dump. The memory dump shall be encrypted, shared only with a limited number of people and many more. They could even share the memory dump with external 3rd parties that offer support for other systems or software that are used on that machine.
For this we need to ensure that data privacy is respected.

If the server is not configured properly, the system might create automatically a memory dump when an error occurred and upload it to different providers. On top of this, M&S teams usually prefer to configure the production system in a such a way that a memory dump is created automatically when a crush is detected.

How to encrypt data in memory
There are different libraries that can help s to achieve this goal, one of them is coming with .NET library - Memory Protection. If give us the possibility to encrypt an array of bytes that is stored in memory.
ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);
ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
As we can see, the encryption is very easy to use. The output of encrypted content is an array that is stored in memory. This array will be encrypted/decrypted. Having an array, it's allowing us to protect any kind of data.

When you are working with encryption, we don't need to forget about block size. As other encryption mechanism this library is using block size of 16 bytes for encryption. It means that if we encrypt a string or other type of data we need to ensure that the byte array block size is multiple of 16.
Unfortunately there is no out of the box support for this.
Below you can find also an extension method for string that add padding with default character. Of course this is not perfect solution, a padding standard like PKCS7 shall be used. When content is unencrypted, the default character used for padding is removed.
public static class StringPaddingExtensions
{
    public static byte[] ToByteArrayWithPadding(this String str)
    {
        const int BlockingSize = 16;
        int byteLength = ((str.Length / BlockingSize) + 1) * BlockingSize;
        byte[] toEncrypt = new byte[byteLength];
        ASCII.GetBytes(str).CopyTo(toEncrypt, 0);
        return toEncrypt;
    }

    public static string RemovePadding(this String str)
    {
        char paddingChar = '\0';
        int indexOfFirstPadding = str.IndexOf(paddingChar);
        string cleanString = str.Remove(indexOfFirstPadding);
        return cleanString;
    }
}
 string contentToEncrypt = "Hello Word!";

 byte[] toEncrypt = contentToEncrypt.ToByteArrayWithPadding();
 ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);

 ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
 string decryptedContent = ASCII.GetString(toEncrypt).RemovePadding();
I added on GitHub a full sample, with a unit tests that shows how to encrypt/decrypt content: https://github.com/vunvulear/Stuff/blob/master/MemoryEncryption/MemoryEncryption.cs

It is not bulletproof
Even if the content is encrypted while it is in memory, this doesn't means that we can do anything with it. There are moment during transition and processing when content will be in clear text. In this moment, a memory dump could catch easily data in clear.
Protected memory encrypts data that is stored in memory, but it doesn't allow us to process it. To be able to process it or do any kind of transformation we will need it in clear text.
The below diagrams shows when data is in a unprotected state.
As we can see, Protected Memory offer us protection only at one step. For the other steps we need to ensure that content is protected using other mechanism. With RED are marked steps when data is in memory and a memory dump could catch data in clear text.
In top of this, we need to ensure that GC clears that memory locations where data is stored in clear text.

Why there is all time a risk
Unfortunately, most of the times we need to process data that is often strings. Strings are one of the weakest data types, especially in .NET and Java. Using pinning you could have data that is saved all the time, but this is more common in C++ then .NET. Wring in this way for .NET will be a challenge and testing a nightmare.

Conclusion
Data protection is not only a complex discussion, but also cannot be bulletproof. Because of this we need to find a balance between the level of protection we need and how much we want to invest in it. In the end it is a risk mitigation.

Comments

Popular posts from this blog

Windows Docker Containers can make WIN32 API calls, use COM and ASP.NET WebForms

After the last post , I received two interesting questions related to Docker and Windows. People were interested if we do Win32 API calls from a Docker container and if there is support for COM. WIN32 Support To test calls to WIN32 API, let’s try to populate SYSTEM_INFO class. [StructLayout(LayoutKind.Sequential)] public struct SYSTEM_INFO { public uint dwOemId; public uint dwPageSize; public uint lpMinimumApplicationAddress; public uint lpMaximumApplicationAddress; public uint dwActiveProcessorMask; public uint dwNumberOfProcessors; public uint dwProcessorType; public uint dwAllocationGranularity; public uint dwProcessorLevel; public uint dwProcessorRevision; } ... [DllImport("kernel32")] static extern void GetSystemInfo(ref SYSTEM_INFO pSI); ... SYSTEM_INFO pSI = new SYSTEM_INFO(

Azure AD and AWS Cognito side-by-side

In the last few weeks, I was involved in multiple opportunities on Microsoft Azure and Amazon, where we had to analyse AWS Cognito, Azure AD and other solutions that are available on the market. I decided to consolidate in one post all features and differences that I identified for both of them that we should need to take into account. Take into account that Azure AD is an identity and access management services well integrated with Microsoft stack. In comparison, AWS Cognito is just a user sign-up, sign-in and access control and nothing more. The focus is not on the main features, is more on small things that can make a difference when you want to decide where we want to store and manage our users.  This information might be useful in the future when we need to decide where we want to keep and manage our users.  Feature Azure AD (B2C, B2C) AWS Cognito Access token lifetime Default 1h – the value is configurable 1h – cannot be modified

What to do when you hit the throughput limits of Azure Storage (Blobs)

In this post we will talk about how we can detect when we hit a throughput limit of Azure Storage and what we can do in that moment. Context If we take a look on Scalability Targets of Azure Storage ( https://azure.microsoft.com/en-us/documentation/articles/storage-scalability-targets/ ) we will observe that the limits are prety high. But, based on our business logic we can end up at this limits. If you create a system that is hitted by a high number of device, you can hit easily the total number of requests rate that can be done on a Storage Account. This limits on Azure is 20.000 IOPS (entities or messages per second) where (and this is very important) the size of the request is 1KB. Normally, if you make a load tests where 20.000 clients will hit different blobs storages from the same Azure Storage Account, this limits can be reached. How we can detect this problem? From client, we can detect that this limits was reached based on the HTTP error code that is returned by HTTP