Skip to main content

In-memory data protection (encryption)

In this post we'll talk about memory encryption and why we need to care about it.

Context
With EU regulations, IT solutions needs to secure data content about citizens End to End (E2E). This means that we need to offer not only a secure storage, a secure connection between different nodes, encryption at payload level, but also we need to offer a secure mechanism to store sensitive information in memory.

Why
You might say that this is not important if the servers where data is processed is secured. Yes in an ideal world. There is no bug free software.
This means that M&S (Maintenance and Support) team that has direct access to that server might need to do a memory dump. The memory dump shall be encrypted, shared only with a limited number of people and many more. They could even share the memory dump with external 3rd parties that offer support for other systems or software that are used on that machine.
For this we need to ensure that data privacy is respected.

If the server is not configured properly, the system might create automatically a memory dump when an error occurred and upload it to different providers. On top of this, M&S teams usually prefer to configure the production system in a such a way that a memory dump is created automatically when a crush is detected.

How to encrypt data in memory
There are different libraries that can help s to achieve this goal, one of them is coming with .NET library - Memory Protection. If give us the possibility to encrypt an array of bytes that is stored in memory.
ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);
ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
As we can see, the encryption is very easy to use. The output of encrypted content is an array that is stored in memory. This array will be encrypted/decrypted. Having an array, it's allowing us to protect any kind of data.

When you are working with encryption, we don't need to forget about block size. As other encryption mechanism this library is using block size of 16 bytes for encryption. It means that if we encrypt a string or other type of data we need to ensure that the byte array block size is multiple of 16.
Unfortunately there is no out of the box support for this.
Below you can find also an extension method for string that add padding with default character. Of course this is not perfect solution, a padding standard like PKCS7 shall be used. When content is unencrypted, the default character used for padding is removed.
public static class StringPaddingExtensions
{
    public static byte[] ToByteArrayWithPadding(this String str)
    {
        const int BlockingSize = 16;
        int byteLength = ((str.Length / BlockingSize) + 1) * BlockingSize;
        byte[] toEncrypt = new byte[byteLength];
        ASCII.GetBytes(str).CopyTo(toEncrypt, 0);
        return toEncrypt;
    }

    public static string RemovePadding(this String str)
    {
        char paddingChar = '\0';
        int indexOfFirstPadding = str.IndexOf(paddingChar);
        string cleanString = str.Remove(indexOfFirstPadding);
        return cleanString;
    }
}
 string contentToEncrypt = "Hello Word!";

 byte[] toEncrypt = contentToEncrypt.ToByteArrayWithPadding();
 ProtectedMemory.Protect(toEncrypt, MemoryProtectionScope.SameProcess);

 ProtectedMemory.Unprotect(toEncrypt, MemoryProtectionScope.SameProcess);
 string decryptedContent = ASCII.GetString(toEncrypt).RemovePadding();
I added on GitHub a full sample, with a unit tests that shows how to encrypt/decrypt content: https://github.com/vunvulear/Stuff/blob/master/MemoryEncryption/MemoryEncryption.cs

It is not bulletproof
Even if the content is encrypted while it is in memory, this doesn't means that we can do anything with it. There are moment during transition and processing when content will be in clear text. In this moment, a memory dump could catch easily data in clear.
Protected memory encrypts data that is stored in memory, but it doesn't allow us to process it. To be able to process it or do any kind of transformation we will need it in clear text.
The below diagrams shows when data is in a unprotected state.
As we can see, Protected Memory offer us protection only at one step. For the other steps we need to ensure that content is protected using other mechanism. With RED are marked steps when data is in memory and a memory dump could catch data in clear text.
In top of this, we need to ensure that GC clears that memory locations where data is stored in clear text.

Why there is all time a risk
Unfortunately, most of the times we need to process data that is often strings. Strings are one of the weakest data types, especially in .NET and Java. Using pinning you could have data that is saved all the time, but this is more common in C++ then .NET. Wring in this way for .NET will be a challenge and testing a nightmare.

Conclusion
Data protection is not only a complex discussion, but also cannot be bulletproof. Because of this we need to find a balance between the level of protection we need and how much we want to invest in it. In the end it is a risk mitigation.

Comments

Popular posts from this blog

ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded

Today blog post will be started with the following error when running DB tests on the CI machine:
threw exception: System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information. at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName) This error happened only on the Continuous Integration machine. On the devs machines, everything has fine. The classic problem – on my machine it’s working. The CI has the following configuration:

TeamCity.NET 4.51EF 6.0.2VS2013
It seems that there …

How to check in AngularJS if a service was register or not

There are cases when you need to check in a service or a controller was register in AngularJS.
For example a valid use case is when you have the same implementation running on multiple application. In this case, you may want to intercept the HTTP provider and add a custom step there. This step don’t needs to run on all the application, only in the one where the service exist and register.
A solution for this case would be to have a flag in the configuration that specify this. In the core you would have an IF that would check the value of this flag.
Another solution is to check if a specific service was register in AngularJS or not. If the service was register that you would execute your own logic.
To check if a service was register or not in AngularJS container you need to call the ‘has’ method of ‘inhector’. It will return TRUE if the service was register.
if ($injector.has('httpInterceptorService')) { $httpProvider.interceptors.push('httpInterceptorService&#…

Fundamental Books of a Software Engineer (version 2018)

More then six years ago I wrote a blog post about fundamental books that any software engineer (developer) should read. Now it is an excellent time to update this list with new entries.

There are 5 different categories of books, that represent the recommended path. For example, you start with Coding books, after that, you read books about Programming, Design and so on.
There are some books about C++ that I recommend not because you shall know C++, only because the concepts that you can learn from it.

Coding

Writing solid codeCode completeProgramming Pearls, more programming pearls(recommended)[NEW] Introduction to Algorithms

Programming

Refactoring (M. Fowler)Pragmatic ProgrammerClean code[NEW] Software Engineering: A Practitioner's Approach[NEW] The Mythical Man-Month[NEW] The Art of Computer Programming

Design

Applying UML and Patterns (GRASP patterns)C++ coding standards (Sutter, Alexandrescu)The C++ programming language (Stroustrup, Part IV)Object-oriented programming (Peter Coad)P…