Did you ever have a requirement that specifies that once a document is written in storage, nobody shall be able to modify or delete it, including the administrators?
OR
Do you have a special business requirement that specifies that audit data can be written only once and nobody shall be able to alter them?
WORM
This kind of business requirements are called WORM (Write Once, Read Many) and are common for industries like financial or healthcare. When you have such a requirement on top of Azure, there were not too many options until now.
What we had until now?
With available features of Azure Storage, you could define groups of users that are allowed only to do read operations. The same thing can be accomplished if you wrap everything behind an API (e.g., REST API) that manage one operation are allowed or not.
In both scenarios, you still have some users or access keys that can do delete or update actions. From this perspective, it is impossible to guaranty or offers an SLA where data it is immutable.
Azure new capability
Starting from now, Azure Storage is supporting WORM by default, allowing us to have data stored inside Azure Blobs that is immutable as long as we want. Even the Administrator of the Storage Account it is not allowed to delete or modify the content an long as the WORM is active.
Azure Storage Blobs support two types of WORM:
- Time-based Retention
- Legal Holds
The two types of WORMS are supported at account or container level. Once the policy is applied at container level all blobs under the container will respect that specific policy, the existing or new one.
This feature is supported on all types of blobs, but it is recommended to use only blob types. For appearing blob and page blob, once you apply the policy, they cannot be modified anymore. Beside this, initially, they are created outside the policy and copied later inside the policy.
Deletion
The deletion procedure will fail if the blob is under a WORM policy. An interesting behavior is during the deletion. You can delete a container that has immutable policies only when you don’t have any blobs inside it. The same rule applies to Storage Account that can be deleted only if no active WORK policies are active or no blobs exist.
Main flow
For time-based retention policy, the existing blobs under an existing container will go in the immutable state from the moment when the blob was created until the end of the retention policy. The new blobs will be locked from the moment of creation until the expiration of the retention policy interval.
In the case of legal holds, the immutable is kept on the blobs until all the legal holds are cleared. When they are combined with time-based retention, the blobs will remain immutable even if the time-base expire.
Limitations
The retention policy can be specified between 1 day to 400 years. If the minimum value is discussable, the maximum value it is more than enough for any scenarios. Keep in mind that once you create a policy and set to lock state, you cannot modify it. It means that you will not be able to delete or modify content as long as the policy is active. Be careful during the test and integration phases.
At Storage Accounts there are some limitations related to the number of immutable policies that can be defined. We can have a maximum of 1000 containers with immutable holds policies on each time (1000 with time-based retention, 1000 with Legal Holds).
An important thing that we need to take into account is the maximum number of holds (tag holds) that we can have on the container – 10. For time-based retention policies, 1 per container, and each of them supports maximum three-time extensions.
Tier change
A nice feature is the capability to change the tier of data even if for immutable blobs. It is allowing us to optimize cost for long-running WORM policies without affecting our policies.
How to configure
At this moment in time, the most important libraries are already supporting immutable storage (Java, Python, Node.JS and of course .NET). Additional to this configuration can be done from Powershell or Azure Portal too.
In the Policy section of each container of a blob, you find the option to add the immutable locks. For the time base, take into account that when you create them, the default state is unlocked. Once you lock the policy, you’ll not be able to do any changes until the time expires.
Audit Logs
Both immutable policies have support for audit. Any changes that are done to the policies are automatical audited. A nice thing related to it is the period for how long this information is kept and is equal to the lifetime of the container. If you keep the container for 400 years, the audit logs are kept for 400 years.
Cost
This feature is free, there is no additional cost if you activate this feature.
Don’t forget that this new WORM policy can be applied on existing storage account too.
OR
Do you have a special business requirement that specifies that audit data can be written only once and nobody shall be able to alter them?
WORM
This kind of business requirements are called WORM (Write Once, Read Many) and are common for industries like financial or healthcare. When you have such a requirement on top of Azure, there were not too many options until now.
What we had until now?
With available features of Azure Storage, you could define groups of users that are allowed only to do read operations. The same thing can be accomplished if you wrap everything behind an API (e.g., REST API) that manage one operation are allowed or not.
In both scenarios, you still have some users or access keys that can do delete or update actions. From this perspective, it is impossible to guaranty or offers an SLA where data it is immutable.
Azure new capability
Starting from now, Azure Storage is supporting WORM by default, allowing us to have data stored inside Azure Blobs that is immutable as long as we want. Even the Administrator of the Storage Account it is not allowed to delete or modify the content an long as the WORM is active.
Azure Storage Blobs support two types of WORM:
- Time-based Retention
- Legal Holds
The two types of WORMS are supported at account or container level. Once the policy is applied at container level all blobs under the container will respect that specific policy, the existing or new one.
This feature is supported on all types of blobs, but it is recommended to use only blob types. For appearing blob and page blob, once you apply the policy, they cannot be modified anymore. Beside this, initially, they are created outside the policy and copied later inside the policy.
Deletion
The deletion procedure will fail if the blob is under a WORM policy. An interesting behavior is during the deletion. You can delete a container that has immutable policies only when you don’t have any blobs inside it. The same rule applies to Storage Account that can be deleted only if no active WORK policies are active or no blobs exist.
Main flow
For time-based retention policy, the existing blobs under an existing container will go in the immutable state from the moment when the blob was created until the end of the retention policy. The new blobs will be locked from the moment of creation until the expiration of the retention policy interval.
In the case of legal holds, the immutable is kept on the blobs until all the legal holds are cleared. When they are combined with time-based retention, the blobs will remain immutable even if the time-base expire.
Limitations
The retention policy can be specified between 1 day to 400 years. If the minimum value is discussable, the maximum value it is more than enough for any scenarios. Keep in mind that once you create a policy and set to lock state, you cannot modify it. It means that you will not be able to delete or modify content as long as the policy is active. Be careful during the test and integration phases.
At Storage Accounts there are some limitations related to the number of immutable policies that can be defined. We can have a maximum of 1000 containers with immutable holds policies on each time (1000 with time-based retention, 1000 with Legal Holds).
An important thing that we need to take into account is the maximum number of holds (tag holds) that we can have on the container – 10. For time-based retention policies, 1 per container, and each of them supports maximum three-time extensions.
Tier change
A nice feature is the capability to change the tier of data even if for immutable blobs. It is allowing us to optimize cost for long-running WORM policies without affecting our policies.
How to configure
At this moment in time, the most important libraries are already supporting immutable storage (Java, Python, Node.JS and of course .NET). Additional to this configuration can be done from Powershell or Azure Portal too.
In the Policy section of each container of a blob, you find the option to add the immutable locks. For the time base, take into account that when you create them, the default state is unlocked. Once you lock the policy, you’ll not be able to do any changes until the time expires.
If you don't see the option available it means that your Azure Storage account needs to be upgraded to V2. This can be done from the Azure Portal. In the Configuration Section, you will find the option to upgrade it.
Audit Logs
Both immutable policies have support for audit. Any changes that are done to the policies are automatical audited. A nice thing related to it is the period for how long this information is kept and is equal to the lifetime of the container. If you keep the container for 400 years, the audit logs are kept for 400 years.
Cost
This feature is free, there is no additional cost if you activate this feature.
Don’t forget that this new WORM policy can be applied on existing storage account too.
Comments
Post a Comment