Designing to purge data based on business requirements

Data purging is another overlapping concept with data retention. Data purging refers to the process of safely deleting data as per business requirements. Let's look at the options available in Azure Data Lake Storage and Azure Synapse SQL.

Purging data in Azure Data Lake Storage Gen2

There are two ways in which we can achieve scheduled data deletion or data purging. One is the same as data retention, using life cycle management in Azure Storage. The second technique is to use Azure Data Factory. Whenever we delete data in Azure, it ensures that the storage locations are overwritten so that the deleted data cannot be retrieved by a malicious user. You can refer to Microsoft's data protection policy here: https://docs.microsoft.com/en-us/azure/security/fundamentals/protection-customer-data.

We can periodically or conditionally delete files from Azure Storage using the Delete Activity in ADF. We have already seen...