Thoughts on data security

One of the very important capabilities required for a Data Lake implementation in an enterprise is security. In a Data Lake we are bringing in data from around the enterprise into one place. You have convinced all the departments who has agreed to ingest data into the Data Lake that the data in the lake is secured and only authenticated and authorized users have access to the data. So, this aspect needs some serious thought so that data is secured and these departments are quite happy with the access rules for their all important data. In addition to security setup, proper governance through adequate processes also should be setup to make security quite sturdy but quite easy for users having access to it to do their deep analysis work.

By data security, it refers to in-flight transaction data (stream), date at rest (batch), both authentication and authorization (attributes).

Data lake does pose a different risk as it is entrusted to bring data from various silos into...