Enterprise data consists of crucial information related to sales, customer interactions, human resources, and so on, and is locked securely within systems such as ERP, CRM, and general ledger systems. In the last decade, enterprise data security has matured significantly as organizations learned their lessons from various data security incidents that caused them losses in billions. As the services industry has grown and matured, most of the systems are outsourced to vendors who deal with crucial client information most of the time. As a result, security and privacy standards such as HIPAA, HITECH, PCI, SOX, ISO, and COBIT have evolved . This requires service providers to comply with these regulatory standards to fully safeguard their client's data assets. This has resulted in a very protective data security enforcement within enterprises including service providers as well as the clients. There is absolutely no tolerance to data security violations. Over the last eight years of its development, Hadoop has now reached a mature state where enterprises have started adopting it for their Big Data processing needs. The prime use case is to gain strategic and operational advantages from their humongous data sets. However, to do any analysis on top of these datasets, we need to bring them to the Hadoop ecosystem for processing. So the immediate question that arises with respect to data security is, how secure is the data storage inside the Hadoop ecosystem?
The question is not just about securing the source data which is moved from the enterprise systems to the Hadoop ecosystem. Once these datasets land into the Hadoop ecosystems, analysts and data scientists perform large-scale analytics and machine-learning-based processing to derive business insights. These business insights are of great importance to the enterprise. Any such insights in the hands of the competitor or any unauthorized personnel could be disastrous to the business. It is these business insights that are highly sensitive and must be fully secured.
Any data security incident will cause business users to lose their trust in the ecosystem. Unless the business teams have confidence in the Hadoop ecosystem, they won't take the risk to invest in Big Data. Hence, the success and failure of Big Data-related projects really depends upon how secure our data ecosystem is going to be.
