Lazy persist writes in HDFS

Enterprise adoption of Hadoop is growing day by day. With increased adoption, there are a variety of application types that are using Hadoop for their enterprise goals. One such adoption is for applications that need to deal with data that amounts to only a few GBs. Keeping performance goals in mind with such small records would incur more latency costs when DISK I/O writes are involved during its execution—especially when such volumes of data can easily fit into memory without any DISK I/O. With the release of Hadoop 2.6, provisions for writes have been introduced that will use the off-heap memory of DataNodes. Eventually, data from memory will be flushed out to disk asynchronously. This will remove any expensive Disk I/O and computations for checksum while write operations are initiated from the HDFS client. Such asynchronous writes are called lazy persist writes, where persistence to disk does not happen immediately but asynchronously after some time. HDFS...