Summary

This chapter focused on YARN and its components. We covered various concepts, such as YARN architecture and resource manager HA in detail. We then went through YARN schedulers and how they work. We also covered some basic configurations for the scheduler and explained when to use what. The primary focus was to introduce the new features that have been added to Hadoop version 3. Thus, we covered the opportunist container and the Timeline server. In the last part, we went through the Docker container executor and a few commonly used Hadoop commands.  In the next chapter, we will cover MapReduce processing in detail, we will deep dive into MapReduce flow, and will look into each step in detail. We will go through different examples and look into a few techniques to improve the performance of applications.