Capacity scheduler

The capacity scheduler makes sure that users get the guaranteed minimum amount of configured resources in the YARN cluster. The use of the Hadoop cluster increases with the use cases in the organization and it is very unlikely that organization creates separate Hadoop clusters for each use case because this will increase maintenance. One use case may be that different users in the same organization want to have a certain amount of resources reserved when they want to execute their tasks. The capacity scheduler helps in sharing the cluster resources in a cost-effective manner across different users in the same organization to meet the SLA by ensuring no other user uses resources configured for some other user in the cluster. In short, the cluster resources are shared across multiple user groups. The capacity scheduler works on the concept of queues. A cluster is divided into partitions known as queues and each queue is assigned with a certain percentage of resources. The...