Reader small image

You're reading from  Hadoop 2.x Administration Cookbook

Product typeBook
Published inMay 2017
PublisherPackt
ISBN-139781787126732
Edition1st Edition
Tools
Right arrow
Author (1)
Aman Singh
Aman Singh
author image
Aman Singh

Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
Read more about Aman Singh

Right arrow

Configuring job queues


In this recipe, we will configure the job queue and allow users to submit jobs to the queues. In production, there might be many departments such as marketing, sales, and finance, sharing a cluster of resources and it is important to have the correct shares proportional to business and funding.

In the previous recipe, although the queues were setup, they were still not used. Queues were dynamically created for jobs submitted by specifying a queue. If no queue is specified, jobs are submitted to a queue by the name of the user who submitted the job. We will explore these a bit more in this recipe.

Getting ready

To complete the recipe, the user must have a running cluster with HDFS and YARN configured and must have completed the previous two recipes.

How to do it...

  1. Connect to the master1.cyrus.com master node in the cluster and switch as user hadoop.

  2. Edit the fair-scheduler.xml allocation file as shown next. Note that there is no user specified for any queue (within the ...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Hadoop 2.x Administration Cookbook
Published in: May 2017Publisher: PacktISBN-13: 9781787126732

Author (1)

author image
Aman Singh

Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing
Read more about Aman Singh