YARN is Apache Hadoop's NextGen Resource Manager. The Spark project provides an easy way to schedule jobs on YARN once you have a Spark assembly built. The Spark web page, http://spark.apache.org/docs/latest/running-on-yarn.html, has the configuration details for YARN, which we had built earlier for compiling with the -Pyarn
switch.
You're reading from Fast Data Processing with Spark 2 - Third Edition
You have been reading a chapter from
Fast Data Processing with Spark 2 - Third EditionPublished in: Oct 2016Publisher: PacktISBN-13: 9781785889271
© 2016 Packt Publishing Limited All Rights Reserved
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
undefined
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime