Hadoop can be installed in multiple ways, either by using repository methods such as Yum/apt-get or by extracting the tarball packages. The project Bigtop http://bigtop.apache.org/ provides Hadoop packages for infrastructure, and can be used by creating a local repository of the packages.
All the steps are to be performed as the root user. It is expected that the user knows how to set up a yum repository and Linux basics.
You are going to need a Linux machine. You can either use the one which has been used in the previous task or set up a new node, which will act as repository server and host all the packages we need.
Connect to a Linux machine that has at least 5 GB disk space to store the packages.
If you are on CentOS or a similar distribution, make sure you have the package
yum-utilsinstalled. This package will provide the commandreposync.Create a file
bigtop.repounder/etc/yum.repos.d/. Note that the file name can be anything—only the extension must be.repo.See the following screenshot for the contents of the file:
Execute the command
reposync –r bigtop. It will create a directory namedbigtopunder the present working directory with all the packages downloaded to it.All the required Hadoop packages can be installed by configuring the repository we downloaded as a repository server.
From step 2 to step 6, the user will be able to configure and use the Hadoop package repository. Setting up a Yum repository is not required, but it makes things easier if we have to do installations on hundreds of nodes. In larger setups, management systems such as Puppet or Chef will be used for deployment configuration to push configuration and packages to nodes.
In this chapter, we will be using the tarball package that was built in the first section to perform installations. This is the best way of learning about directory structure and the configurations needed.
