This chapter deals with the various tools that will aid the user to install, configure, update, and delete software, documentation, and system functionalities from the server machine. Software in Ubuntu is organized in packages. Almost all of these are available online and also on CDs and other media. In this chapter, we will look at the various command-line tools to handle software and package management. GUI tools are also present, but they are seldom used by server administrators. Command-line tools are faster and more secure, and we will be using only a command-line-based terminal throughout the book.
The following topics will be covered in the chapter:
Using dpkg for package management
The apt-get package management tool
Package management with aptitude
Configuration of repositories and extra repositories
Automatic updates of software
Creating a repository mirror
Ubuntu gets its package management from the Debian Linux distribution under the GNU license. A package usually has the following contents: the actual software files, its metadata, and instructions for the user to install it on the Ubuntu machine. The file extension for Debian packages is .deb. A collection of packages is called a repository and the repository list is stored in the local system. Packages are in binary format and are usually precompiled so as to enable faster installation and free the user from having to compile it.
Large and complex packages are built on the concept of dependencies. Dependencies are additional packages that are required for the proper functioning of primary packages. The package management tools in Ubuntu also handle the downloading and installing of these dependencies.
Let's look at some terminologies that are used in package management.
The software and documentation in Linux is organized in packages. These can be considered to be a collection of components for a software or functionality.
A collection of packages is called a repository. Repositories are usually available in one or more centrally distributed servers online. These are tested extensively and are easy to install to or remove from your server machines.
Derivatives of Debian such as Ubuntu, Linux Mint, and others use the .deb packages. To install these .deb packages, the package management tools usually used are apt-get and aptitude. Both of these are a frontend for the dpkg tool.
A software or package that requires other software to function properly is said to be dependent on the latter. Package management tools such as apt-get and aptitude in Ubuntu handle the dependencies during installation and/or removal of packages. However, when using dpkg, the user has to manage the dependencies as dpkg does not handle dependencies.
Many of the packages or software in the repositories are independent, open source software, which means the software is freely available to use, modify, and distribute. These are often compiled and fine-tuned for specific distributions. It is also possible that the source repositories in your machine have the modified source code as packages.
Debian systems have dpkg as their underlying fundamental package management tool. dpkg can be used to install, uninstall, and build packages. One of the important points we need to remember is dpkg cannot download packages and its dependencies. The user has to take care of the dependencies and install them manually. dpkg can install the locally available packages. Let's now see some of the operations that dpkg can perform on an Ubuntu Server machine.
Use the following command to list the software already installed on your server machine:
dpkg –l
This command can generate a large output on your command line, depending upon the installed packages in the server machine, as shown in the following screenshot:
To list out a specific package, use the following command:
dpkg –l | grep apt
This will list out the packages with the name apt occurring in any of the installed packages. A sample output is shown here:
Here, we can see all of the installed packages in the server containing the word apt as whole or part of it in name of the packages that are installed in this server machine.
Another alternative to check whether a specific package is installed or not is to use the dpkg command with the -l option along with the package name. Let's check for the apt package:
dpkg -l apt
The output of this command is shown in the following screenshot. The letters ii at the start indicate the package is installed. The first letter that you see in the output before the name of the package is the desired status: i stands for installed and p stands for purged. The actual status is shown in the second letter. n stands for not installed and i in second place tells us the package is installed. This command also displays the version, architecture, and description, along with the name of the package.
Running dpkg without any arguments or parameters will show the error in the following screenshot. It requests the user to enter any of the options as arguments with dpkg:
dpkg
If you want to check which files a particular package has installed, use the following command with parameter -L and then the package name (here, apt-utils):
dpkg –L apt-utils
The output is shown in the following screenshot:
We listed all the files that the package apt-utils has installed in the machine. As you may see, it gives the absolute path of each file associated with the apt-utils package.
If you are unsure which package a particular file belongs to, you can use the dpkg command along with the optional -S parameter. The following command lists out the packages that are using this particular file:
dpkg -S /etc/logrotate.d
The output is as follows:
We see all the packages that are using /etc/logrotate.d in the preceding output. Let's crosscheck for one of the packages:
dpkg -L apport
The output is as follows:
In some cases, running dpkg -S may not be able to get the package name that uses a particular file. This is because many files are automatically generated during a package installation that dpkg may not be aware of.
To install a .deb package using dpkg, use the following command:
dpkg –i $package_name
Replace $package_name with the actual package you want to install. This will start the installation of the package. Please note that installation of software requires administrative access. So, use the dpkg command with the prefix sudo when installing or removing a package. Hence, this command will now become:
sudo dpkg –i $package_name
Let's now try to remove a package using the dpkg command. Use the option -r with the package name to remove a package. Here, we are removing the byobu package. However, it is recommended you do not use the dpkg tool to remove packages. It generally does not go well as dpkg does not handle dependencies, so it is possible that you may remove a package that is a dependency for some other package and the latter will become unstable or unusable. There are better alternative package management tools such as apt-get and aptitude that can be used for removal of packages, as they handle the dependencies well. We will cover the apt-get and aptitude tools next. The command to remove a package is as follows:
sudo dpkg –r byobu
As we can see in the following screenshot, when trying to remove a package without the sudo prefix, the system did not allow. After prefacing the command with sudo, we were successfully able to remove the byobu package from the server:
The command dpkg -P stands for purge. Configuration files are also removed along with the package when you run this command.
To view all the options dpkg supports, use the following command:
man dpkg
Ubuntu supports an Advanced Packaging Tool (APT) for package management: apt-get is one of the command-line tools to aid the user in installing, removing, upgrading packages, updating package index, and also upgrading the entire Ubuntu OS. Its simplicity is the reason behind its powerful features and why it is better than other GUI package management tools. Server administrators can use it over SSH connections and also they can use it in scripts for administrative purposes. Automation of scripts using cron scheduling is also an added advantage for server administrators when it comes to dealing with package management using apt-get.
Let's first discuss about the apt-get package and its various uses. Later, we will look at other utilities under APT, such as apt-cache, apt-file, apt-ftparchive, and so on.
It is recommended to always update the repository list in the local Ubuntu system before installing and/or removing the packages. It is possible that the package could have been upgraded along with changes in the dependencies. To be on the safe side, update the repository list with the following command:
sudo apt-get update
The preceding command will update the repository list and you will see an output similar to the following screenshot. The repositories are defined in the /etc/apt/sources.list file and the /etc/apt/sources.list.d directory. It is a database of all the packages available in the repositories. When you run the update command from the apt-get tool, it updates the list in the files and directories discussed earlier.
The local package index is updated to the latest list available in the remote repository.
It is quite easy to install a package with the apt-get tool. To install the apache2 package, type the following command:
sudo apt-get install apache2
The output is shown in the following screenshot:
The install command will read the repository list and try downloading the package from the URL listed. Once downloaded, the apt-get install tool will unpack and install the software. If there are additional packages required for the proper functioning of this software, then apt-get will also download the additional dependencies and install them too. This is one of the features that makes apt-get one of the desired package management tools for server administrators working on Debian-based systems.
You will see the preceding output after you install apache2 using the apt-get install tool. The user will be prompted if he approves the downloading of the package. Hit Y and Enter to proceed with download and install.
After successful installation, you will see an output similar to the following screenshot:
Now, let's try to remove this package using the apt-get tool. We will use the remove option. You need to specify the package name that you wish to remove from the machine. If you wish to remove or install multiple packages at once, you can mention them one after the other with a space as the separator. The following command will remove multiple packages apache2, apache2-bin, and ssl-cert:
sudo apt-get remove apache2 apache2-bin ssl-cert
After running the preceding command, apt-get removes the packages. The following screenshot shows those packages were removed. apt-get handles the dependencies, unlike the dpkg package management tool.
Before we upgrade, we need to run the update command. The update command will update all packages to the latest version, and the upgrade command actually downloads and installs the updated packages. The command for updating a package is as follows:
sudo apt-get update
apt-get allows the software packages to be upgraded to newer versions with the upgrade option. Newer versions of software become available with bug fixes or with additional functionalities on the remote repositories. Running the following command will upgrade the installed packages already installed on your machine with newer versions. Be sure to run the update command before you do an upgrade:
sudo apt-get upgrade
You will see an output similar to the following screenshot:
Here, we see that around 98 packages are ready to be upgraded. The server will get 49 MB of data downloaded from the repositories and install them.
Over a period of time, your system will accumulate loads of downloaded packages, copies of which are stored in /var/cache/apt/archives. If you want to remove only the packages that are obsolete, then use the autoclean option instead of clean with the apt-get tool. If you want to remove the .deb packages from the directory, run the following command:
sudo apt-get clean
The content of the folder is shown in the following screenshot:
If you wish to remove a package along with all its configuration files, use the following command:
sudo apt-get purge ssl-cert
You can remove and purge multiple packages too, as we did for the remove package command. Once you execute the preceding command, you will be able to see the output similar to the following screenshot. Once you purge a package, the dkpg tool will have no information about the package. All it will know is that the package was removed.
There are times when your package is not getting installed in spite of multiple tries. Sometimes, you might face a dependency issue whilst you are installing a package using the apt-get install command. Even though the apt-get package management tool is built to handle dependencies, this issue can pop up. To solve this issue, use the following command:
sudo apt-get –f install
This command will try and repair the broken dependencies in your system and the installation of the package will be successful. You might need to run this command when you are using the apt-get install command to install anything for the first time on a new server.
The output is shown in the following screenshot:
The apt-get check command is a diagnostic tool for package management. There can be instances where your system has broken dependencies. To verify the broken dependency issues, you can use the following command:
sudo apt-get check
This will update the package lists and check for the broken dependencies. It also updates the package cache.
This brings us to the end of the apt-get tool. However, there are additional tools from the apt-* family. We will look at apt-ftparchive in the Creating a repository mirror section where we will set up a repository mirror. Let's first look at the apt-cache tool.
The apt-cache tool is used to check for packages that are available in the local system. Search, dependency, and reverse dependency can be checked with the help of the apt-cache tool.
Use the following command to search for a package. Here, we are searching for the apache2-bin package using the apt-cache command with the search option:
sudo apt-cache search apache2-bin
As we can see in the following screenshot, the preceding command lists out the package and a short description:
If you want to check the dependencies for a particular package, then use the depends option with the apt-cache tool. The command is as follows:
sudo apt-cache depends apache2
The output is shown in the following screenshot:
The other way is also supported. If you need to check what packages are dependent on a particular package, then use the following command:
sudo apt-cache rdepends ssl-cert
The output is shown in the following screenshot:
An important point to remember is to run the apt-get update command before doing any of the dependency checks. It's time we learn about another package management tool, aptitude.
Type aptitude as shown here:
aptitude
The resulting screen is shown in the next screenshot. It is an interactive menu-driven frontend window for the APT to perform numerous tasks related to package management on your Ubuntu Server. You can install, remove, and upgrade packages with minimal operations from this window.
aptitude is best suited for a nongraphical environment, as it makes sure the command key operations are performed properly. Some of the experts believe that aptitude is a better suited package management tool compared to apt-get for dealing with packages that have dependencies. aptitude is the newer version and easier to use. Both aptitude and apt are abstractions over dpkg. Press F10 to enable the menu at the top. apt and aptitude use the same configuration, hence these are both interchangeable. aptitude has a menu-driven approach and supports all the apt-* commands.
aptitude presents the packages in categories as seen in the preceding screenshot. Use the navigation keys (Up and Down arrows) to jump to the categories. Each category's brief description is given in the second half of the window. The first category we see is Upgradable Packages, which shows the number of packages that are ready to be upgraded. The second line, New Packages, shows the number of packages that are new and can be installed on your Ubuntu Server. The next two are fairly simple to guess and understand. Virtual Packages gives an option to the administrators to organize the packages as they feel appropriate. Tasks are the packages that aren't categorized into any of these categories.
Some of the command keys to work with aptitude are as follows:
+: This adds a package to install
-: This removes a package
_: This purges a package
u: This updates a package
g: This performs the operation for which the packages are marked
?: This opens a help box and shows all the command keys and their operations
Ctrl + T: This is same as F10, and it accesses the top menu
q: This enables you to go one screen back
Q: This quits aptitude
Take a look at the following screenshot:
When you highlight a package, as shown in this screenshot, aptitude displays a character on the first column. These are the descriptions of the state of the package in your local machine. The following are the letters you may encounter:
i: This means the package is installedp: This means the package is purgedc: This means the configuration file is present but package is not installedv: This means it is a virtual packageu: This means the package is not configured but the files are unpackedB: This means the package is brokenC: This means the package is half-configuredH: This means the package is half-installed (may be due to removal failure)
The APT configurations are stored in the /etc/apt/sources.list.d directory and the /etc/apt/sources.list file in the server. An example of the file can be seen in the following screenshot. This will vary from system to system based on the remote repositories configured. You can enable or disable repositories by commenting the lines in the /etc/apt/sources.list file. To edit this file, you may require root permissions.
There are numerous repositories available other than the official Ubuntu one for installing packages on your server. Two of the popular ones are Universe and Multiverse. Both are open source and are supported widely by the community, and thus make it safe to use with your server. There are no security updates to these packages, but in some countries there might be some issues with the Multiverse package. This is also one of the reasons why Ubuntu does not have the Multiverse package support out of the box. If you take a look at the configuration files, you will observe that these repositories are enabled by default. You can disable them by commenting it out. Comment these lines to disable both the repositories:
deb http://archive.ubuntu.com/ubuntu precise universe multiverse deb-src http://archive.ubuntu.com/ubuntu precise universe multiverse deb http://us.archive.ubuntu.com/ubuntu/ precise universe deb-src http://us.archive.ubuntu.com/ubuntu/ precise universe deb http://us.archive.ubuntu.com/ubuntu/ precise-updates universe deb-src http://us.archive.ubuntu.com/ubuntu/ precise-updates universe deb http://us.archive.ubuntu.com/ubuntu/ precise multiverse deb-src http://us.archive.ubuntu.com/ubuntu/ precise multiverse deb http://us.archive.ubuntu.com/ubuntu/ precise-updates multiverse deb-src http://us.archive.ubuntu.com/ubuntu/ precise-updates multiverse deb http://security.ubuntu.com/ubuntu precise-security universe deb-src http://security.ubuntu.com/ubuntu precise-security universe deb http://security.ubuntu.com/ubuntu precise-security multiverse deb-src http://security.ubuntu.com/ubuntu precise-security multiverse
To view the support status of your Ubuntu machine, run the following command:
ubuntu-support-status
To view all the packages that are supported or not supported run the same command with the --show-all option.
You will be able to see a summary of the package support and its expiry time. It will be something similar to the following screenshot:
One of the cons of using third-party repositories is the chance of dependencies not being available or any package conflicting with any Ubuntu package. This can be a cycle and is termed as dependency hell. Make sure you go through other users' feedback on any issues with a particular package before installing it.
Finding the right mirror is crucial if you want to ensure a clean and fast installation. A mirror is a replica of the repository at a different physical location. You will need to look at the following factors when selecting a mirror and decide on it:
Distance: Try and use a mirror that is physically close to you.
Protocol: Your chosen mirror may not support all of the protocols such as HTTP,FTP, rsync, and so on.
Speed: Less number of people connecting to a mirror means more speed.
Visit https://launchpad.net/ubuntu/+archivemirrors for a list of mirrors and choose a mirror that best fits on the factors.
It is often observed that when a user tries to install a package using apt-get install, they might get an error such as the one shown here:
W: Failed to fetch gzip:/var/lib/apt/lists/partial/us.archive.ubuntu.com_ubuntu_dists_natty_main_source_Sources Hash Sum mismatch, E: Some index files failed to download. They have been ignored, or old ones used instead.
Fix your configuration list. Run the following commands to resolve this issue:
sudo rm /var/lib/apt/lists/* sudo apt-get update
There may be some packages that you require that are not available in any of the repositories. Individuals may create a package and make it available online for others to download and install it manually. The package may be in one of the following formats: .tar.gz or .tgz. These packages always come with README files. Always make a point to read this file before performing any operation related to installation. This file is self-explanatory and has the directions to install the particular package onto your machine.
Let's discuss the three main categories of updates that determines how you keep your system up-to-date and working in optimum conditions. The three broad categories we will discuss here are security updates, kernel updates, and application updates.
It is recommended that server administrators install the security patch updates as soon as it becomes available. However, if a particular package isn't being used in the server, then you may skip the security update for that particular package. Depending on your need, you may choose and install the security updates.
Administrators can refer to Ubuntu Security Notices (USN) to understand the security updates available and if they should be installing a particular update. USN can be accessed at http://www.ubuntu.com/usn/. The admins can also subscribe to RSS feeds for these security updates.
It may seem like kernel updates are of the utmost importance and they should be applied as soon as the updates are available. But it is tricky, as updates to kernel can actually cause some programs not to function in a proper way. Kernel updates rectify the security issues but have to be updated only after considering and making sure that other applications' behavior won't be hampered with it.
By default, the package management tools such as apt-get, aptitude, and synaptic enable the kernel updates. For command-line tools, you can disable automatic kernel updates using the following command:
sudo aptitude hold linux-image-name
Admins need to be careful when installing the application updates. A case may arise wherein some of the functionality of other packages may not work after installing the update. Refer to USN and other Ubuntu related forums to keep yourself updated about the possible application updates and the issues faced by other users.
The unattended-upgrades package can be used to schedule and enforce rules on what packages to be updated and what packages should be restricted from upgrading itself. To use this, install the unattended-upgrades package using the following command:
sudo apt-get install unattended-upgrades
Once the package is installed, configure the unattended-upgrades package in the following file and make necessary changes as per your needs. I am opening the file in the Nano editor:
nano /etc/apt/apt.conf.d/50unattended-upgrades
The output of this command is shown in the following screenshot:
The first section is for the packages where you want to allow automatic upgrades. Here, the first line for security is enabled, so the security updates will be installed by default. The // symbol means the lines are commented and will not be evaluated, as shown here:
Unattended-Upgrade::Allowed-Origins { "Ubuntu precise-security"; // "Ubuntu precise-updates"; };The second section is for the blacklisted packages. You put the package names here for which you want to disable automatic updates. Again, // has the same meaning—those lines will not be evaluated. Add the packages in this section to blacklist them and disallow automatic updates:
Unattended-Upgrade::Package-Blacklist { // "vim"; // "libc6"; // "libc6-dev"; // "libc6-i686"; };One last step is to enable the automatic updates by setting the appropriate configuration in the /etc/apt/apt.conf.d/10periodic file, as shown in the following screenshot:
In this example, the interval is set to 1, which means the packages list will be updated, packages will be downloaded, and they will be upgraded every day.
When there are multiple machines on your network, it makes sense to have local repository mirrored from the remote repository. The main advantage is the reduced bandwidth usage by limiting the number of machines connecting to the remote repositories and downloading huge files associated with packages. Only one system will connect to the remote repository and mirror the contents to the local system. Another advantage is that when all the machines are pointed to this repository the packages in each one of them will be of the same version.
There are three tasks that we will need to perform in order to get the local mirror repository working. First, set up the mirror on one of the machines, and then configure the machine to mirror only certain repositories. This will limit the amount of data downloaded and also keep the unwanted packages away. The third task is to point the local clients to this newly created repository so that they use only this repository to upgrade the packages.
You can set up the mirroring in two ways: rsync and apt-mirror. Traditionally, rsync was being used to synchronize the local and remote files. Now, admins prefer the apt-mirror command, at least for the initial mirroring setup. First, install the apt-mirror package using the following command:
sudo apt-get install apt-mirror
Now, try running the apt-mirror command. Don't be surprised by the sheer size of download data it asks to be downloaded. Now, let's limit the repositories that we want and also set up other configuration for the mirror.
You will be required to configure the /etc/apt/mirror.list file to set up the desired repositories you wish to mirror. The file looks like this:
The first section has the default configuration related to the storage of downloaded files, base mirror path, cleanscript, concurrent threads to be run when downloading, and others. By default, the files are copied to the /var/spool/apt-mirror path. The admin should ensure there is enough disk space to store the files. If all the systems are on the same architecture, then let defaultarch be same as default, which is set to <running host architecture>. If you are mirroring the repositories for any other architecture, then you need to set it explicitly. Next, comment out the lines related to the repositories in the second section. Have only those repositories active that you need and comment out the others as keeping all active will mean you are downloading a very large amount of data, somewhere close to 50 GB maybe.
After you have made the changes, run the apt-mirror command to mirror the remote repository to the local one. Once completed, you can see the directory listing similar to the one you observe on the remote servers.
This is an important task if you want to make sure the local mirror is completely functional. In each of the client machines on your network, change the repository URL to access the locally set up mirror. You will need to point to the IP of the machine acting as the mirror. Open the /etc/apt/sources.list configuration file and make the following changes:
deb http://192.168.1.8/mirror/ubuntu trusty main restricted
We are all set to use the mirror as the repository for all our local client machines.
In this chapter, we discussed various package management tools such as dpkg, apt-get, and aptitude. You learned about the limitations of dpkg, how apt overcomes these limitations, and the ease of using aptitude. You also learned about auto updates and configuring the repositories as per your requirements. Later, we covered how to create, maintain, and use the local mirrors for your cluster.
In the next chapter you will learn about the network and DNS, as well as how to set up and maintain them.
