Download code from GitHub
Following the first industrialrevolution, automation was introduced, making the already-efficient machinery that had been developed even more efficient. This led to the introduction of industrial constructions, vehicle steering and stabilization, and indoor environmental control, among many other developments. After this, the information revolution kicked in, initiating a new process of optimization. This phase is working to reduce human intervention in technological processes and raise productivity.
Nowadays, automation has become the norm across all fields. It started with simple administrator scripts, written to simplify and speed up daily tasks, and quickly developed into fully-fledged configuration management tools. The reasons behind this rapid development were the increase in market demand, the expansion of infrastructure and applications, and the emergence of new technologies, such as continuous integration, continuous development, and machine provisioning, that require a much more complicated setup and configuration.
By nature, systems and network administrators tend to want to reduce repetitive tasks, simplify complicated ones, and try to move on to the next task as quickly as possible. At first, there were a few simple scripts, such as Bash or PowerShell, that were able to optimize tasks in a standard environment. After that, longer and more complicated scripts that involved advanced programming languages such as Python or Ruby were developed. These aimed to address tasks across multiple platforms or in complex environments and to manage infrastructure using automation and orchestration tools, enabling businesses to grow dramatically overnight with more demanding and complicated services. The role of administrators is to manage this growth and act accordingly to ensure a seamless user experience.
This chapter will provide an overview of Ansible. We will demonstrate that Ansible is now a must-have platform for managing a medium to large infrastructure, rather than having a physical, partially virtual or hybrid, private and public cloud. Other automation tools offer different benefits with regard to their installation, usage, speed, and flexibility, so it can be tricky for a first-time user to choose the most appropriate automation tool for their environment. Ansible, Chef, Puppet, and SaltStack are the major configuration management tools available on the market. Each of these follows a different method of deploying, configuring, and managing machines with reduced complexity and increased speed, reliability, and compliance. This chapter willcover the following topics:
- Market study of automation tools
- Introduction to Ansible as a configuration management and task orchestration tool
- Exploration of Ansible's functionalities across operating systems, architectures, and cloud platforms
- Overview of the Ansible project and Tower
The major configuration management tools currently used in the market are Ansible, Chef, Puppet, and SaltStack. Each one of these has their own pros and cons, so finding the right one can be a bit challenging, depending on which features are valued or which programming language is preferred. In this section, we will briefly introduce each of the tools and explain why we have chosen Ansible in this book.
Chef is an open source, client-server configuration management tool. It offers a flexible infrastructure automation framework using Ruby and domain-specific language (DSL) for the administration of hosts. This covers all types of hosts, including bare metal, virtual, or on the cloud. Chef is very common with code developers due to its flexibility, stability, and reliability in large cloud deployments. However, it can be challenging to set up and learn its functionalities, so it might take a new user some time before properly mastering it.
Puppet is a Ruby-based configuration management and orchestration tool. It follows an agent/master architecture, where the hosts to be controlled require a Puppet agent to allow their management. Puppet features a strong automation and reporting capability, via its UI interface, for task submission and host real-time reporting. Like Chef, Puppet can be challenging for new users to set up and configure. A prior knowledge of Ruby and DSL is required to perform personalized and complex tasks.
Note
Puppet and Chef are two of the oldest configuration management platforms. They both use Ruby and DSL to control their agents.
SaltStack is a Python-coded platform built to allow high-speed, master-agent communication. Its configuration management tasks are coded in Yet Another Markdown Language(YAML). The master (or multiple masters) uses the SSH protocol to control the agents/minions. SaltStack is very scalable, meaning it can respond well to environmental changes, it is easy to use, and it has a strong community. On the other hand, its installation can be difficult for a new user, its UI is not well-developed, it focuses on Linux with an average cover of other operating systems, and its documentation lacks good management.
Note
SaltStack is very similar to Ansible. They both employ easy-to-use coding languages, that is, Python and YAML. Also, both SaltStack and Ansible execute tasks very quickly because they rely on SSH to send comments to the hosts.
Ansible is a relatively new tool compared to the others. It was built to simplify the complexity of task automation and orchestration. It is built on Python and uses YAML for scripting its jobs, which is a language that is very simple and close to English. This allows new users to understand it easily and write it themselves. Ansible does not require an agent to be installed in the hosts. It supports both push and pull models to send commands to its Linux nodes via the SSH protocol, and the WinRM protocol to send commands to its Windows nodes. It allows for a seamless deployment and provisioning for both VMs, applications, and containers, and scales easily to match the environment growth. It is simple to install and configure, and it is fairly easy to learn how to use it and code its scripts. Ansible does not require agent installation, which improves its communication speed. It is predominantly advanced in configuration management tasks, but it can also behave as an infrastructure orchestration tool. However, it requires extra permission for the master nodes. Users can easily end up with multiple scripts for multiple tasks, which can get confusing, and it lacks a good GUI and a mature platform when compared to older tools.
Each of these tools is built for a specific audience. They have many well-developed features to cover a user's unique requirements to either simplify their daily tasks, improve productivity, speed up a host configuration, or close the gap in a hybrid environment.
We have chosen to cover Ansible in this book to make it future-proof. We can all agree that Ansible is a new platform, so it is not as well-designed and customizable as many of the other tools, but it is easy to see how fast Ansible is on the rise. We are not just talking about the number of new technologies it supports, the number of modules it is introducing and enhancing, the community support that is present on the Ansible Galaxy form, or the GitHub project forks and starred repositories. We are also following its popularity and its demand within the market.
Note
Red Hat acquired Ansible in October 2015 with the strong belief that Ansible is the leader in IT automation and DevOps delivery, with the ability to simplify management for hybrid clouds, OpenStack environments, and container-based services.
"Ansible is a clear leader in IT automation and DevOps, and helps Red Hat take a significant step forward in our goal of creating frictionless IT."
– Joe Fitzgerald, Vice President, Management, Red Hat
Ansible is being used more frequently than ever, as shown in the following diagram, which shows the number of downloads of the main package for each of the tools from the Debian repository per year:
The previous figure was an Ansible, Puppet, Chef, and SaltStack Popularity contest statistics on the Debian repository. It was generated using the following link https://qa.debian.org/popcon-graph.php?packages=ansible%2C+puppetmaster%2C+salt-master%2C+libchef-ruby&show_installed=on&want_legend=on&want_ticks=on&from_date=2010&to_date=2018&hlght_date=&date_fmt=%25Y-%25m&beenhere=1. This link can be used to generate time graphs about other Debian packages through time.
Ansible is a leading orchestration platform that allows for automation, host configuration management, and the deployment of applications and virtual machines. Ansible can automate a range of IT infrastructure features, from simple, daily, and repetitive tasks to machine provisioning or the continuous integration and deployment of DevOps applications. It is very flexible and can cover bare-metal machines, virtual machines and platforms, and public or private cloud environments. Ansible can also manage network devices from switches, routers, and firewalls. It can also cover the setup of applications, the configuration and behavior of database management systems, package managers, and simple user applications:
Ansible logo
Note
If this is your first book about configuration management and you are looking for an easy and simple method to follow, you are in the right place.
One command is enough to install Ansible on Linux using the system's package manager from the distribution repository. Another way is to use Python's PyPI package manager for a faster and simpler installation. After that, it can simply be used in a similar way to execute any command. We would recommend going an extra step for a more complex or larger environment by editing the Ansible configuration file, so that it reads /etc/ansible/ansible.conf, filling in the inventory, and adding some group variables. Ansible does not require an agent installation on the client, but with a few extra steps, the connection can be made more secure. Ansible uses YAML, a simple configuration management language for Ansible playbooks, which is a human-readable coding language, so scripts can be written without much difficulty. When sending commands for specific tasks, the Ansible server translates the YAML code to the actual configuration code to the clients for immediate execution.
Note
For most of the tutorials in this book, Debian-based systems will be used for the servers and Linux clients. The location and package names of the configuration files may vary between distributions.
The Ansible host server is the only machine where recommended computing resources are to be met in order to run the engine correctly. As it is agentless, the clients only receive tasks in the form of commands that get executed on the system directly. Some Ansible modules may consume a fair amount of the network traffic by sending data from one machine to another. This is the lowest amount of traffic required to carry out a task, since Ansible only uses a tiny portion of traffic to submit the command to the hosts.
The rapid growth of Ansible has made it a very powerful tool. It is now considered the leading automation engine in the market. With its huge community support (Ansible Galaxy and GitHub projects) and Red Hat's proprietary management add-ons (Ansible Tower), its users have a wide variety of roles, modules, and add-ons to choose from that can automate every conceivable IT task.
Ansible offers its users the following features:
- Systems configuration management
- Agile application deployment following the best DevOps practices
- Simplified orchestration and automation
- Zero-downtime, continuous deployment
- Support for cloud-native applications
- Simple and optimized container adoption
- Embedded security and compliance policy in automated tasks
- Streamlined host provisioning
- Support for multi-tier deployment
- Support for heterogenic IT infrastructures
- Support for multi-layered computer architecture
- Support for infrastructure-as-a-service (IaaS) deployment
- Support for platform-as-a-service (PaaS) deployment
- Support for scalability for a fast-growing environment
- Support for push and pull models for task execution
- Fast host fact-sharing between servers for better redundancy and performance
- Configuration for a variety of network devices
- Management and monitoring of storage devices
- Control of database management systems
With the rapid growth of IT infrastructures and a shift in the way applications are being deployed, IT administrators' tasks have grown in scale and complexity. Ansible seamlessly merges orchestration and configuration management in a very handy platform that allows IT administrators to define a selected number of nodes, applications, and network devices to be configured in a desired state by making clear which actions should be taken to remove repetition and reduce complexity. Ansible can be used in a variety of ways, which we will cover in the next section.
As well as configuration management, Ansible also offers high-end orchestration. This makes the organization and management of the interactions between multiple configuration tasks well-structured. It simplifies and orders complex and chaotic configuration management and administration tasks. According to the status of the infrastructure, and the users' demands, applications, and data-versioned behaviors, Ansible orchestration will generally bring the infrastructure back to the desired state by configuring the appropriate services and policies via the CM tool into the failed component and make it work properly.
IT orchestration can get very complex when dealing with DevOps class tasks, such as the continuous integration and deployment (CI/CD) of applications or infrastructure as a code (IaC). Ansible is capable of converting those tasks to automated workflows that run a number of playbooks in a well-defined structure, featuring all sorts of Ansible pre-defined modules, plugins, and APIs to communicate, execute commands, and report facts from any number of hosts, devices, and services.
Ansible is the path to take for better infrastructure automation, application deployment, and provisioning. It is the open source approach to an automated and modernized IT environment. Ansible is the key to enabling IT administrators to automate their daily tasks, freeing up their time to allow them to focus on delivering quality services. This not only impacts the IT department, but the business as a whole. The following diagram shows the reach of Ansible's multiple functionalities:
Instance provisioning using Ansible covers the configuration and setup of bare-metal machines and servers. It relies on its predefined APIs to create and configure the local virtualized infrastructure. It can also manage hybrid, private, and public cloud instances, resources, and applications. Ansible can automatically install and configure an application and its libraries. It uses OS bootstrap and a kickstart script to initiate bare-metal machine provisioning using very simple playbooks and built-in modules. Using the same simple playbooks and different modules, Ansible can also provision instances, networking, and VMs in a public, private, or hybrid cloud very easily.
Using the power of playbooks and inventory, IT administrators can use Ansible to execute an update, patch, or configuration modification on a number of hosts, network devices, and applications. Playbooks describe the infrastructure in both simple, human-readable terms for other people to use, and machine-parsable code that can be used on any machine running Ansible. The execution of an Ansible configuration is state-driven, which means that it does not need to check the system or service state to know how to adapt in order to increase the task's reliability.
When we talk about applications that are managed by Ansible, we are talking about full life cycle control. Any users who have access to the Ansible server node, from IT administrators to application developers and project managers, will be able to manage all aspects of the application. Ansible takes the application package, deploys it to all the production servers, sets it up, and configures and initiates it. It can even test the package and report its status. This feature covers multi-tier applications, allowing zero-downtime rolling for a seamless application update.
Ansible ensures a stable environment for both developers and IT administrators for the continuous delivery and integration of applications. Automating as much as possible of the application turnaround means it is quick and unnoticeable to the application users. Ansible automation and orchestration is multi-tier and multi-step, which allows for finer control over operations and hosts. We can write Playbooks to manage the continuous integration and delivery of applications while ensuring the desired state of various components, such as load balancers and several server nodes.
After being bought by Red Hat, Ansible continued to offer a free open source platform, which is currently called the Ansible Project. Red Hat has created proprietary management add-ons that offer an advanced control and centralization of the infrastructure, called Ansible Tower. Red Hat runs the Ansible Automation platform, which is composed of the Ansible Engine and Ansible Tower. This product is fully supported by Red Hat as one of its lead projects.
The Ansible project is a build-up of functionalities that come from the original company, AnsibleWorks. It is a community-built automation engine. It is free, open source, and available for anyone to download or install on any Linux OS, using the package manager, source compiling, or Python PyPI. It is very simple, powerful, and agentless.
To use the Ansible automation engine, users do not need any third-party applications or interfaces. They can simply send a command or write a playbook and execute it directly to the engine. This allows the user to access a variety of predefined modules, plugins, and APIs working as building blocks for managing all kinds of IT tasks and network objects. As it is agentless, Ansible relies on SSH to manage the Linux hosts, and WinRM for the Windows hosts. The SSH protocol is also used to control some of the network devices. Some more unsual devices or cloud and virtualization services require the use of Ansible pre-defined APIs to help manage or access them.
Nodes can be defined by their IP addresses or hostname; for the latter, we will have to rely on a DNS server or the local DNS file. APIs are used to communicate with third-party services, such as public or private clouds. Modules, which constitute Ansible's biggest pre-defined function library, allow the users to simplify long and complex tasks into a few lines in a playbook. They cover a large number of tasks, systems, packages, files, datastores, API calls, network device configurations, and so on. Finally, Ansible plugins are used to improve Ansible's core functionality, such as fast host caching, to avoid facts gathering on the network.
Ansible Tower is the Red Hat proprietary layer that sits on top of the Ansible project engine. It is made up of a number of add-ons and modules, composed of REST APIs and web services, that work together to create a friendly web interface that acts as an automation hub from which the IT administrator can select a number of tasks or playbooks to be executed on a number of machines. It still relies on the Ansible Engine to send commands and collect the reports. Ansible Tower cleverly collects the status of tasks and the reports that come back from hosts. All of this data is presented in the Ansible dashboard, showing hosts, the status of the inventory, and the recent jobs, activities, and snapshots:
Ansible Tower scales as the environment grows, and acts accordingly by showing in real-time all the statuses of the hosts, tasks, and playbooks. It highlights the successful playbook jobs, as well as those that failed to run, in order to troubleshoot any issues. In its multi-playbook workflows, the user can create pipelines of playbooks to be executed in sequence on any type of inventory, using one or more users' credentials and on a personalized timescale. With pipelining enabled, an IT administrator can automate complex operations (application provisioning, continuous deployment with containers, running test workflows) by breaking them down into smaller tasks using pipelines and, depending on the output (success or failure), run a specific play.
Ansible Tower offers a smart inventory platform that enables you to pull the host's inventory from any source, including a public or private cloud, or a local CMDB. The smart inventory builds hosts caching, which allows the user to run playbooks based on the facts of the hosts, which are pieces of information and properties related to them and gathered by Ansible. It also allows you to set up built-in notifications about the status of tasks, workflows, and playbooks via email, SMS, and push notifications on third-party platforms, such as Slack or Hipchat. Ansible Tower also allows task scheduling for routine updates, device patching, and custom backup schedule options. The following diagram shows the layers of the full Ansible Engine provided by Red Hat:
Note
Currently, Red Hat Ansible Tower offers a 30-day trial license for a hands-on exploration and test of its features. Any user can use it to decide if their environment will benefit from it.
In this book, we will mainly focus on the open source Ansible Engine, as it is free and accessible to everyone. We believe that learning Ansible has to be started with the basic no-UI version to better understand the inner mechanics of the tool. The reader can migrate easily to Ansible Tower with the engine skills that they have already acquired.
Note
There are a number of open source solutions that provide some of the functionalities of Ansible Tower. The most common of these is Ansible Semaphore, which is available at https://github.com/ansible-semaphore/semaphore.
In this chapter, we introduced Ansible and listed its key features and the advantages that it can offer to the user when employed properly. We have also discussed the enterprise version of Ansible Tower, which is developed and supported by RedHat. In Chapter 2, Ansible Setup and Configuration, we are going to start the real learning. We will begin by setting up Ansible and showing the best ways of configuring it to take it for a test drive.
This chapter's references are as follows:
- Ansible website: https://www.ansible.com/
- Red Hat website: https://www.redhat.com/en/technologies/management/ansible
- Puppet website: https://puppet.com/
- Chef website: https://www.chef.io/chef/
- SaltStack website: https://saltstack.com/
