Security Automation with Ansible 2: Leverage Ansible 2 to automate complex security tasks like application security, network security, and malware analysis

A playbook, in the classic sense, is about offensive and defensive plays in football. The players keep a record of the plays (plan of action) in a book, usually in the form of a diagram.

In Ansible, a playbook is a series of ordered steps or instructions for an IT process. Think of a nicely-written instruction manual that can be read and understood by humans and computers alike. 

In the subsequent chapters, all the automation we will focus on regarding security will lead us toward building both simple and complex playbooks. 

This is what an Ansible playbook command looks like:

ansible-playbook -i inventory playbook.yml

Ignore the -i flag for now and notice the extension of the playbook file. 

As stated in http://docs.ansible.com/ansible/playbooks_intro.html:

Ansible ships with a number of modules (called the module library) that can be executed directly on remote hosts or through playbooks.Tasks in playbooks call modules to do the work. 

Ansible has many modules, most of which are community contributed and maintained. Core modules are maintained by the Ansible core engineering team and will always ship with Ansible itself.

Modules can be executed via the command line as well. We will be using modules to write all the tasks inside our playbooks. All modules technically return JSON format data. 

Documentation for each module can be accessed from the command line with the ansible-doc tool:

$ ansible-doc apt

We can list all the modules available on our host:

$ ansible-doc -l

Start the Apache web server on all nodes grouped under webservers by executing the httpd module. Note the use of the -m flag:

$ ansible webservers -m service -a "name=httpd state=started"

This snippet shows the exact same command but inside a playbook in YAML syntax:

-name:restart webserverservice:name:httpdstate:started

Each module contains multiple parameters and options, get to know more about the features of the modules by looking at their documentation and examples.

Ansible playbooks are written in YAML, which stands for YAML Ain't Markup Language. 

According to the official document (http://yaml.org/spec/current.html):

Ansible uses YAML because it is easier for humans to read and write than other common data formats, such as XML or JSON. All YAML files (regardless of their association with Ansible or not) can optionally begin with --- and end with .... This is part of the YAML format and indicates the start and end of a document.

Here is an example of a simple playbook to showcase YAML syntax from Ansible documentation (http://docs.ansible.com/ansible/playbooks_intro.html#playbook-language-example):

-hosts:webserversvars:http_port:80max_clients:200remote_user:root
tasks:-name: Ensure apache is at the latest versionyum:
      name: httpd
      state: latest-name: Write the apache config filetemplate:
      src: /srv/httpd.j2
      dest: /etc/httpd.conf
notify:-restart apache
-name: Ensure apache is running (and enable it at boot)service:
      name: httpd
      state: started
      enabled: yes
handlers:-name: Restart apacheservice:
        name: httpd
        state: restarted

While playbooks offer a great way to execute plays in a pre-defined order, there is a brilliant feature on Ansible that takes the whole idea to a completely different level. Roles are a convenient way to bundle tasks, supporting assets such as files and templates, coupled with an automatic set of search paths.

By using a concept most programmers would be familiar with, of including files and folders and ascribing what is being included, a playbook becomes infinitely more readable and understandable. Roles are basically made up of tasks, handlers, and configurations, but by adding an additional layer to how a playbook is structured, we can easily get the big picture overview as well as the low-level details. 

This allows for reusable code and a division of work in a team tasked with writing playbooks. For example, the database guru writes a role (almost like a partial playbook) for setting up the database and the security guru writes one on hardening such a database.

The benefits of using roles while building large playbooks include:

This is an example of what a possible LAMP stack site.yml can look like:

- name: LAMP stack setup on ubuntu 16.04
  hosts: all
  gather_facts: False
  remote_user: "{{remote_username}}"
  become: yes

 roles:
   - common
   - web
   - db
   - php

Note the list of roles. Just by reading the role names we can get an idea of the kind of tasks possibly under that role. 

Ansible uses Jinja2 templating to enable dynamic expressions and access to variables. Jinja2 variables and expressions within playbooks and tasks allow us to create roles that are very flexible. By passing variables to a role written this way, we can have the same role perform different tasks or configurations. Using a templating language, such as Jinja2, we are able to write playbooks that are succinct and easier to read.

By ensuring that all the templating takes place on the Ansible controller, Jinja2 is not required on the target machine. Only the required data is copied over, which reduces the data that needs to be transferred. As we know, less data transfer usually results in faster execution and feedback. 

tasks:-name:"shutdownDebianflavoredsystems"command:/sbin/shutdown -t nowwhen:ansible_os_family == "Debian"

-name:add several usersuser:name:"{{item.name}}"state:presentgroups:"{{item.groups}}"with_items:-{name:'testuser1',groups:'wheel'}-{name:'testuser2',groups:'root'}

We will look at how to write a LAMP stack playbook using the skills we have learned so far. Here is the high-level hierarchy structure of the entire playbook:

inventory               # inventory file
group_vars/             #
   all.yml              # variables
site.yml                # master playbook (contains list of roles)
roles/                  #
    common/             # common role
        tasks/          #
            main.yml    # installing basic tasks
    web/                # apache2 role
        tasks/          #
            main.yml    # install apache
        templates/      #
            web.conf.j2 # apache2 custom configuration
        vars/           # 
            main.yml    # variables for web role 
        handlers/       #
            main.yml    # start apache2
    php/                # php role
        tasks/          # 
            main.yml    # installing php and restart apache2
    db/                 # db role
        tasks/          #
            main.yml    # install mysql and include harden.yml
            harden.yml  # security hardening for mysql
        handlers/       #
            main.yml    # start db and restart apache2
        vars/           #
            main.yml    # variables for db role

Let's start with creating an inventory file. The following inventory file is created using static manual entry. Here is a very basic static inventory file where we will define a since host and set the IP address used to connect to it.

Configure the following inventory file as required:

[lamp]
lampstack    ansible_host=192.168.56.10

The following file is group_vars/lamp.yml, which has the configuration of all the global variables:

remote_username: "hodor"

The following file is the site.yml, which is the main playbook file to start:

- name: LAMP stack setup on Ubuntu 16.04
 hosts: lamp
 gather_facts: False
 remote_user: "{{ remote_username }}"
 become: True

 roles:
   - common
   - web
   - db
   - php

The following is the roles/common/tasks/main.yml file, which will install python2, curl, and git:

# In ubuntu 16.04 by default there is no python2
- name: install python 2
  raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal)

- name: install curl and git
  apt:
    name: "{{ item }}"
    state: present
    update_cache: yes

  with_items:
    - curl
    - git

The following task, roles/web/tasks/main.yml, performs multiple operations, such as installation and configuration of apache2. It also adds the service to the startup process:

- name: install apache2 server
  apt:
    name: apache2
    state: present

- name: update the apache2 server configuration
  template: 
    src: web.conf.j2
    dest: /etc/apache2/sites-available/000-default.conf
    owner: root
    group: root
    mode: 0644

- name: enable apache2 on startup
  systemd:
    name: apache2
    enabled: yes
  notify:
    - start apache2

The notify parameter will trigger the handlers found in roles/web/handlers/main.yml:

- name: start apache2
  systemd:
    state: started
    name: apache2

- name: stop apache2
  systemd:
    state: stopped
    name: apache2

- name: restart apache2
  systemd:
    state: restarted
    name: apache2
    daemon_reload: yes

The template files will be taken from role/web/templates/web.conf.j2, which uses Jinja templating, it also takes values from local variables:

<VirtualHost *:80><VirtualHost *:80>
    ServerAdmin {{server_admin_email}}
    DocumentRoot {{server_document_root}}

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

The local variables file is located in roles/web/vars/main.yml:

server_admin_email: hodor@localhost.local
server_document_root: /var/www/html

Similarly, we will write database roles as well. The following file roles/db/tasks/main.yml includes installation of the database server with assigned passwords when prompted. At the end of the file, we included harden.yml, which executes another set of tasks:

- name: set mysql root password
  debconf:
    name: mysql-server
    question: mysql-server/root_password
    value: "{{ mysql_root_password | quote }}"
    vtype: password

- name: confirm mysql root password
  debconf: 
    name: mysql-server
    question: mysql-server/root_password_again
    value: "{{ mysql_root_password | quote }}"
    vtype: password

- name: install mysqlserver
  apt:
    name: "{{ item }}"
    state: present 
  with_items:
    - mysql-server
    - mysql-client

- include: harden.yml

The harden.yml performs hardening of MySQL server configuration:

- name: deletes anonymous mysql user
  mysql_user:
    user: ""
    state: absent
    login_password: "{{ mysql_root_password }}"
    login_user: root

- name: secures the mysql root user
  mysql_user: 
    user: root
    password: "{{ mysql_root_password }}"
    host: "{{ item }}"
    login_password: "{{mysql_root_password}}"
    login_user: root
 with_items:
   - 127.0.0.1
   - localhost
   - ::1
   - "{{ ansible_fqdn }}"

- name: removes the mysql test database
  mysql_db:
    db: test
    state: absent
    login_password: "{{ mysql_root_password }}"
    login_user: root

- name: enable mysql on startup
  systemd:
    name: mysql
    enabled: yes

  notify:
    - start mysql

The db server role also has roles/db/handlers/main.yml and local variables similar to the web role:

- name: start mysql
  systemd:
    state: started
    name: mysql

- name: stop mysql
  systemd:
    state: stopped
    name: mysql

- name: restart mysql
  systemd:
    state: restarted
    name: mysql
    daemon_reload: yes

The following file is roles/db/vars/main.yml, which has the mysql_root_password while configuring the server. We will see how we can secure these plaintext passwords using ansible-vault in future chapters:

mysql_root_password: R4nd0mP4$$w0rd

Now, we will install PHP and configure it to work with apache2 by restarting the roles/php/tasks/main.yml service:

- name: install php7
  apt:
    name: "{{ item }}"
    state: present
  with_items:
    - php7.0-mysql
    - php7.0-curl
    - php7.0-json
    - php7.0-cgi
    - php7.0
    - libapache2-mod-php7

- name: restart apache2
  systemd:
    state: restarted
    name: apache2
    daemon_reload: yes

To run this playbook, we need to have Ansible installed in the system path. Please refer to http://docs.ansible.com/ansible/intro_installation.html for installation instructions. 

Then execute the following command against the Ubuntu 16.04 server to set up LAMP stack. Provide the password when it prompts for system access for user hodor:

$ ansible-playbook -i inventory site.yml

After successful completion of the playbook execution, we will be ready to use LAMP stack in a Ubuntu 16.04 machine. You might have observed that each task or role is configurable as we need throughout the playbook. Roles give the power to generalize the playbook and customize easily using variables and templating.