Learning Puppet Security

At the core of Puppet is software that allows you to specify the state of the system and let Puppet get the system there. It differs from many of the other products in the configuration management space due to its declarative nature.

In a declarative system, we model the desired state of the resources (things being managed).

Declarative systems have the following properties:

Imperative systems, on the other hand, use algorithms and steps to express their desired state. Most traditional programming languages, such as C and Java, are considered imperative. Imperative systems have the following properties:

In Puppet, which is declarative, the users can describe how they want the system to look in the end, and leave the implementation details of how to get there up to the types and providers within Puppet. Puppet uses types, which represent resources, such as files or packages. Each type can optionally be implemented by one or more provider.

Types provide the core functionality available in Puppet. The type system is extensible, and additional types can be added using pure Ruby code. Later on in this chapter, we'll use the file and package types in our example.

Providers include the code for the type that actually does the low level implementation of a resource. Many types have several providers that implement their functionality in different ways. An example of this is the package type. It has providers for RPM, Yum, dpkg, Windows using MSI, and several others. While it is not a requirement that all types have multiple providers, it is not uncommon to see them, especially for resources that have different implementation details across operating systems.

This system of types and providers isolates the user from having to have specific knowledge of how a given task is done. This allows them to focus on how the system should be configured, and leave specific implementation details, such as how to put it in that state, to Puppet.

A few tools, such as Chef, actually use more of a hybrid approach. They can be used in a declarative state, but also allow the use of loops and other flow control structures that are imperative. Puppet is slowly starting to gain some support for this in their new future parser, however these are experimental and advanced features at this point.

While the declarative approach may have a larger learning curve, especially around dependency management, many sysadmins find it a much better fit with their way of thinking once they learn how it works.

Puppet uses a client-server model in the most common configurations. In this mode, one or more systems, called Puppet Masters, contain files called manifests. Manifests are code written in the Puppet DSL. A DSL is a language designed to be used for a specific application. In this case, the language is used to describe the desired state of a system. This differs from more general purpose languages, such as C and Ruby, in that it contains specialized constructs for the problem being solved. In this case, the resources in the language are specific to the configuration management domain.

Manifests contain the classes and resources which Puppet uses to describe the state of the system. They also contain declarations of the dependencies between these resources.

Classes are often bundled up into modules which package up classes into reusable chunks that can be managed separately. As your system becomes more complicated, using modules helps you manage each subsystem independently of the others.

The client systems contain the Puppet agent, which is the component that communicates with the master. At specified run intervals (30 minutes by default), the agent will run and the following actions will take place:

The catalog, sent to the client by the master, contains a compiled state of the system resources of the client. The client then applies this information using types and providers to bring the system into the desired state. The following illustration shows how data flows between the components:

It is also possible to run Puppet in a masterless mode. In this mode, the Puppet manifests and other needed components, such as custom facts, types, and providers, are distributed to each system using an out of band method, such as scp or rsync. Puppet is then applied on the local node using cron or some other tool.

cron has the advantage of not requiring the server setup with open ports that the master-based setup has. In some organizations, this makes it easier to get past information security teams. However, many of the reporting and other benefits we will explore in this book are less effective when run in this fashion. The book Puppet 3: Beginners Guide, John Arundel, Packt Publishing, has a good amount of information about such a masterless setup.

Puppet has a number of other components that form part of the Puppet ecosystem, which are worth exploring due to their use as security tools. The specific components we are going to explore here include PuppetDB and Hiera.

PuppetDB is an application used to store information on the Puppet infrastructure. Released in 2012, PuppetDB solved performance issues present in the older storeconfigs method that stored information about Puppet runs.

PuppetDB allows you to store facts, catalogs, reports, and resource information (via exported resources). Mining this data, using one of the reporting APIs, is an easy and powerful way to get a view of your infrastructure. More information on PuppetDB will be presented in Chapter 3, Puppet for Compliance, as well as Chapter 4, Security Reporting with Puppet.

Hiera was a new feature introduced in Puppet 3. It is a hierarchal data store, which helps to keep information about your environment. This allows you to separate data about the environment from code that acts on the environment. By doing so, you can apply separate security policies to the code that drives the environment and data about the systems.

Before Hiera, it was not uncommon to see large sections of Puppet code dedicated to maintaining sites or installation of specific information on the systems under management. This area was often difficult to maintain if the ability to override parameters using many different factors was needed.

By adding a hierarchy that can depend on any facts, it becomes much easier to store the data needed for the systems under management. A model of most specific to least specific can then be applied, which makes it much easier to override the default data at a site, environment, or system level.

For example, let's say you had a set of development environments where a certain group of development accounts needed to get created, and SSH access to those accounts was granted. However, these accounts and the access granted should only exist in the development machines, and not in production. Without Hiera, there would likely be site-specific information in the modules to manage the SSH configuration, and perhaps in the user creation module to manage the users. Using Hiera, we can add a fact for the type of system (production or development) and store which users get created there, or have access. This moves the list of users with access to the system out of the code itself, and into a data file.

As our examples get more complicated later in this book, we will explore using Hiera to store some system data.

The currently recommended way to install Puppet on CentOS machines is to use the Puppet Labs Yum repository. This repository, which can be found at https://yum.puppetlabs.com, contains all the Puppet Labs software as well as the dependencies required to install them, such as several Ruby gems not present in the main CentOS repository. On installation, Ruby and these dependencies will also be installed.

Adding this repository is relatively simple. Execute the following command as a root (or using sudo, as shown here):

sudo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

After running this command, you will see an output similar to this:

Retrieving https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
Preparing...                ########################################### [100%]
   1:puppetlabs-release     ########################################### [100%]

Once this is complete, you're done! The Puppet Labs repository is added and we can use it to install the current version of any of the Puppet Labs products.

The next step is to install the Puppet Master. As mentioned earlier, this system acts as the controller that all of your client agents will then use to communicate with to receive catalog information. This package is normally installed on only a few systems that act as servers for configuration management information.

Installing the master with the repository is as easy as executing the following command:

sudo yum -y install puppet-server

This will instruct yum to install the Puppet server without confirmation. The output will be as follows:

On all the systems that we wish to manage by using Puppet, we'll need to install the Puppet agent. This agent is a piece of software that is responsible for communicating with the master and applying changes.

Installing the Puppet agent is very easy and similar to installing the master in the preceding section. You simply run the following:

sudo yum -y install puppet

After this is complete, you'll see that the the Puppet agent is installed on the local machine and is ready to talk to the master.

Now that we have a perfectly working Puppet Master, we need to configure it. Installation of the packages will include a base level configuration. There are some changes we will want to make to the base Puppet configuration to enable some features that we'll use in the future. As we go through this book, we'll make changes to these files several times.

The main configuration files in use by Puppet are present in the /etc/puppet directory.

In this directory, there are a number of configuration files that control how Puppet behaves. Information on these files can be found at https://docs.puppetlabs.com/puppet/3.7/reference/config_about_settings.html. For now, we only need to concern ourselves with the Puppet configuration file.

Open the /etc/puppet/puppet.conf file with your favorite editor (make sure that you use sudo) and edit it to look similar to the following:

[main]
    # The Puppet log directory.
    # The default value is '$vardir/log'.
    logdir = /var/log/puppet

    # Where Puppet PID files are kept.
    # The default value is '$vardir/run'.
    rundir = /var/run/puppet

    # Where SSL certificates are kept.
    # The default value is '$confdir/ssl'.
    ssldir = $vardir/ssl

[agent]
    # The file in which puppetd stores a list of the classes
    # associated with the retrieved configuratiion.  Can be loaded in
    # the separate ``puppet`` executable using the ``--loadclasses``
    # option.
    # The default value is '$confdir/classes.txt'.
    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An
    # extension indicating the cache format is added automatically.
    # The default value is '$confdir/localconfig'.
    localconfig = $vardir/localconfig
    report = true
    pluginsync = true
[master]
    reports = store

We've made a handful of changes to the file from the default version and will cover them here.

The first change is adding the report = true section to the agent configuration section. This will cause clients to send reports containing information about the Puppet run. We'll use these reports for later analysis in Chapter 4, Security Reporting with Puppet.

The second change is to add pluginsync = true to the agent section. While this has become the default in the more recent versions of Puppet, it does not hurt to add it in. This causes the clients to sync custom facts, providers, and other Puppet libraries from the master. We will see how this is used in later chapters.

The final change we have made is to add the master section and add reports = store. This causes the master to save reports to the local filesystem on the Puppet Master. We'll use this later to do analysis of our Puppet runs for security-related purposes.

Both the Puppet Master and the agent are usually run as services. This allows the agent to check its run frequency and apply any changes. We've not explicitly started the services here, although we'll need to start the master in order to use it from our agent. To do this, we run the following command:

sudo service puppetmaster start

In order for the Puppet Master to start at boot, we'll also issue the following command to enable it to autostart:

sudo chkconfig puppetmaster on

It's pretty common to use Puppet to manage Puppet, and in a later chapter, we'll do this to show how we can use Puppet to secure the Puppet Master.

In order for us to use Vagrant, we must first install it. To do this, we need to install the required dependencies followed by Vagrant itself. We'll be using VirtualBox to host the virtual machines in these examples, since it is the most supported virtual machine provider.

VirtualBox can be downloaded from http://www.virtualbox.org. On this site, you will find packages for installing a variety of operating systems. You simply need to pick the package for your chosen operating system and install it using the instructions found on the site.

Once we have VirtualBox installed, we can approach installing Vagrant. Vagrant has several methods of installation. These methods include OS packages for Linux, as well as installers for OS X and Windows. Older versions of Vagrant supported installation via the Ruby gem utility, but this has been removed in later versions.

Vagrant can be found at http://www.vagrantup.com. Once you're there, you can download the package or installer for your OS. Once downloaded, you can install the package using your operating system's package manager, or by executing the downloaded package. In Windows and OS X, this is sufficient to have a working installation of Vagrant.

More in-depth installation instructions can be found on the Documentation tab on the Vagrant website; however, the package or installer will do most of the work.

It is worth noting that if you are using Windows, you will perform most of the work we're doing in a command shell on the DOS command box. However, if you use a local editor, you should be able to follow along with no issues.

Vagrant.configure(2) do |config|
  config.vm.define :puppetmaster do |master|

    master.vm.box = "centos65-x64-puppet"
    master.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-puppet.box"
    master.vm.hostname = "puppet.book.local"
    master.vm.network "private_network", ip: "10.78.78.30", netmask: "255.255.255.0"

    master.vm.provision "shell", inline: "yum –y update puppet"

    master.vm.provision "puppet" do |puppet|
      puppet.manifests_path = "master_manifests"
      puppet.manifest_file = "init.pp"
    end

  end
end

Vagrant.configure(2) do |config|

config.vm.define :puppetmaster do |master|

master.vm.box = "centos65-x64-puppet"

master.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-puppet.box"

master.vm.hostname = "puppet.book.local"

master.vm.network "private_network", ip: "10.78.78.30", netmask: "255.255.255.0"

master.vm.provision "shell", inline: "yum –y update puppet

master.vm.provision "puppet" do |puppet|

puppet.manifests_path = "master_manifests"

puppet.manifest_file = "init.pp"

end
end
end

package { 'puppet-server':
  ensure => 'present',
}

file { '/etc/puppet/puppet.conf':
  ensure  => 'present',
  owner   => 'root',
  group   => 'root',
  mode    => '0644',
  source  => '/vagrant/master_manifests/files/puppet.conf',
  require => Package['puppet-server'],
}

service { 'puppetmaster':
  ensure  => 'running',
  require => File['/etc/puppet/puppet.conf'],
}

.
├── Vagrantfile
└── master_manifests
    ├── files
    │   └── puppet.conf
    └── init.pp

Since this is our first time using Vagrant, we'll cover how to start a virtual machine. In the directory with the Vagrantfile, run the following command:

vagrant up

Once this is done, you'll see the output from Vagrant indicating the actions it's taking, as well as output from the commands it runs—this includes the Shell provisioner and the Puppet provisioner. When it's done, you'll end up with something that is similar to the following:

You'll notice some warnings on the screen here. These are options that are changing with the newer version of Puppet. Our manifest could add an allow_virtual setting to get rid of the second warning. The first warning, however, is a result of how Vagrant is calling Puppet.

Once your machine has booted, simply issue the following command to connect:

vagrant ssh

This will connect you to the machine using ssh. Once this is complete, we can start working on our module.

We'll be using a Puppet module to secure SSH. As such, we should go ahead and create the directory to hold our module. You can issue the following commands to create the module skeleton on the guest virtual machine:

sudo mkdir –p /etc/puppet/modules/openssh/manifests
sudo mkdir –p /etc/puppet/modules/openssh/files

These directories will hold the manifests for Puppet to compile as well as our configuration file. For our first simplistic example, we will use a static SSH configuration file. In later chapters, we will build upon it and make it dynamic with the various options that are available.

Now that we have the framework, we'll build our first module. Much like the preceding code, we'll go through it section by section covering what each resource does. The manifest we're building will be very similar to the one we used to provision the Puppet Master for the use of.

First, we'll edit the /etc/puppet/modules/openssh/manifests/init.pp file to create the module's main manifest. This manifest is the main unit of the Puppet code, which is invoked when we include the module. As we go through each of the sections, we'll go through what they do. A complete manifest file can be found on this book's website, but you should really build it along with us. This will help you with understanding and memorization:

class openssh {

The preceding line defines the class. The class in the init.pp file is always named after the module. It's a new construct we've not seen before that is unique to creating modules:

  package { 'openssh-server':
    ensure => 'latest',
  }

The preceding section is similar to the puppetmaster section. The only difference is that we're using latest instead of present. Being a security-related package, it may make sense to make sure that you keep openssh up to date.

Alternatively, if your environment requires it, you could specify a fixed version to install. This might be useful if you require pretested versions or have validated versions. You must weigh the benefits, ensuring that you run the most recent version of the software, including the risk of almost immediately installing it when it is available, and that you're using the latest tag:

  file { '/etc/ssh/sshd_config':
    ensure => 'present',
    owner  => 'root',
    group  => 'root',
    mode   => '0600',
    source => 'puppet:///modules/openssh/sshd_config',
  }

This is similar to the Puppet Master configuration file, but we introduced a new construct here. We're sourcing the file from puppet master by using the special puppet:// uniform resource identifier (URL). When Puppet runs, it will fetch the file from the master for use on the agent. The source file should be present in the /etc/puppet/modules/openssh/files directory on the master:

  service { 'sshd':
    ensure => 'running',
  }

Here, as before, we ensure that ssh is running when we run Puppet:

  Package['openssh-server'] 
  -> File['/etc/ssh/sshd_config']
  ~> Service['sshd']
}

This is also a new construct called resource chaining. It is an alternative way to specify that we do things in the order listed: first, the package, followed by the file, and then the service. Note the tilde on the service dependency. This shows that we're notifying the service. It means that if the configuration file changes, the service will be restarted.

To build the configuration file we're going to use, we'll start with the openssh configuration file shipped with CentOS and make a few changes. First, we'll copy the existing configuration file with the following command:

sudo cp /etc/ssh/sshd_config /etc/puppet/modules/openssh/files/

Next, we'll edit the file with your favorite editor. Make sure you run it in sudo as you won't have permission to edit the file. We'll uncomment and change the following lines in the file:

PermitRootLogin no
MaxAuthTries 3

We'll start with these changes to demonstrate how the process works. Then, save the file.

Next, we need to make sure the Puppet agent can read it. We'll set the permissions in such a manner that the Puppet user can read it. Execute the following:

sudo chgrp puppet /etc/puppet/modules/openssh/files/sshd_config
sudo chmod 640 /etc/puppet/modules/openssh/files/sshd_config

Now, we need to bring it all together to tell Puppet to use our module. By default, Puppet runs a file called site.pp on the master to determine what actions to take when a node checks in. We need to add the new module to the file for Puppet to run it.

The file lives in /etc/puppet/manifests on our Vagrant guest. Go ahead and open it in your favorite editor and add the following section:

node default {
  include openssh
}

This adds a default node declaration and includes our openssh module on that node. It will ensure that our new module gets used.

Now that we've got it all built, let's go ahead and see the fruits of our labor. Execute the following command:

sudo puppet agent --test

You should see the output as follows:

Victory! You can see that Puppet changed the file to disallow root logins and change the maximum authentication attempts to 3.

As with any new technology, the learning curve can seem somewhat overwhelming at first. We've now gone through a rather lengthy example to effectively make a two-line edit to a configuration file on a single machine. This was a short and simple example to explore some base concepts of Puppet. Using this concept, we could apply this same edit to hundreds or even thousands of machines in our infrastructure with very little additional effort. We'll also be exploring more in-depth examples as we gain a skillset. With some practice, you will find that applying changes across one of many machines becomes second nature with Puppet.