Learning Puppet

Before we get our hands dirty with Puppet, I'd like to expand the topic a little bit.

As you may already know, Puppet is a configuration management tool that enables you to build application stacks from a set of files that the Puppet community refers to as the manifests.

Manifests are a set of instructions that describes how operating systems and application resources are managed by Puppet and how the system configuration should look like after the manifest has been applied to the system.

Manifests are written in a language called Puppet DSL, where DSL stands for Domain Specific Language. DSL is a commonly used term for programming languages that are not general-purpose languages.

When I write Puppet manifests, I consider it a development process. I call it a development process because the process consists of multiple rounds of iterations during which the manifest evolves.

Here is a simplified overview of the Puppet manifest development process:

For iteration 1, follow the given steps:

Begin by writing the initial manifest that installs a software package > Apply the manifest > Ensure that package is installed.

For iteration 2, follow the given steps:

Extend the manifest to apply the configuration for the package > Apply the manifest > Ensure that the configuration was correctly applied.

For iteration 3, follow the next steps:

Add the logic to start up the service > Apply the manifest > Ensure that the service started > Finish.

In this example, the development processes had three rounds of iterations, each of them containing a task called Apply the manifest.

Imagine a situation where we write a manifest that creates a Linux user account with a root level access but no password. An account with a root level access is equivalent to a local administrator account on a Windows computer. When you apply the manifest on your computer, Puppet will create a user account on your computer without a password, which makes your computer vulnerable to attacks.

In contrast, if you apply the manifest in an isolated development environment, the configuration change is easy to undo as you can quickly tear down the environment and rebuild it from scratch.

Another reason for developing manifests in isolation is consistency. My choice of operating system is Ubuntu Linux and I run it on MacBook Pro hardware. You may run Mac OS X on the Mac mini, and a friend of mine just downgraded to Windows 7 as she was unhappy with the functionality offered by Windows 8.1.

Each of these operating system flavors will behave slightly differently, although all of them do share similar capabilities such as running a virtualization software.

To ensure that the examples and exercises covered in this book produce consistent results for you and me, we will start our journey by installing the VirtualBox virtualization software package, which enables us to run a set of virtual machines that forms our isolated development environment.

If you prefer to use an alternative virtualization technology, you are free to do so as long as the software supports the following functionalities, and you know how to configure the software to enable the following functionalities:

This book is based on Oracle VirtualBox Version 4.3. To ensure that the configuration examples this book provides work well, I'd recommend that you download VirtualBox Version 4.3.

VirtualBox 4.3 can be downloaded for free from the VirtualBox website at https://www.virtualbox.org/wiki/Download_Old_Builds_4_3 (Google: virtualbox download 4.3).

On the download page, you should see a category for the VirtualBox platform packages. Select the download option that is most suitable for your operating system:

When you have downloaded the VirtualBox installation package, it is time to install it. Double-click on the installation package that you downloaded, and you will see the installation wizard pop up on the screen.

Install VirtualBox with the default options, and we will take a look at how to configure VirtualBox to optimize it for our development environment.

You now should have VirtualBox installed, which can run virtual machines for our isolated development environment.

We will be using the Puppet Learning VM Version 3.7.1, which I've uploaded to an Amazon S3 bucket.

The Puppet Learning VM is a virtual machine image that comes with preinstalled Puppet software. Installing Puppet is not a difficult or too time-consuming task, but it will save us all a little bit of our valuable time if we use the Puppet Learning VM, so we can start experimenting with Puppet quicker.

Follow these steps to complete the download process:

Next, we will import the virtual machine to the VirtualBox, take a snapshot of it, and then we should be ready to start experimenting with Puppet.

The extracted virtual machine image file has to be imported to VirtualBox before we can launch it. Here are the steps to import the image to VirtualBox:

Here, we can configure the virtual machine settings, such as increasing the amount of memory or adding more processor cores.

We don't need to change the default settings, so let's just click on the Import button to start the import process:

While the virtual machine appliance is importing, I'll give you a quick introduction to virtual machine snapshots and how we can use them.

A virtual machine snapshot stores the state of the virtual machine at a particular point of time. We can have multiple snapshots of a virtual machine, and we can easily switch between them.

The example I've just given may give you impression that snapshots are backups. They are not!

Snapshots only contain the data that has changed since the previous snapshot, and therefore, an individual snapshot cannot be used to reconstruct the whole virtual machine. To reconstruct the virtual machine from the snapshot, VirtualBox will need the virtual disk file (learn_puppet_centos-6.5-disk1.vmdk), all prior snapshots plus the snapshot you want to restore the state to.

When you create a snapshot of the virtual machine, which we will do shortly, you will tell VirtualBox to start writing changes to a snapshot file instead of the virtual machine disk file.

Every time you create a snapshot, VirtualBox creates a new snapshot file and starts writing changes to it.

The snapshots are laid out in the following type of tree structure:

If a virtual machine has only one snapshot and we delete it, then VirtualBox writes changes in the snapshot file onto the disk. If a virtual machine contains more than one snapshot and you delete one, then VirtualBox merges two consecutive snapshots.

Having many snapshots may have an impact on the disk's performance, because the disk operations have to traverse through many snapshots to find the file to make the changes. For disk performance reasons it is recommended that you delete older snapshots when they are no longer needed.

Before we start the virtual machine, we should take a snapshot of it so that we can quickly revert to the point of time where we started.

To take a snapshot of the virtual machine and name it, follow the given steps:

Now, we have a snapshot that enables us to go back to the previous state of the VM as we make changes to the system configuration.

Now it is the right time to start the real Puppet work:

puppet --version

puppet describe --list | less

Managing resources from the command line

We can manage Puppet resources from the command line using the following syntax: puppet resource <type> <name> <attribute1>=<value> <attribute2>=<value>.

Let's create a user called Elisa on the system using Puppet. In the Learning VM terminal, type in the following command and and hit the Enter key:

puppet resource user Elisa ensure=present

When the command is executed successfully, it produces the following output:

The first line of the output displays a notice confirming that the user account Elisa was created by Puppet. Lines 2-4 show the syntax that we will be using when we declare resources in the Puppet manifest files. The manifest files will be explained more in detail later.

Let's take a look at the syntax line by line:

• Line 1: Notice: /User[Elisa]/ensure: created

  This displays a confirmation of an action that Puppet has created a user called Elisa.

• Line 2: user { 'Elisa':

  This declares a resource for a type user, which follows the opening curly brace ({), that indicates that the user resource name and optional attributes are to follow. The name 'Elisa': at the end of the line sets the Puppet resource name, which will become the name of the user account. The Puppet resource name must contain a colon at the end.

• Line 3: ensure => 'present'

  This attribute means that a user account must be created unless it already exists.

• Line 4: }

  The closing curly brace indicates the end of the resource statement.

The name Elisa on line 2 has two use cases. Firstly, it declares the name of the Puppet resource. Each Puppet resource must have a unique name, otherwise Puppet reports an error.

Secondly, the name Elisa is used as the name of the user account that was created. If the user statement contains a name attribute (alongside the ensure attribute), then the value of the name attribute would become the name of the user account and the name Elisa would only be used as the Puppet resource name.

As the name attribute is omitted, Puppet will use the name for the Puppet resource name as well as for the user account name.

Declaring the following statement in the Puppet manifest would result in the Jakob user account being created in the system, as the name attribute would take priority over the Puppet resource name Elisa:

user { 'Elisa':
  ensure => 'present',
  name   => 'Jakob',
}

This produces an output that is similar to the following, although the numeric information in the output may vary between the systems:

uid=501(Elisa) gid=501(Elisa) groups=501(Elisa)

Next, we can remove the account Elisa from the system by setting the ensure attribute value to absent.

puppet resource user Elisa ensure=absent

Assuming that the command executes successfully, you should see the following output:

This output is very similar to how we created the user account Elisa except that line 1 confirms that the user account Elisa was removed and line 3 has the ensure attribute value set to absent, which results in the account being removed when declaring a resource in the Puppet manifest file.

To confirm whether Puppet really removed the account, you can run the command id Elisa again, which will confirm that the account no longer exists in the system.

The command id Elisa should produce the following output:

id: Elisa: No such user

Congratulations! You just did two system configuration changes using Puppet. It wasn't hard, right?

Puppet dry run

Sometimes, you want to simulate configuration changes without applying the changes to the system. This can be done by adding the --noop parameter after the resource keyword in the Puppet command.

To simulate account creation without creating the account, we can extend the previous user account creation command with the --noop option:

puppet resource user Elisa ensure=present --noop

Line1 in the output tells us that the user account does not exist in the system, and if you run the following command without the --noop option, Puppet would create an account:

Notice: /User[Elisa]/ensure: current_value absent, should be present (noop)
user { 'Elisa':
  ensure => 'absent',
}

The --noop option comes in handy when testing Puppet commands for a syntax. To demonstrate this, we declare an invalid value removed for the attribute ensure:

puppet resource user Elisa ensure=removed --noop

Puppet will return an error that tells you that we have used an invalid value for the ensure attribute:

Use Puppet to examine the current state of resources

Puppet can also be used to query resources from the system. Information produced by the query can be helpful when we are uncertain about what syntax we should be using to create a resource.

Earlier, we created a user account called Elisa. This account was created without a password, which means that the account cannot be used for interactive logins. Let's recreate the user account and use the password attribute to set a password for the account.

As passwords on Linux are encrypted, we must provide them to the password attribute in the encrypted format.

We know that the user account root on the Learning VM uses the password puppet, but we yet don't know how the password would look like in the encrypted format.

No problem, as we can query the password with the command and then use the encrypted password when recreating the user account Elisa.

The following command shows all the attributes for the root user account:

Puppet resource user root

The preceding command produces the following output:

Now we can create the user account Elisa with the password attribute, which is the same as the value of the password attribute for the user root.

The following command will create the user account Elisa that uses the password puppet:

puppet resource user Elisa ensure=present \
managehome=true \
password='$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/'

I've split the command into three lines with the backslash character at the end of the first two lines.

Note

The $1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/ string is a hash of the password puppet. Please note that we have to use single quotes around the password hash '$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/' because the string contains characters that the Linux command line otherwise interprets as a control character.

Now we can test whether we can log on to the system as the user Elisa.

Log out from the system using the logout command. Then, log in with the username Elisa and password puppet. You will see the following welcome screen, which confirms that the login of Elisa was successful:

Puppet is run as a user root

The root account in Linux is equivalent to an administrator account in the Windows operating system. This is the user account that is commonly used for system configuration changes.

The user account Elisa that we created does not have the same amount of privileges as the root account.

To change the system configuration in the protected areas of the operating system, we must run Puppet as a root user.

If you are still logged onto the system as user Elisa, you can try creating a user account and see what happens.

As the user Elisa is not configured to run Puppet, we will do our test using the Linux adduser command.

Let's see what response we get if we try to create a user called Jakob using the user Elisa:

useradd Jakob

The output of the preceding command shows you that the user Elisa did not have sufficient permissions to add a new user to the system:

/usr/sbin/useradd: Permission denied

To avoid possible permission issues on Puppet managed systems, Puppet runs as a user root that provides Puppet full control of the system to add and remove users, install and uninstall software packages, as well as manage system services.

In the end, Puppet is your new system administrator, which manages the system according to the instructions you have provided from the command line or in the form of the Puppet manifest.

Puppet DSL and manifests

I've mentioned Puppet manifests earlier, but I haven't yet explained what manifests are. Puppet manifests are text files that declare one or more Puppet resources. Instead of running Puppet resource commands on the command line, you can declare resources in the manifest file and apply the manifest to the system.

Puppet manifests uses the Puppet Domain Specific Language (DSL) and resource statements in the manifest file, which are described in a syntax that looks very similar to a Hash data type in the Ruby language.

We can use our user account Elisa as a simple example of the Puppet manifest syntax.

First, log out from the user account Elisa by running the logout command. Then, log on to the system as a root user and remove the user Elisa from the system with the following command:

puppet resource user Elisa ensure=absent

Then, inspect the state of the user account Elisa with the following commands:

puppet resource user Elisa
user { 'Elisa':
  ensure => 'absent',
}

In the Ruby language, if we try to declare a hash called User that contains a key Elisa with the value of the ensure attribute as absent, we will declare it using the following syntax:

User = { 'Elisa' =>
  { 'ensure' => 'absent' }
}

If you compare the preceding two code blocks, you can see that the Puppet DSL syntax looks similar to the Ruby language syntax, but it is slightly simpler and easier to read than the Ruby equivalent.

The output of the preceding Puppet resource command is spread across three lines only in order to make it easier for us to read. The Puppet parser that reads the manifest file doesn't care about the line feed characters.

The preceding user resource can be declared on a single line as follows:

user { 'Elisa': ensure => 'absent', }

We now have the Puppet DSL representation of the user resource Elisa.

We can apply this resource to the --execute (or -e) puppet apply command, which will remove the user Elisa from the system.

As Elisa no longer exists in the system, let's change the ensure attribute value to 'present' so that Puppet can create the user Elisa:

puppet apply --execute "user { 'Elisa':  ensure => 'present', }"

Puppet will display the following output on the screen:

As you might have noticed, the output of the puppet apply command looks different from the puppet resource command that we used earlier to create user Elisa. Let's examine the output line by line:

The preceding three lines relate to the following different stages of the Puppet run:

We covered the Puppet DSL syntax that is used in the Puppet manifests. Let's try to create a manifest and learn how to apply it to the system.

The following are the steps to create a manifest:

You must have probably noticed that this time we ran the puppet apply command without the -–execute option. The --execute option is only used to provide the manifest content from the command line. Now that we have created the manifest file, and if we want to apply it, the --execute option can be omitted. Typically, the --execute option is used to pass parameters to the Puppet class that is declared in the manifest. We will discuss the Puppet classes more in detail later on in this book.

puppet apply user.pp

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.14 seconds
Notice: Finished catalog run in 0.27 seconds

puppet resource user Markus ensure=absent

puppet apply user.pp

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.15 seconds
Notice: /Stage[main]/Main/User[Markus]/ensure: created
Notice: Finished catalog run in 0.35 seconds

file {  '/root/Documents':
  ensure => directory;
}

puppet resource user Jakob ensure=absent

puppet resource user Jakob > jakobs-login.pp

puppet resource file /home/Jakob/.bash_profile >> jakobs-login.pp

nano jakobs-login.pp

user { 'Jakob':
  ensure => 'absent',
}
file { '/home/Jakob/.bash_profile':
  ensure => 'absent',
}

user { 'Jakob':
  ensure => 'present',
  managehome => true,
  password => '$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/',
}

puppet apply jakobs-login.pp

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.63 seconds
Notice: /Stage[main]/Main/User[Jakob]/ensure: created
Notice: /Stage[main]/Main/File[/home/Jakob/.bash_profile]/ensure: removed
Notice: Finished catalog run in 0.35 seconds

So far, we have discussed how to configure a system using Puppet. But what about Puppet's own configuration? Can the Puppet configuration be managed by Puppet itself?

The answer is yes, but if you decide to do so, do it with caution. Test your Puppet configuration changes thoroughly in isolation, and test it multiple times before pushing it into a live environment. It only requires a minor error in your configuration, and your Puppet agents become non-functional.

There are two ways to manage the Puppet configuration. The Puppet configuration can be managed from the command line by running the puppet config commands. Or the configuration can be changed by editing the file in /etc/puppetlabs/puppet/puppet.conf, if you are using the Puppet Enterprise edition as we are doing. In the open source Puppet, the configuration file path is /etc/puppet/puppet.conf.

Let's view the contents of the file with the utility called less , which enables us to browse the file with the arrow keys:

less /etc/puppetlabs/puppet/puppet.conf

The content of the puppet.conf file is similar to the ini configuration files, which are commonly used with Windows applications. The data structure basically is a key value pair separated by the = equivalence sign.

There are also sections in the configuration file that are marked with the section name wrapped inside the block brackets. The sections are as follows:

The [main] section contains the configuration that is shared by the [master] and [agent] sections.

The [master] section contains the configuration for the Puppet master, which we will discuss in detail later in this book.

The [agent] section contains the configuration for the Puppet agent, which we have already been using when managing resources on the command line.

When you take a look at the second line in the /etc/puppetlabs/puppet/puppet.conf file, you can see a configuration key called certname with the learning.puppetlabs.vm value. Using the arrow keys, when scrolling down to the [agent] section, we find a key called environment with the value production.

Do you recall seeing these values before? You probably do from the output of the puppet apply command that we ran earlier. Here is the output that I'm referring to:

Notice: Compiled catalog for learning.puppetlabs.vm  in environment production in 0.15 seconds

The Compiled catalog for learning.puppetlabs.vm string and the environment production are defined in the Puppet configuration file. When running Puppet in the standalone mode, as we are at this point, the configuration is not that relevant; but later on in this book, when we link the Puppet Agents with the Puppet Master, we will benefit from knowing how to change the Puppet configuration.

To change the Puppet Agent configuration, we can use the Nano text editor and edit the file manually, but as an alternative, we can use the Puppet command-line utility to change the configuration.

As an exercise, we can change the Puppet Agent's identity with the following command:

puppet config set certname brandnew

While we are at it, let's change the environment as well. As we are developing Puppet, a suitable environment name for it is development, which we can set with the following command:

puppet config set environment development

Puppet expects to find an environment-specific directory in the filesystem, so let's create one with the following command:

mkdir /etc/puppetlabs/puppet/environments/development

Now run the puppet apply user.pp command, and you can see that the configuration changes have become effective:

Notice: Compiled catalog for brandnew in environment development in 0.13 seconds
Notice: Finished catalog run in 0.27 seconds

Now we can try changing the configuration manually in the Nano editor. Open the configuration file:

nano /etc/puppetlabs/puppet/puppet.conf

Press Ctrl + W to search for a certname key and replace the brandnew value with the learning.puppetlabs.vm string .

Then, search for an environment key name, and Nano will take you a couple of lines down to the end of the [main] configuration section. This line got added when we changed the configuration with the puppet config set command. Now repeat the search with Alt + W, and you will find another key called environment in the [agent] configuration block with the original value production. Why duplicate keys? Well, by default, the puppet config set command manages the configuration under the [main] block of the configuration file. The keys specified in this section will take priority over the configuration in the [master] and [agent] sections.

So, to revert to the environment value production, we can just remove the environment development from the [main] configuration block.

Once the line has been removed, save the puppet.conf file by pressing Ctrl + X, confirm the save operation by pressing Y for Yes, and then press Enter.

To confirm that the configuration changes were successfully applied, we can query specific keys in the configuration file with the puppet config print command:

puppet config print certname environment

The output of the command should show that the configuration was successfully changed. Here is a screenshot of the puppet config print certname environment command before and after the change.

In this chapter, we thoroughly covered the basics of Puppet, such as how to run Puppet on the command line, and how to use, generate, and edit manifests. We also learned how to get the development environment up and running quickly with a little installation effort using VirtualBox.

In the next chapter, we will be adding a little bit more functionality to VirtualBox, and then start experimenting with the new type of Puppet resources; mainly, let's take a look at how to remove resources with Puppet. This will contribute toward the goal, which is to build a Puppet-managed environment that consists of multiple virtual machines.