DevOps: Puppet, Docker, and Kubernetes

4.8 (6 reviews total)
By Thomas Uphill , John Arundel , Neependra Khare and 3 more
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies

About this book

With so many IT management and DevOps tools on the market, both open source and commercial, it’s difficult to know where to start. DevOps is incredibly powerful when implemented correctly, and here’s how to get it done.This Learning Path covers three broad areas: Puppet, Docker, and Kubernetes. This Learning Path is a large resource of recipes to ease your daily DevOps tasks. We begin with recipes that help you develop a complete and expert understanding of Puppet’s latest and most advanced features. Then we provide recipes that help you efficiently work with the Docker environment. Finally, we show you how to better manage containers in different scenarios in production using Kubernetes

This course is based on these books:

  1. Puppet Cookbook - Third Edition
  2. Docker Cookbook
  3. Kubernetes Cookbook
Publication date:
March 2017
Publisher
Packt
Pages
925
ISBN
9781788297615

   

This recipe will introduce the language and show you the basics of writing Puppet code. A beginner may wish to reference Puppet 3: Beginner's Guide, John Arundel, Packt Publishing in addition to this section. Puppet code files are called manifests; manifests declare resources. A resource in Puppet may be a type, class, or node. A type is something like a file or package or anything that has a type declared in the language. The current list of standard types is available on puppetlabs website at https://docs.puppetlabs.com/references/latest/type.html. I find myself referencing this site very often. You may define your own types, either using a mechanism, similar to a subroutine, named defined types, or you can extend the language using a custom type. Types are the heart of the language; they describe the things that make up a node (node is the word Puppet uses for client computers/devices). Puppet uses resources to describe the state of a node; for example, we will declare the following package resource for a node using a site manifest (site.pp).

How to do it...

Create a site.pp file and place the following code in it:

node default { package { 'httpd': ensure => 'installed' } } How it works...

This manifest will ensure that any node, on which this manifest is applied, will install a package called 'httpd'. The default keyword is a wildcard to Puppet; it applies anything within the node default definition to any node. When Puppet applies the manifest to a node, it uses a Resource Abstraction Layer (RAL) to translate the package type into the package management system of the target node. What this means is that we can use the same manifest to install the httpd package on any system for which Puppet has a Provider for the package type. Providers are the pieces of code that do the real work of applying a manifest. When the previous code is applied to a node running on a YUM-based distribution, the YUM provider will be used to install the httpd RPM packages. When the same code is applied to a node running on an APT-based distribution, the APT provider will be used to install the httpd DEB package (which may not exist, most debian-based systems call this package apache2; we'll deal with this sort of naming problem later).

Facter is a separate utility upon which Puppet depends. It is the system used by Puppet to gather information about the target system (node); facter calls the nuggets of information facts. You may run facter from the command line to obtain real-time information from the system.

Running facter without any arguments causes facter to print all the facts known about the system. We will see in later chapters that facter can be extended with your own custom facts. All facts are available for you to use as variables; variables are discussed in the next section.

How to do it...

Use facter to find

Running facter without any arguments causes facter to print all the facts known about the system. We will see in later chapters that facter can be extended with your own custom facts. All facts are available for you to use as variables; variables are discussed in the next section.

How it works...

When facter is installed (as a dependency for puppet), several fact definitions are installed by default. You can reference each of these facts by name from the command line.

Running facter without any arguments causes facter to print all the facts known about the system. We will see in later chapters that facter can be extended with your own custom facts. All facts are available for you to use as variables; variables are discussed in the next section.

There's more...

Running facter without any arguments causes facter to print all the facts known about the system. We will see in later chapters that facter can be extended with your own custom facts. All facts are available for you to use as variables; variables are discussed in the next section. Variables

Variables Scope

In the variable example explained in the There's more… section, the fully qualified domain name was referred to as ${::fqdn} rather than ${fqdn}; the double colons are how Puppet differentiates

To show how ordering works, we'll create a manifest that installs httpd and then ensures the httpd package service is running.

In this example, the package will be installed before the service is started. Using require within the definition of the httpd service ensures that the package is installed first, regardless of the order within the manifest file.

All the manifests that will be used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed?

We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used:

The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file.

How to do it...

We start by creating a manifest that defines the service:
  service {'httpd':
    ensure  => running,
    require => Package['httpd'],
  }
The service definition references a package resource named httpd; we now need to define that resource:
  package {'httpd':
    ensure => 'installed',
  }

In this example, the package will be installed before the service is started. Using require within the definition of the httpd service ensures that the package is installed first, regardless of the order within the manifest file.

All the manifests that will be used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed?

We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used:

The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file.

How it works...

In this example, the

package will be installed before the service is started. Using require within the definition of the httpd service ensures that the package is installed first, regardless of the order within the manifest file.

All the manifests that will be used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed?

We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used:

The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file.

Capitalization

Capitalization

All the manifests that will be used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed?

We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used:

The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file.

Learning metaparameters and ordering

All the manifests that will be

used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed?

We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used:

The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file.

Trifecta

The relationship between package and service previously mentioned is an important and powerful paradigm of Puppet. Adding one more resource-type file into the fold, creates what puppeteers refer to as the Idempotency

A key concept of Puppet is that the state of the system when a catalog is applied to a node cannot affect the outcome of Puppet run. In other words, at the end of Puppet run (if the run was successful), the system will be in a known state and any further application of the catalog will result in a system that is in the same state. This property of Puppet is known as idempotency. Idempotency

There are many examples of this pattern online. In our simple example, we will create an Apache configuration file under /etc/httpd/conf.d/cookbook.conf. The /etc/httpd/conf.d directory will not exist until the httpd package is installed. After this file is created, we would want httpd to restart to notice the change; we can achieve this with a notify parameter.

How to do it...

We will need the same
How it works…

The require attribute to the file resources tell Puppet that we need the /var/www/cookbook directory created before we can create the index.html file. The important concept to remember is that we cannot assume anything about the target system (node). We need to define everything on which the target depends. Anytime you create a file in a manifest, you have to ensure that the directory containing that file exists. Anytime you specify that a service should be running, you have to ensure that the package providing that service is installed.

In this example, using metaparameters, we can be confident that no matter what state the node is in before running Puppet, after Puppet runs, the following will be true:

httpd will be running
The VirtualHost

If other people need to read or maintain your manifests, or if you want to share code with the community, it's a good idea to follow the existing style conventions as closely as possible. These govern such aspects of your code as layout, spacing, quoting, alignment, and variable references, and the official puppetlabs recommendations on style are available at http://docs.puppetlabs.com/guides/style_guide.html.

In this section, I'll show you a few of the more important examples and how to make sure that your code is style compliant.

How to do it…

In this section, I'll show you a few of the more important examples and how to make sure that your code is style compliant.

Indentation

Indent your manifests using
Quoting

Always quote your
False

There is only one thing in Puppet that is false, that is, the word false without any quotes. The string "false" evaluates to true and the string "true" also evaluates to true. Actually, everything besides the literal
Variables

Always include curly
Parameters

Always end lines that declare Symlinks

When declaring file

If you already have some Puppet code (known as a Puppet manifest), you can skip this section and go on to the next. If not, we'll see how to create and apply a simple manifest.

To create and apply a simple manifest, follow these steps:

  1. First, install Puppet locally on your machine or create a virtual machine and install Puppet on that machine. For YUM-based systems, use https://yum.puppetlabs.com/ and for APT-based systems, use https://apt.puppetlabs.com/. You may also use gem to install Puppet. For our examples, we'll install Puppet using gem on a Debian Wheezy system (hostname: cookbook). To use gem, we need the rubygems package as follows:
    [email protected]:~$ sudo apt-get install rubygems
    Reading package lists... Done
    Building dependency tree        
    Reading state information... Done
    The following NEW packages will be installed:
      rubygems
    0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
    Need to get 0 B/597 kB of archives.
    After this operation, 3,844 kB of additional disk space will be used.
    Selecting previously unselected package rubygems.
    (Reading database ... 30390 files and directories currently installed.)
    Unpacking rubygems (from .../rubygems_1.8.24-1_all.deb) ...
    Processing triggers for man-db ...
    Setting up rubygems (1.8.24-1) ...
    
  2. Now, use gem to install Puppet:
    [email protected] $ sudo gem install puppet
    Successfully installed hiera-1.3.4
    Fetching: facter-2.3.0.gem (100%)
    Successfully installed facter-2.3.0
    Fetching: puppet-3.7.3.gem (100%)
    Successfully installed puppet-3.7.3
    Installing ri documentation for hiera-1.3.4
    Installing ri documentation for facter-2.3.0
    Installing ri documentation for puppet-3.7.3
    Done installing documentation for hiera, facter, puppet after 239 seconds
    
  3. Three gems are installed. Now, with Puppet installed, we can create a directory to contain our Puppet code:
    [email protected]:~$ mkdir -p .puppet/manifests
    [email protected]:~$ cd .puppet/manifests
    [email protected]:~/.puppet/manifests$
    
  4. Within your manifests directory, create the site.pp file with the following content:
      node default {
        file { '/tmp/hello':
          content => "Hello, world!\n",
        }
      }
  5. Test your manifest with the puppet apply command. This will tell Puppet to read the manifest, compare it to the state of the machine, and make any necessary changes to that state:
    [email protected]:~/.puppet/manifests$ puppet apply site.pp
    Notice: Compiled catalog for cookbook in environment production in 0.14 seconds
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/hello]/ensure: defined content as '{md5}746308829575e17c3331bbcb00c0898b'
    Notice: Finished catalog run in 0.04 seconds
    
  6. To see if Puppet did what we expected (create the /tmp/hello file with the Hello, world! content), run the following command:
    [email protected]:~/puppet/manifests$ cat /tmp/hello
    Hello, world!
     [email protected]:~/puppet/manifests$
    
How to do it...

To create and apply a simple manifest, follow these steps:

First, install Puppet locally on your machine or create a virtual machine and install Puppet on that machine. For
  1. YUM-based systems, use https://yum.puppetlabs.com/ and for APT-based systems, use https://apt.puppetlabs.com/. You may also use gem to install Puppet. For our examples, we'll install Puppet using gem on a Debian Wheezy system (hostname: cookbook). To use gem, we need the rubygems package as follows:
    [email protected]:~$ sudo apt-get install rubygems
    Reading package lists... Done
    Building dependency tree        
    Reading state information... Done
    The following NEW packages will be installed:
      rubygems
    0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
    Need to get 0 B/597 kB of archives.
    After this operation, 3,844 kB of additional disk space will be used.
    Selecting previously unselected package rubygems.
    (Reading database ... 30390 files and directories currently installed.)
    Unpacking rubygems (from .../rubygems_1.8.24-1_all.deb) ...
    Processing triggers for man-db ...
    Setting up rubygems (1.8.24-1) ...
    
  2. Now, use gem to install Puppet:
    [email protected] $ sudo gem install puppet
    Successfully installed hiera-1.3.4
    Fetching: facter-2.3.0.gem (100%)
    Successfully installed facter-2.3.0
    Fetching: puppet-3.7.3.gem (100%)
    Successfully installed puppet-3.7.3
    Installing ri documentation for hiera-1.3.4
    Installing ri documentation for facter-2.3.0
    Installing ri documentation for puppet-3.7.3
    Done installing documentation for hiera, facter, puppet after 239 seconds
    
  3. Three gems are installed. Now, with Puppet installed, we can create a directory to contain our Puppet code:
    [email protected]:~$ mkdir -p .puppet/manifests
    [email protected]:~$ cd .puppet/manifests
    [email protected]:~/.puppet/manifests$
    
  4. Within your manifests directory, create the site.pp file with the following content:
      node default {
        file { '/tmp/hello':
          content => "Hello, world!\n",
        }
      }
  5. Test your manifest with the puppet apply command. This will tell Puppet to read the manifest, compare it to the state of the machine, and make any necessary changes to that state:
    [email protected]:~/.puppet/manifests$ puppet apply site.pp
    Notice: Compiled catalog for cookbook in environment production in 0.14 seconds
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/hello]/ensure: defined content as '{md5}746308829575e17c3331bbcb00c0898b'
    Notice: Finished catalog run in 0.04 seconds
    
  6. To see if Puppet did what we expected (create the /tmp/hello file with the Hello, world! content), run the following command:
    [email protected]:~/puppet/manifests$ cat /tmp/hello
    Hello, world!
     [email protected]:~/puppet/manifests$
    
There's more…

When several people

The puppetlabs official style guide outlines a number of style conventions for Puppet code, some of which we've touched on in the preceding section. For example, according to the style guide, manifests:

Following the style guide will make sure that your Puppet code is easy to read and maintain, and if you're planning to release your code to the public, style compliance is essential.

The puppet-lint tool will automatically check your code against the style guide. The next section explains how to use it.

You can find out more about Puppet-lint at https://github.com/rodjek/puppet-lint.

Should you follow Puppet style guide and, by extension, keep your code lint-clean? It's up to you, but here are a couple of things to think about:

  • It makes sense to use some style conventions, especially when you're working collaboratively on code. Unless you and your colleagues can agree on standards for whitespace, tabs, quoting, alignment, and so on, your code will be messy and difficult to read or maintain.
  • If you're choosing a set of style conventions to follow, the logical choice would be that issued by puppetlabs and adopted by the community for use in public modules.

Having said that, it's possible to tell Puppet-lint to ignore certain checks if you've chosen not to adopt them in your codebase. For example, if you don't want Puppet-lint to warn you about code lines exceeding 80 characters, you can run Puppet-lint with the following option:

[email protected] ~$ puppet-lint --no-80chars-check

Run puppet-lint --help to see the complete list of check configuration commands.

Getting ready

Here's what you need to

You can find out more about Puppet-lint at https://github.com/rodjek/puppet-lint.

Should you follow Puppet style guide and, by extension, keep your code lint-clean? It's up to you, but here are a couple of things to think about:

  • It makes sense to use some style conventions, especially when you're working collaboratively on code. Unless you and your colleagues can agree on standards for whitespace, tabs, quoting, alignment, and so on, your code will be messy and difficult to read or maintain.
  • If you're choosing a set of style conventions to follow, the logical choice would be that issued by puppetlabs and adopted by the community for use in public modules.

Having said that, it's possible to tell Puppet-lint to ignore certain checks if you've chosen not to adopt them in your codebase. For example, if you don't want Puppet-lint to warn you about code lines exceeding 80 characters, you can run Puppet-lint with the following option:

[email protected] ~$ puppet-lint --no-80chars-check

Run puppet-lint --help to see the complete list of check configuration commands.

How to do it...

Follow these

You can find out more about Puppet-lint at https://github.com/rodjek/puppet-lint.

Should you follow Puppet style guide and, by extension, keep your code lint-clean? It's up to you, but here are a couple of things to think about:

  • It makes sense to use some style conventions, especially when you're working collaboratively on code. Unless you and your colleagues can agree on standards for whitespace, tabs, quoting, alignment, and so on, your code will be messy and difficult to read or maintain.
  • If you're choosing a set of style conventions to follow, the logical choice would be that issued by puppetlabs and adopted by the community for use in public modules.

Having said that, it's possible to tell Puppet-lint to ignore certain checks if you've chosen not to adopt them in your codebase. For example, if you don't want Puppet-lint to warn you about code lines exceeding 80 characters, you can run Puppet-lint with the following option:

[email protected] ~$ puppet-lint --no-80chars-check

Run puppet-lint --help to see the complete list of check configuration commands.

There's more...

You can find out more about

Puppet-lint at https://github.com/rodjek/puppet-lint.

Should you follow Puppet style guide and, by extension, keep your code lint-clean? It's up to you, but here are a couple of things to think about:

  • It makes sense to use some style conventions, especially when you're working collaboratively on code. Unless you and your colleagues can agree on standards for whitespace, tabs, quoting, alignment, and so on, your code will be messy and difficult to read or maintain.
  • If you're choosing a set of style conventions to follow, the logical choice would be that issued by puppetlabs and adopted by the community for use in public modules.

Having said that, it's possible to tell Puppet-lint to ignore certain checks if you've chosen not to adopt them in your codebase. For example, if you don't want Puppet-lint to warn you about code lines exceeding 80 characters, you can run Puppet-lint with the following option:

[email protected] ~$ puppet-lint --no-80chars-check

Run puppet-lint --help to see the complete list of check configuration commands.

See also

The Automatic syntax checking with Git hooks recipe in
  • Chapter 2, Puppet Infrastructure
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem

One of the most important things you can do to make your Puppet manifests clearer and more maintainable is to organize them into modules.

Modules are self-contained bundles of Puppet code that include all the files necessary to implement a thing. Modules may contain flat files, templates, Puppet manifests, custom fact declarations, augeas lenses, and custom Puppet types and providers.

Separating things into modules makes it easier to reuse and share code; it's also the most logical way to organize your manifests. In this example, we'll create a module to manage memcached, a memory caching system commonly used with web applications.

Following are the steps to create an example module:

  1. We will use Puppet's module subcommand to create the directory structure for our new module:
    [email protected]:~$ mkdir -p .puppet/modules
    [email protected]:~$ cd .puppet/modules
    [email protected]:~/.puppet/modules$ puppet module generate thomas-memcached
    We need to create a metadata.json file for this module.  Please answer the following questions; if the question is not applicable to this module, feel free to leave it blank. Puppet uses Semantic Versioning (semver.org) to version modules.What version is this module?  [0.1.0]
    --> Who wrote this module?  [thomas]
    --> What license does this module code fall under?  [Apache 2.0]
    --> How would you describe this module in a single sentence?
    --> A module to install memcached Where is this module's source code repository?
    --> Where can others go to learn more about this module?
    --> Where can others go to file issues about this module?
    --> 
    ----------------------------------------
    {
      "name": "thomas-memcached",
      "version": "0.1.0",
      "author": "thomas",
      "summary": "A module to install memcached",
      "license": "Apache 2.0",
      "source": "",
      "issues_url": null,
      "project_page": null,
      "dependencies": [
        {
          "version_range": ">= 1.0.0",
          "name": "puppetlabs-stdlib"
        }
      ]
    }
    ----------------------------------------
    About to generate this metadata; continue? [n/Y]
    --> y
    Notice: Generating module at /home/thomas/.puppet/modules/thomas-memcached...
    Notice: Populating ERB templates...
    Finished; module generated in thomas-memcached.
    thomas-memcached/manifests
    thomas-memcached/manifests/init.pp
    thomas-memcached/spec
    thomas-memcached/spec/classes
    thomas-memcached/spec/classes/init_spec.rb
    thomas-memcached/spec/spec_helper.rb
    thomas-memcached/README.md
    thomas-memcached/metadata.json
    thomas-memcached/Rakefile
    thomas-memcached/tests
    thomas-memcached/tests/init.pp
    

    This command creates the module directory and creates some empty files as starting points. To use the module, we'll create a symlink to the module name (memcached).

    [email protected]:~/.puppet/modules$ ln –s thomas-memcached memcached
    
  2. Now, edit memcached/manifests/init.pp and change the class definition at the end of the file to the following. Note that puppet module generate created many lines of comments; in a production module you would want to edit those default comments:
    class memcached {
      package { 'memcached':
        ensure => installed,
      }
    
      file { '/etc/memcached.conf':
        source  => 'puppet:///modules/memcached/memcached.conf',
        owner   => 'root',
        group   => 'root',
        mode    => '0644',
        require => Package['memcached'],
      }
      service { 'memcached':
        ensure  => running,
        enable  => true,
        require => [Package['memcached'],
                    File['/etc/memcached.conf']],
      }
    }
  3. Create the modules/thomas-memcached/files directory and then create a file named memcached.conf with the following contents:
    -m 64
    -p 11211
    -u nobody
    -l 127.0.0.1
  4. Change your site.pp file to the following:
    node default {
      include memcached
    }
  5. We would like this module to install memcached. We'll need to run Puppet with root privileges, and we'll use sudo for that. We'll need Puppet to be able to find the module in our home directory; we can specify this on the command line when we run Puppet as shown in the following code snippet:
    [email protected]:~$ sudo puppet apply --modulepath=/home/thomas/.puppet/modules /home/thomas/.puppet/manifests/site.pp
    Notice: Compiled catalog for cookbook.example.com in environment production in 0.33 seconds
    Notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/content: content changed '{md5}a977521922a151c959ac953712840803' to '{md5}9429eff3e3354c0be232a020bcf78f75'
    Notice: Finished catalog run in 0.11 seconds
  6. Check whether the new service is running:
    [email protected]:~$ sudo service memcached status
    [ ok ] memcached is running.

When we created the module using Puppet's module generate command, we used the name thomas-memcached. The name before the hyphen is your username or your username on Puppet forge (an online repository of modules). Since we want Puppet to be able to find the module by the name memcached, we make a symbolic link between thomas-memcached and memcached.

Modules have a specific directory structure. Not all of these directories need to be present, but if they are, this is how they should be organized:

All manifest files (those containing Puppet code) live in the manifests directory. In our example, the memcached class is defined in the manifests/init.pp file, which will be imported automatically.

Inside the memcached class, we refer to the memcached.conf file:

The preceding source parameter tells Puppet to look for the file in:

MODULEPATH/ (/home/thomas/.puppet/modules)

└memcached/

└files/

└memcached.conf

How to do it…

Following are the steps to create an

example module:

  1. We will use Puppet's module subcommand to create the directory structure for our new module:
    [email protected]:~$ mkdir -p .puppet/modules
    [email protected]:~$ cd .puppet/modules
    [email protected]:~/.puppet/modules$ puppet module generate thomas-memcached
    We need to create a metadata.json file for this module.  Please answer the following questions; if the question is not applicable to this module, feel free to leave it blank. Puppet uses Semantic Versioning (semver.org) to version modules.What version is this module?  [0.1.0]
    --> Who wrote this module?  [thomas]
    --> What license does this module code fall under?  [Apache 2.0]
    --> How would you describe this module in a single sentence?
    --> A module to install memcached Where is this module's source code repository?
    --> Where can others go to learn more about this module?
    --> Where can others go to file issues about this module?
    --> 
    ----------------------------------------
    {
      "name": "thomas-memcached",
      "version": "0.1.0",
      "author": "thomas",
      "summary": "A module to install memcached",
      "license": "Apache 2.0",
      "source": "",
      "issues_url": null,
      "project_page": null,
      "dependencies": [
        {
          "version_range": ">= 1.0.0",
          "name": "puppetlabs-stdlib"
        }
      ]
    }
    ----------------------------------------
    About to generate this metadata; continue? [n/Y]
    --> y
    Notice: Generating module at /home/thomas/.puppet/modules/thomas-memcached...
    Notice: Populating ERB templates...
    Finished; module generated in thomas-memcached.
    thomas-memcached/manifests
    thomas-memcached/manifests/init.pp
    thomas-memcached/spec
    thomas-memcached/spec/classes
    thomas-memcached/spec/classes/init_spec.rb
    thomas-memcached/spec/spec_helper.rb
    thomas-memcached/README.md
    thomas-memcached/metadata.json
    thomas-memcached/Rakefile
    thomas-memcached/tests
    thomas-memcached/tests/init.pp
    

    This command creates the module directory and creates some empty files as starting points. To use the module, we'll create a symlink to the module name (memcached).

    [email protected]:~/.puppet/modules$ ln –s thomas-memcached memcached
    
  2. Now, edit memcached/manifests/init.pp and change the class definition at the end of the file to the following. Note that puppet module generate created many lines of comments; in a production module you would want to edit those default comments:
    class memcached {
      package { 'memcached':
        ensure => installed,
      }
    
      file { '/etc/memcached.conf':
        source  => 'puppet:///modules/memcached/memcached.conf',
        owner   => 'root',
        group   => 'root',
        mode    => '0644',
        require => Package['memcached'],
      }
      service { 'memcached':
        ensure  => running,
        enable  => true,
        require => [Package['memcached'],
                    File['/etc/memcached.conf']],
      }
    }
  3. Create the modules/thomas-memcached/files directory and then create a file named memcached.conf with the following contents:
    -m 64
    -p 11211
    -u nobody
    -l 127.0.0.1
  4. Change your site.pp file to the following:
    node default {
      include memcached
    }
  5. We would like this module to install memcached. We'll need to run Puppet with root privileges, and we'll use sudo for that. We'll need Puppet to be able to find the module in our home directory; we can specify this on the command line when we run Puppet as shown in the following code snippet:
    [email protected]:~$ sudo puppet apply --modulepath=/home/thomas/.puppet/modules /home/thomas/.puppet/manifests/site.pp
    Notice: Compiled catalog for cookbook.example.com in environment production in 0.33 seconds
    Notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/content: content changed '{md5}a977521922a151c959ac953712840803' to '{md5}9429eff3e3354c0be232a020bcf78f75'
    Notice: Finished catalog run in 0.11 seconds
  6. Check whether the new service is running:
    [email protected]:~$ sudo service memcached status
    [ ok ] memcached is running.

When we created the module using Puppet's module generate command, we used the name thomas-memcached. The name before the hyphen is your username or your username on Puppet forge (an online repository of modules). Since we want Puppet to be able to find the module by the name memcached, we make a symbolic link between thomas-memcached and memcached.

Modules have a specific directory structure. Not all of these directories need to be present, but if they are, this is how they should be organized:

All manifest files (those containing Puppet code) live in the manifests directory. In our example, the memcached class is defined in the manifests/init.pp file, which will be imported automatically.

Inside the memcached class, we refer to the memcached.conf file:

The preceding source parameter tells Puppet to look for the file in:

MODULEPATH/ (/home/thomas/.puppet/modules)

└memcached/

└files/

└memcached.conf

How it works…

When we created the module using Puppet's module generate command, we used the name thomas-memcached. The name before the hyphen is your username or your username on Puppet forge (an online repository of modules). Since we want Puppet to be able to find the module by the name memcached, we make a symbolic link between thomas-memcached and memcached.

Modules have a specific directory structure. Not all of these directories need to be present, but if they are, this is how they should be organized:

modules/ └MODULE_NAME/ never use a dash (-) in a module name └examples/ example usage of the module └files/ flat files used by the module └lib/ └facter/ define new facts for facter └puppet/ └parser/ └functions/ define a new puppet function, like sort() └provider/ define a provider for a new or existing type └util/ define helper functions (in ruby) └type/ define a new type in puppet └manifests/ └init.pp class MODULE_NAME { } └spec/ rSpec tests └templates/ erb template files used by the module

All manifest
There's more…

Learn to love modules because they'll make your Puppet life a lot easier. They're not complicated, however, practice and experience will help you judge when things should be grouped into modules, and how best to arrange your module structure. Modules can hold more than manifests and files as we'll see in the next two sections. Templates

If you need to use a template Facts, functions, types, and providers

Modules can also contain Third-party modules

You can download modules provided by
Module organization

For more details on how to
See also

The Creating custom facts recipe in
  • Chapter 9, External Tools and the Puppet Ecosystem
  • The Using public modules recipe in Chapter 7, Managing Applications
  • The Creating your own resource types recipe in Chapter 9, External Tools and the Puppet Ecosystem
  • The Creating your own providers recipe in Chapter 9, External Tools and the Puppet Ecosystem

Choosing appropriate and informative names for your modules and classes will be a big help when it comes to maintaining your code. This is even truer if other people need to read and work on your manifests.

Here are some tips on how to name things in your manifests:

  1. Name modules after the software or service they manage, for example, apache or haproxy.
  2. Name classes within modules (subclasses) after the function or service they provide to the module, for example, apache::vhosts or rails::dependencies.
  3. If a class within a module disables the service provided by that module, name it disabled. For example, a class that disables Apache should be named apache::disabled.
  4. Create a roles and profiles hierarchy of modules. Each node should have a single role consisting of one or more profiles. Each profile module should configure a single service.
  5. The module that manages users should be named user.
  6. Within the user module, declare your virtual users within the class user::virtual (for more on virtual users and other resources, see the Using virtual resources recipe in Chapter 5, Users and Virtual Resources).
  7. Within the user module, subclasses for particular groups of users should be named after the group, for example, user::sysadmins or user::contractors.
  8. When using Puppet to deploy the config files for different services, name the file after the service, but with a suffix indicating what kind of file it is, for example:
    • Apache init script: apache.init
    • Logrotate config snippet for Rails: rails.logrotate
    • Nginx vhost file for mywizzoapp: mywizzoapp.vhost.nginx
    • MySQL config for standalone server: standalone.mysql
  9. If you need to deploy a different version of a file depending on the operating system release, for example, you can use a naming convention like the following:
    memcached.lucid.conf
    memcached.precise.conf
  10. You can have Puppet automatically select the appropriate version as follows:
    source = > "puppet:///modules/memcached
      /memcached.${::lsbdistrelease}.conf",
  11. If you need to manage, for example, different Ruby versions, name the class after the version it is responsible for, for example, ruby192 or ruby186.

Puppet community maintains a set of best practice guidelines for your Puppet infrastructure, which includes some hints on naming conventions:

http://docs.puppetlabs.com/guides/best_practices.html

Some people prefer to include multiple classes on a node by using a comma-separated list, rather than separate include statements, for example:

  node 'server014' inherits 'server' {
    include mail::server, repo::gem, repo::apt, zabbix
  }

This is a matter of style, but I prefer to use separate include statements, one on a line, because it makes it easier to copy and move around class inclusions between nodes without having to tidy up the commas and indentation every time.

I mentioned inheritance in a couple of the preceding examples; if you're not sure what this is, don't worry, I'll explain this in detail in the next chapter.

How to do it…

Here are some tips on how to name things in your manifests:

Name modules after the software or service they manage, for example, apache or haproxy.
Name classes within modules (subclasses) after the function or service they provide to the module, for example, apache::vhosts or rails::dependencies.
If a class within a module disables the service provided by that module, name it disabled. For example, a class that disables Apache should be named apache::disabled.
Create a roles and profiles hierarchy of modules. Each node should have a single role consisting of one or more profiles. Each profile module should configure a single service.
The module that manages users should be named user.
Within the user module, declare your virtual users within the class user::virtual (for more on virtual users and other resources, see the Using virtual resources recipe in

Puppet community maintains a set of best practice guidelines for your Puppet infrastructure, which includes some hints on naming conventions:

http://docs.puppetlabs.com/guides/best_practices.html

Some people prefer to include multiple classes on a node by using a comma-separated list, rather than separate include statements, for example:

  node 'server014' inherits 'server' {
    include mail::server, repo::gem, repo::apt, zabbix
  }

This is a matter of style, but I prefer to use separate include statements, one on a line, because it makes it easier to copy and move around class inclusions between nodes without having to tidy up the commas and indentation every time.

I mentioned inheritance in a couple of the preceding examples; if you're not sure what this is, don't worry, I'll explain this in detail in the next chapter.

There's more…

Puppet community maintains a set of best practice guidelines

for your Puppet infrastructure, which includes some hints on naming conventions:

http://docs.puppetlabs.com/guides/best_practices.html

Some people prefer to include multiple classes on a node by using a comma-separated list, rather than separate include statements, for example:

  node 'server014' inherits 'server' {
    include mail::server, repo::gem, repo::apt, zabbix
  }

This is a matter of style, but I prefer to use separate include statements, one on a line, because it makes it easier to copy and move around class inclusions between nodes without having to tidy up the commas and indentation every time.

I mentioned inheritance in a couple of the preceding examples; if you're not sure what this is, don't worry, I'll explain this in detail in the next chapter.

Templates are a powerful way of using Embedded Ruby (ERB) to help build config files dynamically. You can also use ERB syntax directly without having to use a separate file by calling the inline_template function. ERB allows you to use conditional logic, iterate over arrays, and include variables.

How to do it…

Here's an example of
How it works…

Anything inside the string passed to inline_template is executed as if it were an ERB template. That is, anything inside the <%= and %> delimiters will be executed as Ruby code, and the rest will be treated as a string.

In this example, we use inline_template to compute a different hour for this cron resource (a scheduled job) for each machine, so that the same job does not run at the same time on all machines. For more on this technique, see the Distributing cron jobs efficiently recipe in There's more...

In ERB code, whether See also

The Using ERB templates recipe in
  • Chapter 4, Working with Files and Packages
  • The Using array iteration in templates recipe in Chapter 4, Working with Files and Packages

Arrays are a powerful feature in Puppet; wherever you want to perform the same operation on a list of things, an array may be able to help. You can create an array just by putting its content in square brackets:

Although arrays will take you a long way with Puppet, it's also useful to know about an even more flexible data structure: the hash.

You can declare literal arrays using square brackets, as follows:

Now, when we run Puppet on the preceding code, we see the following notice messages in the output:

However, Puppet can also create arrays for you from strings, using the split function, as follows:

Running puppet apply against this new manifest, we see the same messages in the output:

Note that split takes two arguments: the first argument is the string to be split. The second argument is the character to split on; in this example, a single space. As Puppet works its way through the string, when it encounters a space, it will interpret it as the end of one item and the beginning of the next. So, given the string 'egg beans chips', this will be split into three items.

The character to split on can be any character or string:

The character can also be a regular expression, for example, a set of alternatives separated by a | (pipe) character:

How to do it…

Here's a common example of how arrays are used:

Add the following code to your manifest:
$packages = [ 'ruby1.8-dev',
  'ruby1.8',
  'ri1.8',
  'rdoc1.8',
  'irb1.8',
  'libreadline-ruby1.8',
  'libruby1.8',
  'libopenssl-ruby' ]

package { $packages: ensure => installed }
Run Puppet and note that each package should now be installed.

Although arrays will take you a long way with Puppet, it's also useful to know about an even more flexible data structure: the hash.

You can declare literal arrays using square brackets, as follows:

Now, when we run Puppet on the preceding code, we see the following notice messages in the output:

However, Puppet can also create arrays for you from strings, using the split function, as follows:

Running puppet apply against this new manifest, we see the same messages in the output:

Note that split takes two arguments: the first argument is the string to be split. The second argument is the character to split on; in this example, a single space. As Puppet works its way through the string, when it encounters a space, it will interpret it as the end of one item and the beginning of the next. So, given the string 'egg beans chips', this will be split into three items.

The character to split on can be any character or string:

The character can also be a regular expression, for example, a set of alternatives separated by a | (pipe) character:

How it works…

Where Puppet encounters an array as the name of a resource, it creates a resource for each element in the array. In the example, a new package resource is created for each of the packages in the $packages array, with the same parameters (ensure => installed). This is a very compact way to instantiate many similar resources.

Although arrays will take you a long way with Puppet, it's also useful to know about an even more flexible data structure: the hash.

You can declare literal arrays using square brackets, as follows:

Now, when we run Puppet on the preceding code, we see the following notice messages in the output:

However, Puppet can also create arrays for you from strings, using the split function, as follows:

Running puppet apply against this new manifest, we see the same messages in the output:

Note that split takes two arguments: the first argument is the string to be split. The second argument is the character to split on; in this example, a single space. As Puppet works its way through the string, when it encounters a space, it will interpret it as the end of one item and the beginning of the next. So, given the string 'egg beans chips', this will be split into three items.

The character to split on can be any character or string:

The character can also be a regular expression, for example, a set of alternatives separated by a | (pipe) character:

There's more…

Although arrays will take you

a long way with Puppet, it's also useful to know about an even more flexible data structure: the hash.

You can declare literal arrays using square brackets, as follows:

Now, when we run Puppet on the preceding code, we see the following notice messages in the output:

However, Puppet can also create arrays for you from strings, using the split function, as follows:

Running puppet apply against this new manifest, we see the same messages in the output:

Note that split takes two arguments: the first argument is the string to be split. The second argument is the character to split on; in this example, a single space. As Puppet works its way through the string, when it encounters a space, it will interpret it as the end of one item and the beginning of the next. So, given the string 'egg beans chips', this will be split into three items.

The character to split on can be any character or string:

The character can also be a regular expression, for example, a set of alternatives separated by a | (pipe) character:

Using hashes

A hash is like an

You can declare literal arrays using square brackets, as follows:

Now, when we run Puppet on the preceding code, we see the following notice messages in the output:

However, Puppet can also create arrays for you from strings, using the split function, as follows:

Running puppet apply against this new manifest, we see the same messages in the output:

Note that split takes two arguments: the first argument is the string to be split. The second argument is the character to split on; in this example, a single space. As Puppet works its way through the string, when it encounters a space, it will interpret it as the end of one item and the beginning of the next. So, given the string 'egg beans chips', this will be split into three items.

The character to split on can be any character or string:

The character can also be a regular expression, for example, a set of alternatives separated by a | (pipe) character:

Creating arrays with the split function

You can declare literal

Puppet's if statement allows you to change the manifest behavior based on the value of a variable or an expression. With it, you can apply different resources or parameter values depending on certain facts about the node, for example, the operating system, or the memory size.

You can also set variables within the manifest, which can change the behavior of included classes. For example, nodes in data center A might need to use different DNS servers than nodes in data center B, or you might need to include one set of classes for an Ubuntu system, and a different set for other systems.

How to do it…

Here's an example of a useful
How it works…

Puppet treats whatever follows an if keyword as an expression and evaluates it. If the expression evaluates to true, Puppet will execute the code within the curly braces.

Optionally, you can add an else branch, which will be executed if the expression evaluates to false. There's more…

Here are some more tips on using if statements. Elseif branches

You can add Comparisons

You can check whether two values Combining expressions

You can put together the See also

The Using the in operator recipe in this chapter
The Using selectors and case statements recipe in this chapter

Another kind of expression you can test in if statements and other conditionals is the regular expression. A regular expression is a powerful way to compare strings using pattern matching.

How to do it…

This is one example of using a regular expression in a conditional statement. Add the following to your manifest:

if $::architecture =~ /64/ { notify { '64Bit OS Installed': } } else { notify { 'Upgrade to 64Bit': } fail('Not 64 Bit') }
How it works…

Puppet treats the text supplied
There's more…

Regular expressions are very powerful, but can be difficult to understand and debug. If you find yourself using a regular expression so complex that you can't see at a glance what it does, think about simplifying your design to make it easier. However, one particularly useful feature of regular expressions is the ability to capture patterns. Capturing patterns

You can not only match text using a Regular expression syntax

Puppet's regular See also

Refer to the Using regular expression substitutions recipe in this chapter

Although you could write any conditional statement using if, Puppet provides a couple of extra forms to help you express conditionals more easily: the selector and the case statement.

Our example demonstrates both the selector and the case statement, so let's see in detail how each of them works.

How to do it…

Here are some

Our example demonstrates both the selector and the case statement, so let's see in detail how each of them works.

How it works…

Our example demonstrates both the selector and the case statement, so let's see in detail how each of them works.

Selector

In the first example, we
Case statement

Unlike selectors, the case statement does
There's more…

Once you've got a grip of the basic use of selectors and case statements, you may find the following tips useful. Regular expressions

As with if statements, you can use Defaults

Both selectors

The in operator tests whether one string contains another string. Here's an example:

The preceding expression is true if the spring string is a substring of springfield, which it is. The in operator can also test for membership of arrays as follows:

When in is used with a hash, it tests whether the string is a key of the hash:

How to do it…

The following steps will show you how to use the in operator:

Add the following code to your manifest:
if $::operatingsystem in [ 'Ubuntu', 'Debian' ] {
  notify { 'Debian-type operating system detected': }
} elseif $::operatingsystem in [ 'RedHat', 'Fedora', 'SuSE', 'CentOS' ] {
  notify { 'RedHat-type operating system detected': }
} else {
  notify { 'Some other operating system detected': }
}
Run There's more…

The value of an in expression is Boolean (true or false) so you can assign it to a variable:

$debianlike = $::operatingsystem in [ 'Debian', 'Ubuntu' ] if $debianlike { notify { 'You are in a maze of twisty little packages, all alike': } }

Puppet's regsubst function provides an easy way to manipulate text, search and replace expressions within strings, or extract patterns from strings. We often need to do this with data obtained from a fact, for example, or from external programs.

In this example, we'll see how to use regsubst to extract the first three octets of an IPv4 address (the network part, assuming it's a /24 class C address).

How to do it…

Follow these steps to build the example:

Add the following code to your manifest:
$class_c = regsubst($::ipaddress, '(.*)\..*', '\1.0')
notify { "The network part of ${::ipaddress} is ${class_c}": }
Run Puppet:
[email protected]:~/.puppet/manifests$ puppet apply ipaddress.pp 
Notice: Compiled catalog for cookbook.example.com in environment production in 0.02 seconds
Notice: The network part of 192.168.122.148 is
  192.168.122.0
Notice: /Stage[main]/Main/Notify[The network part of 192.168.122.148 is
  192.168.122.0]/message: defined 'message' as 'The network part of 192.168.122.148 is
  192.168.122.0'
Notice: Finished catalog run in 0.03 seconds
How it works…

The regsubst function There's more…

The pattern function can be any regular expression, using the same (Ruby) syntax as regular expressions in if statements.

See also

The Importing dynamic information recipe in
  • Chapter 3, Writing Better Manifests
  • The Getting information about the environment recipe in Chapter 3, Writing Better Manifests
  • The Using regular expressions in if statements recipe in this chapter

Puppet language is evolving at the moment; many features that are expected to be included in the next major release (4) are available if you enable the future parser.

Many of the experimental features deal with how code is evaluated, for example, in an earlier example we compared the value of the $::facterversion fact with a number, but the value is treated as a string so the code fails to compile. Using the future parser, the value is converted and no error is reported as shown in the following command line output:

You can concatenate arrays with the + operator or append them with the << operator. In the following example, we use the ternary operator to assign a specific package name to the $apache variable. We then append that value to an array using the << operator:

If we have two arrays, we can use the + operator to concatenate the two arrays. In this example, we define an array of system administrators ($sysadmins) and another array of application owners ($appowners). We can then concatenate the array and use it as an argument to our allowed users:

When we apply this manifest, we see that the two arrays have been joined as shown in the following command line output:

If we have two hashes, we can merge them using the same + operator we used for arrays. Consider our $interfaces hash from a previous example; we can add another interface to the hash:

When we apply this manifest, we see that the route attribute has been merged into the hash (your results may differ, the order in which the hash prints is unpredictable), as follows:

Lambda functions are iterators applied to arrays or hashes. You iterate through the array or hash and apply an iterator function such as each, map, filter, reduce, or slice to each element of the array or key of the hash. Some of the lambda functions return a calculated array or value; others such as each only return the input array or hash.

Lambda functions such as map and reduce use temporary variables that are thrown away after the lambda has finished. Use of lambda functions is something best shown by example. In the next few sections, we will show an example usage of each of the lambda functions.

Getting ready

Ensure that the rgen gem is installed.
Set parser = future in the [main] section of your puppet.conf(/etc/puppet/puppet.conf for open source Puppet as root,/etc/puppetlabs/puppet/puppet.conf for Puppet Enterprise, and~/.puppet/puppet.conf for a non-root user running puppet).
To temporarily test with the future parser, use --parser=future on the command line.

Many of the experimental features deal with how code is evaluated, for example, in an earlier example we compared the value of the $::facterversion fact with a number, but the value is treated as a string so the code fails to compile. Using the future parser, the value is converted and no error is reported as shown in the following command line output:

You can concatenate arrays with the + operator or append them with the << operator. In the following example, we use the ternary operator to assign a specific package name to the $apache variable. We then append that value to an array using the << operator:

If we have two arrays, we can use the + operator to concatenate the two arrays. In this example, we define an array of system administrators ($sysadmins) and another array of application owners ($appowners). We can then concatenate the array and use it as an argument to our allowed users:

When we apply this manifest, we see that the two arrays have been joined as shown in the following command line output:

If we have two hashes, we can merge them using the same + operator we used for arrays. Consider our $interfaces hash from a previous example; we can add another interface to the hash:

When we apply this manifest, we see that the route attribute has been merged into the hash (your results may differ, the order in which the hash prints is unpredictable), as follows:

Lambda functions are iterators applied to arrays or hashes. You iterate through the array or hash and apply an iterator function such as each, map, filter, reduce, or slice to each element of the array or key of the hash. Some of the lambda functions return a calculated array or value; others such as each only return the input array or hash.

Lambda functions such as map and reduce use temporary variables that are thrown away after the lambda has finished. Use of lambda functions is something best shown by example. In the next few sections, we will show an example usage of each of the lambda functions.

How to do it...

Many of the experimental features deal with how code is evaluated, for example, in an earlier example we compared the value of the $::facterversion fact with a number, but the value is treated as a string so the code fails to compile. Using the future parser, the value is converted and no error is reported as shown in the following command line output:

[email protected]:~/.puppet/manifests$ puppet apply --parser=future version.pp Notice: Compiled catalog for cookbook.example.com in environment production in 0.36 seconds Notice: Finished catalog run in 0.03 seconds

You can concatenate arrays with the + operator or append them with the << operator. In the following example, we use the ternary operator to assign a specific package name to the $apache variable. We then append that value to an array using the << operator:

If we have two arrays, we can use the + operator to concatenate the two arrays. In this example, we define an array of system administrators ($sysadmins) and another array of application owners ($appowners). We can then concatenate the array and use it as an argument to our allowed users:

When we apply this manifest, we see that the two arrays have been joined as shown in the following command line output:

If we have two hashes, we can merge them using the same + operator we used for arrays. Consider our $interfaces hash from a previous example; we can add another interface to the hash:

When we apply this manifest, we see that the route attribute has been merged into the hash (your results may differ, the order in which the hash prints is unpredictable), as follows:

Lambda functions are iterators applied to arrays or hashes. You iterate through the array or hash and apply an iterator function such as each, map, filter, reduce, or slice to each element of the array or key of the hash. Some of the lambda functions return a calculated array or value; others such as each only return the input array or hash.

Lambda functions such as map and reduce use temporary variables that are thrown away after the lambda has finished. Use of lambda functions is something best shown by example. In the next few sections, we will show an example usage of each of the lambda functions.

Appending to and concatenating arrays

You can concatenate

arrays with the + operator or append them with the << operator. In the following example, we use the ternary operator to assign a specific package name to the $apache variable. We then append that value to an array using the << operator:

If we have two arrays, we can use the + operator to concatenate the two arrays. In this example, we define an array of system administrators ($sysadmins) and another array of application owners ($appowners). We can then concatenate the array and use it as an argument to our allowed users:

When we apply this manifest, we see that the two arrays have been joined as shown in the following command line output:

If we have two hashes, we can merge them using the same + operator we used for arrays. Consider our $interfaces hash from a previous example; we can add another interface to the hash:

When we apply this manifest, we see that the route attribute has been merged into the hash (your results may differ, the order in which the hash prints is unpredictable), as follows:

Lambda functions are iterators applied to arrays or hashes. You iterate through the array or hash and apply an iterator function such as each, map, filter, reduce, or slice to each element of the array or key of the hash. Some of the lambda functions return a calculated array or value; others such as each only return the input array or hash.

Lambda functions such as map and reduce use temporary variables that are thrown away after the lambda has finished. Use of lambda functions is something best shown by example. In the next few sections, we will show an example usage of each of the lambda functions.

Lambda functions

Lambda functions

are iterators applied to arrays or hashes. You iterate through the array or hash and apply an iterator function such as each, map, filter, reduce, or slice to each element of the array or key of the hash. Some of the lambda functions return a calculated array or value; others such as each only return the input array or hash.

Lambda functions such as map and reduce use temporary variables that are thrown away after the lambda has finished. Use of lambda functions is something best shown by example. In the next few sections, we will show an example usage of each of the lambda functions.

Reduce

Reduce
Filter

Filter is
Map

Map
Slice

Slice is useful Each

Each Other features

There are other new features of Puppet language available when using the future parser. Some increase readability or compactness of code. For more information, refer to the documentation on
 
 

"Computers in the future may have as few as 1,000 vacuum tubes and weigh only 1.5 tons."

 
 --Popular Mechanics, 1949

In this chapter, we will cover:

In this chapter, we will cover how to deploy Puppet in a centralized and decentralized manner. With each approach, we'll see a combination of best practices, my personal experience, and community solutions.

We'll configure and use both PuppetDB and Hiera. PuppetDB is used with exported resources, which we will cover in Chapter 5, Users and Virtual Resources. Hiera is used to separate variable data from Puppet code.

Finally, I'll introduce Git and see how to use Git to organize our code and our infrastructure.

Because Linux distributions, such as Ubuntu, Red Hat, and CentOS, differ in the specific details of package names, configuration file paths, and many other things, I have decided that for reasons of space and clarity the best approach for this book is to pick one distribution (Debian 7 named as Wheezy) and stick to that. However, Puppet runs on most popular operating systems, so you should have very little trouble adapting the recipes to your own favorite OS and distribution.

At the time of writing, Puppet 3.7.2 is the latest stable version available, this is the version of Puppet used in the book. The syntax of Puppet commands changes often, so be aware that while older versions of Puppet are still perfectly usable, they may not support all of the features and syntax described in this book. As we saw in Chapter 1, Puppet Language and Style, the future parser showcases features of the language scheduled to become default in Version 4 of Puppet.

Getting ready

If your Linux distribution uses APT for package management, go to How to do it...

Once you have found the appropriate Puppet labs release package for your distribution, the steps to install Puppet are the same for either APT or YUM:
  • Install Puppet labs release package
  • Install Puppet package
Once you have installed Puppet, verify the version of Puppet as shown in the following example:
[email protected]:~ puppet --version 3.7.2

Now that

It's a great idea to put your Puppet manifests in a version control system such as Git or Subversion (Git is the de facto standard for Puppet). This gives you several advantages:

Follow these steps:

  1. First, install Git on your Git server (git.example.com in our example). The easiest way to do this is using Puppet. Create the following manifest, call it git.pp:
      package {'git':
        ensure => installed }
  2. Apply this manifest using puppet apply git.pp, this will install Git.
  3. Next, create a Git user that the nodes will use to log in and retrieve the latest code. Again, we'll do this with puppet. We'll also create a directory to hold our repository (/home/git/repos) as shown in the following code snippet:
    group { 'git': gid => 1111, }
    user {'git': uid => 1111, gid => 1111, comment => 'Git User', home => '/home/git', require => Group['git'], }
    file {'/home/git': ensure => 'directory', owner => 1111, group => 1111, require => User['git'], }
    file {'/home/git/repos': ensure => 'directory', owner => 1111, group => 1111, require => File['/home/git'] }
  4. After applying that manifest, log in as the Git user and create an empty Git repository using the following command:
    # sudo -iu git [email protected] $ cd repos [email protected] $ git init --bare puppet.git Initialized empty Git repository in /home/git/repos/puppet.git/
    
  5. Set a password for the Git user, we'll need to log in remotely after the next step:
    [[email protected] ~]# passwd git
    Changing password for user git.
    New password: 
    Retype new password: 
    passwd: all authentication tokens updated successfully.
    
  6. Now back on your local machine, create an ssh key for our nodes to use to update the repository:
    [email protected] ~ $ cd .ssh
    [email protected] ~/.ssh $ ssh-keygen -b 4096 -f git_rsa
    Generating public/private rsa key pair.
    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again: 
    Your identification has been saved in git_rsa.
    Your public key has been saved in git_rsa.pub.
    The key fingerprint is:
    87:35:0e:4e:d2:96:5f:e4:ce:64:4a:d5:76:c8:2b:e4 [email protected]
  7. Now copy the newly created public key to the authorized_keys file. This will allow us to connect to the Git server using this new key:
    [email protected] ~/.ssh $ ssh-copy-id -i git_rsa [email protected]
    [email protected]'s password: 
    Number of key(s) added: 1
  8. Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added.
  9. Next, configure ssh to use your key when accessing the Git server and add the following to your ~/.ssh/config file:
    Host git git.example.com
      User git
      IdentityFile /home/thomas/.ssh/git_rsa
    
  10. Clone the repo onto your machine into a directory named Puppet (substitute your server name if you didn't use git.example.com):
    [email protected] ~$ git clone [email protected]:repos/puppet.git
    Cloning into 'puppet'...
    warning: You appear to have cloned an empty repository.
    Checking connectivity... done.

    We've created a Git repository; before we commit any changes to the repository, it's a good idea to set your name and e-mail in Git. Your name and e-mail will be appended to each commit you make.

  11. When you are working in a large team, knowing who made a change is very important; for this, use the following code snippet:
    [email protected] puppet$ git config --global user.email"[email protected]"
    [email protected] puppet$ git config --global user.name "ThomasUphill"
    
  12. You can verify your Git settings using the following snippet:
    [email protected] ~$ git config --global --list
    user.name=Thomas Uphill
    user[email protected]
    core.editor=vim
    merge.tool=vimdiff
    color.ui=true
    push.default=simple
    
  13. Now that we have Git configured properly, change directory to your repository directory and create a new site manifest as shown in the following snippet:
    [email protected] ~$ cd puppet
    [email protected] puppet$ mkdir manifests
    [email protected] puppet$ vim manifests/site.pp
    node default {
      include base
    }
    
  14. This site manifest will install our base class on every node; we will create the base class using the Puppet module as we did in Chapter 1, Puppet Language and Style:
    [email protected] puppet$ mkdir modules
    [email protected] puppet$ cd modules
    [email protected] modules$ puppet module generate thomas-base
    Notice: Generating module at /home/tuphill/puppet/modules/thomas-base
    thomas-base
    thomas-base/Modulefile
    thomas-base/README
    thomas-base/manifests
    thomas-base/manifests/init.pp
    thomas-base/spec
    thomas-base/spec/spec_helper.rb
    thomas-base/tests
    thomas-base/tests/init.pp
    [email protected] modules$ ln -s thomas-base base
    
  15. As a last step, we create a symbolic link between the thomas-base directory and base. Now to make sure our module does something useful, add the following to the body of the base class defined in thomas-base/manifests/init.pp:
    class base {
      file {'/etc/motd':
        content => "${::fqdn}\nManaged by puppet ${::puppetversion}\n"
      }
    }
    
  16. Now add the new base module and site manifest to Git using git add and git commit as follows:
    [email protected] modules$ cd ..
    [email protected] puppet$ git add modules manifests
    [email protected] puppet$ git status
    On branch master
    Initial commit
    Changes to be committed:
      (use "git rm --cached <file>..." to unstage)
    new file:   manifests/site.pp
    new file:   modules/base
    new file:   modules/thomas-base/Modulefile
    new file:   modules/thomas-base/README
    new file:   modules/thomas-base/manifests/init.pp
    new file:   modules/thomas-base/spec/spec_helper.rb
    new file:   modules/thomas-base/tests/init.pp
    [email protected] puppet$ git commit -m "Initial commit with simple base module"
    [master (root-commit) 3e1f837] Initial commit with simple base module
     7 files changed, 102 insertions(+)
     create mode 100644 manifests/site.pp
     create mode 120000 modules/base
     create mode 100644 modules/thomas-base/Modulefile
     create mode 100644 modules/thomas-base/README
     create mode 100644 modules/thomas-base/manifests/init.pp
     create mode 100644 modules/thomas-base/spec/spec_helper.rb
     create mode 100644 modules/thomas-base/tests/init.pp
    
  17. At this point your changes to the Git repository have been committed locally; you now need to push those changes back to git.example.com so that other nodes can retrieve the updated files:
    [email protected] puppet$ git push origin master
    Counting objects: 15, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (9/9), done.
    Writing objects: 100% (15/15), 2.15 KiB | 0 bytes/s, done.
    Total 15 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
     * [new branch]      master -> master
    
Getting ready

In this section, we'll import your existing manifest files into Git. If you have created a Puppet directory in a previous section use that, otherwise, use your existing manifest directory.

In this example, we'll create a new Git repository on a server accessible from all our nodes. There are several steps we need to take to have our code held in a Git repository:

Install Git on a central server.
Create a user to run Git and own the repository.
Create a repository to hold the code.
Create SSH keys

Follow these steps:

  1. First, install Git on your Git server (git.example.com in our example). The easiest way to do this is using Puppet. Create the following manifest, call it git.pp:
      package {'git':
        ensure => installed }
  2. Apply this manifest using puppet apply git.pp, this will install Git.
  3. Next, create a Git user that the nodes will use to log in and retrieve the latest code. Again, we'll do this with puppet. We'll also create a directory to hold our repository (/home/git/repos) as shown in the following code snippet:
    group { 'git': gid => 1111, }
    user {'git': uid => 1111, gid => 1111, comment => 'Git User', home => '/home/git', require => Group['git'], }
    file {'/home/git': ensure => 'directory', owner => 1111, group => 1111, require => User['git'], }
    file {'/home/git/repos': ensure => 'directory', owner => 1111, group => 1111, require => File['/home/git'] }
  4. After applying that manifest, log in as the Git user and create an empty Git repository using the following command:
    # sudo -iu git [email protected] $ cd repos [email protected] $ git init --bare puppet.git Initialized empty Git repository in /home/git/repos/puppet.git/
    
  5. Set a password for the Git user, we'll need to log in remotely after the next step:
    [[email protected] ~]# passwd git
    Changing password for user git.
    New password: 
    Retype new password: 
    passwd: all authentication tokens updated successfully.
    
  6. Now back on your local machine, create an ssh key for our nodes to use to update the repository:
    [email protected] ~ $ cd .ssh
    [email protected] ~/.ssh $ ssh-keygen -b 4096 -f git_rsa
    Generating public/private rsa key pair.
    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again: 
    Your identification has been saved in git_rsa.
    Your public key has been saved in git_rsa.pub.
    The key fingerprint is:
    87:35:0e:4e:d2:96:5f:e4:ce:64:4a:d5:76:c8:2b:e4 [email protected]
  7. Now copy the newly created public key to the authorized_keys file. This will allow us to connect to the Git server using this new key:
    [email protected] ~/.ssh $ ssh-copy-id -i git_rsa [email protected]
    [email protected]'s password: 
    Number of key(s) added: 1
  8. Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added.
  9. Next, configure ssh to use your key when accessing the Git server and add the following to your ~/.ssh/config file:
    Host git git.example.com
      User git
      IdentityFile /home/thomas/.ssh/git_rsa
    
  10. Clone the repo onto your machine into a directory named Puppet (substitute your server name if you didn't use git.example.com):
    [email protected] ~$ git clone [email protected]:repos/puppet.git
    Cloning into 'puppet'...
    warning: You appear to have cloned an empty repository.
    Checking connectivity... done.

    We've created a Git repository; before we commit any changes to the repository, it's a good idea to set your name and e-mail in Git. Your name and e-mail will be appended to each commit you make.

  11. When you are working in a large team, knowing who made a change is very important; for this, use the following code snippet:
    [email protected] puppet$ git config --global user.email"[email protected]"
    [email protected] puppet$ git config --global user.name "ThomasUphill"
    
  12. You can verify your Git settings using the following snippet:
    [email protected] ~$ git config --global --list
    user.name=Thomas Uphill
    [email protected]
    core.editor=vim
    merge.tool=vimdiff
    color.ui=true
    push.default=simple
    
  13. Now that we have Git configured properly, change directory to your repository directory and create a new site manifest as shown in the following snippet:
    [email protected] ~$ cd puppet
    [email protected] puppet$ mkdir manifests
    [email protected] puppet$ vim manifests/site.pp
    node default {
      include base
    }
    
  14. This site manifest will install our base class on every node; we will create the base class using the Puppet module as we did in Chapter 1, Puppet Language and Style:
    [email protected] puppet$ mkdir modules
    [email protected] puppet$ cd modules
    [email protected] modules$ puppet module generate thomas-base
    Notice: Generating module at /home/tuphill/puppet/modules/thomas-base
    thomas-base
    thomas-base/Modulefile
    thomas-base/README
    thomas-base/manifests
    thomas-base/manifests/init.pp
    thomas-base/spec
    thomas-base/spec/spec_helper.rb
    thomas-base/tests
    thomas-base/tests/init.pp
    [email protected] modules$ ln -s thomas-base base
    
  15. As a last step, we create a symbolic link between the thomas-base directory and base. Now to make sure our module does something useful, add the following to the body of the base class defined in thomas-base/manifests/init.pp:
    class base {
      file {'/etc/motd':
        content => "${::fqdn}\nManaged by puppet ${::puppetversion}\n"
      }
    }
    
  16. Now add the new base module and site manifest to Git using git add and git commit as follows:
    [email protected] modules$ cd ..
    [email protected] puppet$ git add modules manifests
    [email protected] puppet$ git status
    On branch master
    Initial commit
    Changes to be committed:
      (use "git rm --cached <file>..." to unstage)
    new file:   manifests/site.pp
    new file:   modules/base
    new file:   modules/thomas-base/Modulefile
    new file:   modules/thomas-base/README
    new file:   modules/thomas-base/manifests/init.pp
    new file:   modules/thomas-base/spec/spec_helper.rb
    new file:   modules/thomas-base/tests/init.pp
    [email protected] puppet$ git commit -m "Initial commit with simple base module"
    [master (root-commit) 3e1f837] Initial commit with simple base module
     7 files changed, 102 insertions(+)
     create mode 100644 manifests/site.pp
     create mode 120000 modules/base
     create mode 100644 modules/thomas-base/Modulefile
     create mode 100644 modules/thomas-base/README
     create mode 100644 modules/thomas-base/manifests/init.pp
     create mode 100644 modules/thomas-base/spec/spec_helper.rb
     create mode 100644 modules/thomas-base/tests/init.pp
    
  17. At this point your changes to the Git repository have been committed locally; you now need to push those changes back to git.example.com so that other nodes can retrieve the updated files:
    [email protected] puppet$ git push origin master
    Counting objects: 15, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (9/9), done.
    Writing objects: 100% (15/15), 2.15 KiB | 0 bytes/s, done.
    Total 15 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
     * [new branch]      master -> master
    
How to do it...

Follow these steps:

First, install
  1. Git on your Git server (git.example.com in our example). The easiest way to do this is using Puppet. Create the following manifest, call it git.pp:
      package {'git':
        ensure => installed }
  2. Apply this manifest using puppet apply git.pp, this will install Git.
  3. Next, create a Git user that the nodes will use to log in and retrieve the latest code. Again, we'll do this with puppet. We'll also create a directory to hold our repository (/home/git/repos) as shown in the following code snippet:
    group { 'git': gid => 1111, }
    user {'git': uid => 1111, gid => 1111, comment => 'Git User', home => '/home/git', require => Group['git'], }
    file {'/home/git': ensure => 'directory', owner => 1111, group => 1111, require => User['git'], }
    file {'/home/git/repos': ensure => 'directory', owner => 1111, group => 1111, require => File['/home/git'] }
  4. After applying that manifest, log in as the Git user and create an empty Git repository using the following command:
    # sudo -iu git [email protected] $ cd repos [email protected] $ git init --bare puppet.git Initialized empty Git repository in /home/git/repos/puppet.git/
    
  5. Set a password for the Git user, we'll need to log in remotely after the next step:
    [[email protected] ~]# passwd git
    Changing password for user git.
    New password: 
    Retype new password: 
    passwd: all authentication tokens updated successfully.
    
  6. Now back on your local machine, create an ssh key for our nodes to use to update the repository:
    [email protected] ~ $ cd .ssh
    [email protected] ~/.ssh $ ssh-keygen -b 4096 -f git_rsa
    Generating public/private rsa key pair.
    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again: 
    Your identification has been saved in git_rsa.
    Your public key has been saved in git_rsa.pub.
    The key fingerprint is:
    87:35:0e:4e:d2:96:5f:e4:ce:64:4a:d5:76:c8:2b:e4 [email protected]
  7. Now copy the newly created public key to the authorized_keys file. This will allow us to connect to the Git server using this new key:
    [email protected] ~/.ssh $ ssh-copy-id -i git_rsa [email protected]
    [email protected]'s password: 
    Number of key(s) added: 1
  8. Now try logging into the machine, with: "ssh '[email protected]'" and check to make sure that only the key(s) you wanted were added.
  9. Next, configure ssh to use your key when accessing the Git server and add the following to your ~/.ssh/config file:
    Host git git.example.com
      User git
      IdentityFile /home/thomas/.ssh/git_rsa
    
  10. Clone the repo onto your machine into a directory named Puppet (substitute your server name if you didn't use git.example.com):
    [email protected] ~$ git clone [email protected]:repos/puppet.git
    Cloning into 'puppet'...
    warning: You appear to have cloned an empty repository.
    Checking connectivity... done.

    We've created a Git repository; before we commit any changes to the repository, it's a good idea to set your name and e-mail in Git. Your name and e-mail will be appended to each commit you make.

  11. When you are working in a large team, knowing who made a change is very important; for this, use the following code snippet:
    [email protected] puppet$ git config --global user.email"[email protected]"
    [email protected] puppet$ git config --global user.name "ThomasUphill"
    
  12. You can verify your Git settings using the following snippet:
    [email protected] ~$ git config --global --list
    user.name=Thomas Uphill
    [email protected]
    core.editor=vim
    merge.tool=vimdiff
    color.ui=true
    push.default=simple
    
  13. Now that we have Git configured properly, change directory to your repository directory and create a new site manifest as shown in the following snippet:
    [email protected] ~$ cd puppet
    [email protected] puppet$ mkdir manifests
    [email protected] puppet$ vim manifests/site.pp
    node default {
      include base
    }
    
  14. This site manifest will install our base class on every node; we will create the base class using the Puppet module as we did in Chapter 1, Puppet Language and Style:
    [email protected] puppet$ mkdir modules
    [email protected] puppet$ cd modules
    [email protected] modules$ puppet module generate thomas-base
    Notice: Generating module at /home/tuphill/puppet/modules/thomas-base
    thomas-base
    thomas-base/Modulefile
    thomas-base/README
    thomas-base/manifests
    thomas-base/manifests/init.pp
    thomas-base/spec
    thomas-base/spec/spec_helper.rb
    thomas-base/tests
    thomas-base/tests/init.pp
    [email protected] modules$ ln -s thomas-base base
    
  15. As a last step, we create a symbolic link between the thomas-base directory and base. Now to make sure our module does something useful, add the following to the body of the base class defined in thomas-base/manifests/init.pp:
    class base {
      file {'/etc/motd':
        content => "${::fqdn}\nManaged by puppet ${::puppetversion}\n"
      }
    }
    
  16. Now add the new base module and site manifest to Git using git add and git commit as follows:
    [email protected] modules$ cd ..
    [email protected] puppet$ git add modules manifests
    [email protected] puppet$ git status
    On branch master
    Initial commit
    Changes to be committed:
      (use "git rm --cached <file>..." to unstage)
    new file:   manifests/site.pp
    new file:   modules/base
    new file:   modules/thomas-base/Modulefile
    new file:   modules/thomas-base/README
    new file:   modules/thomas-base/manifests/init.pp
    new file:   modules/thomas-base/spec/spec_helper.rb
    new file:   modules/thomas-base/tests/init.pp
    [email protected] puppet$ git commit -m "Initial commit with simple base module"
    [master (root-commit) 3e1f837] Initial commit with simple base module
     7 files changed, 102 insertions(+)
     create mode 100644 manifests/site.pp
     create mode 120000 modules/base
     create mode 100644 modules/thomas-base/Modulefile
     create mode 100644 modules/thomas-base/README
     create mode 100644 modules/thomas-base/manifests/init.pp
     create mode 100644 modules/thomas-base/spec/spec_helper.rb
     create mode 100644 modules/thomas-base/tests/init.pp
    
  17. At this point your changes to the Git repository have been committed locally; you now need to push those changes back to git.example.com so that other nodes can retrieve the updated files:
    [email protected] puppet$ git push origin master
    Counting objects: 15, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (9/9), done.
    Writing objects: 100% (15/15), 2.15 KiB | 0 bytes/s, done.
    Total 15 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
     * [new branch]      master -> master
    
How it works...

Git tracks changes to files, and stores a complete history of all changes. The history of the repo is made up of commits. A commit represents the state of the repo at a particular point in time, which you create with the git commit command and annotate with a message.

You've now added your Puppet manifest files to the repo and created your first commit. This updates the history of the repo, but only in your local working copy. To synchronize the changes with the git.example.com copy, the git push command pushes all changes made since the last sync. There's more...

Now that you have a central Git repository for your Puppet manifests, you can check out multiple copies of it in different places and work on them before committing your changes. For example, if you're working in a team, each member can have their own local copy of the

Puppet is a configuration management tool. You can use Puppet to configure and prevent configuration drift in a large number of client computers. If all your client computers are easily reached via a central location, you may choose to have a central Puppet server control all the client computers. In the centralized model, the Puppet server is known as the Puppet master. We will cover how to configure a central Puppet master in a few sections.

If your client computers are widely distributed or you cannot guarantee communication between the client computers and a central location, then a decentralized architecture may be a good fit for your deployment. In the next few sections, we will see how to configure a decentralized Puppet architecture.

As we have seen, we can run the puppet apply command directly on a manifest file to have Puppet apply it. The problem with this arrangement is that we need to have the manifests transferred to the client computers.

We can use the Git repository we created in the previous section to transfer our manifests to each new node we create.

Create a bootstrap.pp manifest that will perform the following configuration steps on our new node:

  1. Install Git:
    package {'git':
      ensure => 'installed'
    }
  2. Install the ssh key to access git.example.com in the Puppet user's home directory (/var/lib/puppet/.ssh/id_rsa):
    File {
      owner => 'puppet',
      group => 'puppet',
    }
    file {'/var/lib/puppet/.ssh':
      ensure => 'directory',
    }
    file {'/var/lib/puppet/.ssh/id_rsa':
      content => "
    -----BEGIN RSA PRIVATE KEY-----
    …
    NIjTXmZUlOKefh4MBilqUU3KQG8GBHjzYl2TkFVGLNYGNA0U8VG8SUJq
    -----END RSA PRIVATE KEY-----
    ",
      mode    => 0600,
      require => File['/var/lib/puppet/.ssh']
    }
  3. Download the ssh host key from git.example.com (/var/lib/puppet/.ssh/known_hosts):
    exec {'download git.example.com host key': 
      command => 'sudo -u puppet ssh-keyscan git.example.com >> /var/lib/puppet/.ssh/known_hosts',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      unless  => 'grep git.example.com /var/lib/puppet/.ssh/known_hosts',
      require => File['/var/lib/puppet/.ssh'],
    }
  4. Create a directory to contain the Git repository (/etc/puppet/cookbook):
    file {'/etc/puppet/cookbook':
      ensure => 'directory',
    }
  5. Clone the Puppet repository onto the new machine:
    exec {'create cookbook':
      command => 'sudo -u puppet git clone [email protected]:repos/puppet.git /etc/puppet/cookbook',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      require => [Package['git'],File['/var/lib/puppet/.ssh/id_rsa'],Exec['download git.example.com host key']],
      unless  => 'test -f /etc/puppet/cookbook/.git/config',
    }
  6. Now when we run Puppet apply on the new machine, the ssh key will be installed for the Puppet user. The Puppet user will then clone the Git repository into /etc/puppet/cookbook:
    [email protected] /tmp# puppet apply bootstrap.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.40 seconds
    Notice: /Stage[main]/Main/File[/etc/puppet/cookbook]/ensure: created
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh]/ensure: created
    Notice: /Stage[main]/Main/Exec[download git.example.com host key]/returns: executed successfully
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh/id_rsa]/ensure: defined content as '{md5}da61ce6ccc79bc6937bd98c798bc9fd3'
    Notice: /Stage[main]/Main/Exec[create cookbook]/returns: executed successfully
    Notice: Finished catalog run in 0.82 seconds
    
  7. Now that your Puppet code is available on the new node, you can apply it using puppet apply, specifying that /etc/puppet/cookbook/modules will contain the modules:
    [email protected] ~# puppet apply --modulepath=/etc/puppet/cookbook/modules /etc/puppet/cookbook/manifests/site.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.12 seconds
    Notice: /Stage[main]/Base/File[/etc/motd]/content: content changed '{md5}86d28ff83a8d49d349ba56b5c64b79ee' to '{md5}4c4c3ab7591d940318279d78b9c51d4f'
    Notice: Finished catalog run in 0.11 seconds
    [email protected] /tmp# cat /etc/motd
    testnode.example.com
    Managed by puppet 3.6.2
    
Getting ready

Create a new test node, call this new node whatever you wish, I'll use testnode for mine. Install Puppet on the machine as we have previously done.

Create a bootstrap.pp manifest that will perform the following configuration steps on our new node:

  1. Install Git:
    package {'git':
      ensure => 'installed'
    }
  2. Install the ssh key to access git.example.com in the Puppet user's home directory (/var/lib/puppet/.ssh/id_rsa):
    File {
      owner => 'puppet',
      group => 'puppet',
    }
    file {'/var/lib/puppet/.ssh':
      ensure => 'directory',
    }
    file {'/var/lib/puppet/.ssh/id_rsa':
      content => "
    -----BEGIN RSA PRIVATE KEY-----
    …
    NIjTXmZUlOKefh4MBilqUU3KQG8GBHjzYl2TkFVGLNYGNA0U8VG8SUJq
    -----END RSA PRIVATE KEY-----
    ",
      mode    => 0600,
      require => File['/var/lib/puppet/.ssh']
    }
  3. Download the ssh host key from git.example.com (/var/lib/puppet/.ssh/known_hosts):
    exec {'download git.example.com host key': 
      command => 'sudo -u puppet ssh-keyscan git.example.com >> /var/lib/puppet/.ssh/known_hosts',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      unless  => 'grep git.example.com /var/lib/puppet/.ssh/known_hosts',
      require => File['/var/lib/puppet/.ssh'],
    }
  4. Create a directory to contain the Git repository (/etc/puppet/cookbook):
    file {'/etc/puppet/cookbook':
      ensure => 'directory',
    }
  5. Clone the Puppet repository onto the new machine:
    exec {'create cookbook':
      command => 'sudo -u puppet git clone [email protected]:repos/puppet.git /etc/puppet/cookbook',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      require => [Package['git'],File['/var/lib/puppet/.ssh/id_rsa'],Exec['download git.example.com host key']],
      unless  => 'test -f /etc/puppet/cookbook/.git/config',
    }
  6. Now when we run Puppet apply on the new machine, the ssh key will be installed for the Puppet user. The Puppet user will then clone the Git repository into /etc/puppet/cookbook:
    [email protected] /tmp# puppet apply bootstrap.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.40 seconds
    Notice: /Stage[main]/Main/File[/etc/puppet/cookbook]/ensure: created
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh]/ensure: created
    Notice: /Stage[main]/Main/Exec[download git.example.com host key]/returns: executed successfully
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh/id_rsa]/ensure: defined content as '{md5}da61ce6ccc79bc6937bd98c798bc9fd3'
    Notice: /Stage[main]/Main/Exec[create cookbook]/returns: executed successfully
    Notice: Finished catalog run in 0.82 seconds
    
  7. Now that your Puppet code is available on the new node, you can apply it using puppet apply, specifying that /etc/puppet/cookbook/modules will contain the modules:
    [email protected] ~# puppet apply --modulepath=/etc/puppet/cookbook/modules /etc/puppet/cookbook/manifests/site.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.12 seconds
    Notice: /Stage[main]/Base/File[/etc/motd]/content: content changed '{md5}86d28ff83a8d49d349ba56b5c64b79ee' to '{md5}4c4c3ab7591d940318279d78b9c51d4f'
    Notice: Finished catalog run in 0.11 seconds
    [email protected] /tmp# cat /etc/motd
    testnode.example.com
    Managed by puppet 3.6.2
    
How to do it...

Create a bootstrap.pp manifest that will perform the following configuration steps on our new node:

Install Git:
package {'git':
  ensure => 'installed'
}
Install
  1. the ssh key to access git.example.com in the Puppet user's home directory (/var/lib/puppet/.ssh/id_rsa):
    File {
      owner => 'puppet',
      group => 'puppet',
    }
    file {'/var/lib/puppet/.ssh':
      ensure => 'directory',
    }
    file {'/var/lib/puppet/.ssh/id_rsa':
      content => "
    -----BEGIN RSA PRIVATE KEY-----
    …
    NIjTXmZUlOKefh4MBilqUU3KQG8GBHjzYl2TkFVGLNYGNA0U8VG8SUJq
    -----END RSA PRIVATE KEY-----
    ",
      mode    => 0600,
      require => File['/var/lib/puppet/.ssh']
    }
  2. Download the ssh host key from git.example.com (/var/lib/puppet/.ssh/known_hosts):
    exec {'download git.example.com host key': 
      command => 'sudo -u puppet ssh-keyscan git.example.com >> /var/lib/puppet/.ssh/known_hosts',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      unless  => 'grep git.example.com /var/lib/puppet/.ssh/known_hosts',
      require => File['/var/lib/puppet/.ssh'],
    }
  3. Create a directory to contain the Git repository (/etc/puppet/cookbook):
    file {'/etc/puppet/cookbook':
      ensure => 'directory',
    }
  4. Clone the Puppet repository onto the new machine:
    exec {'create cookbook':
      command => 'sudo -u puppet git clone [email protected]:repos/puppet.git /etc/puppet/cookbook',
      path    => '/usr/bin:/usr/sbin:/bin:/sbin',
      require => [Package['git'],File['/var/lib/puppet/.ssh/id_rsa'],Exec['download git.example.com host key']],
      unless  => 'test -f /etc/puppet/cookbook/.git/config',
    }
  5. Now when we run Puppet apply on the new machine, the ssh key will be installed for the Puppet user. The Puppet user will then clone the Git repository into /etc/puppet/cookbook:
    [email protected] /tmp# puppet apply bootstrap.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.40 seconds
    Notice: /Stage[main]/Main/File[/etc/puppet/cookbook]/ensure: created
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh]/ensure: created
    Notice: /Stage[main]/Main/Exec[download git.example.com host key]/returns: executed successfully
    Notice: /Stage[main]/Main/File[/var/lib/puppet/.ssh/id_rsa]/ensure: defined content as '{md5}da61ce6ccc79bc6937bd98c798bc9fd3'
    Notice: /Stage[main]/Main/Exec[create cookbook]/returns: executed successfully
    Notice: Finished catalog run in 0.82 seconds
    
  6. Now that your Puppet code is available on the new node, you can apply it using puppet apply, specifying that /etc/puppet/cookbook/modules will contain the modules:
    [email protected] ~# puppet apply --modulepath=/etc/puppet/cookbook/modules /etc/puppet/cookbook/manifests/site.pp 
    Notice: Compiled catalog for testnode.example.com in environment production in 0.12 seconds
    Notice: /Stage[main]/Base/File[/etc/motd]/content: content changed '{md5}86d28ff83a8d49d349ba56b5c64b79ee' to '{md5}4c4c3ab7591d940318279d78b9c51d4f'
    Notice: Finished catalog run in 0.11 seconds
    [email protected] /tmp# cat /etc/motd
    testnode.example.com
    Managed by puppet 3.6.2
    
How it works...

First, our bootstrap.pp manifest ensures that Git is installed. The manifest then goes on to ensure

We'd like to make it as quick and easy as possible to apply Puppet on a machine; for this we'll write a little script that wraps the puppet apply command with the parameters it needs. We'll deploy the script where it's needed with Puppet itself.

Follow these steps:

  1. In your Puppet repo, create the directories needed for a Puppet module:
    [email protected] ~$ cd puppet/modules
    [email protected] modules$ mkdir -p puppet/{manifests,files}
    
  2. Create the modules/puppet/files/papply.sh file with the following contents:
    #!/bin/sh sudo puppet apply /etc/puppet/cookbook/manifests/site.pp \--modulepath=/etc/puppet/cookbook/modules $*
  3. Create the modules/puppet/manifests/init.pp file with the following contents:
    class puppet {
      file { '/usr/local/bin/papply':
        source => 'puppet:///modules/puppet/papply.sh',
        mode   => '0755',
      }
    }
  4. Modify your manifests/site.pp file as follows:
    node default {
      include base
      include puppet
    }
  5. Add the Puppet module to the Git repository and commit the change as follows:
    [email protected] puppet$ git add manifests/site.pp modules/puppet
    [email protected] puppet$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes to be committed:
      (use "git reset HEAD <file>..." to unstage)
    modified:   manifests/site.pp
    new file:   modules/puppet/files/papply.sh
    new file:   modules/puppet/manifests/init.pp
    [email protected] puppet$ git commit -m "adding puppet module to include papply"
    [master 7c2e3d5] adding puppet module to include papply
     3 files changed, 11 insertions(+)
     create mode 100644 modules/puppet/files/papply.sh
     create mode 100644 modules/puppet/manifests/init.pp
    
  6. Now remember to push the changes to the Git repository on git.example.com:
    [email protected] puppet$ git push origin master Counting objects: 14, done. Delta compression using up to 4 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (10/10), 894 bytes | 0 bytes/s, done. Total 10 (delta 0), reused 0 (delta 0) To [email protected]:repos/puppet.git 23e887c..7c2e3d5  master -> master
    
  7. Pull the latest version of the Git repository to your new node (testnode for me) as shown in the following command line:
    [email protected] ~# sudo -iu puppet
    [email protected] ~$ cd /etc/puppet/cookbook/[email protected] /etc/puppet/cookbook$ git pull origin master remote: Counting objects: 14, done. remote: Compressing objects: 100% (7/7), done. remote: Total 10 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (10/10), done. From git.example.com:repos/puppet * branch            master     -> FETCH_HEAD Updating 23e887c..7c2e3d5 Fast-forward manifests/site.pp                |    1 + modules/puppet/files/papply.sh   |    4 ++++ modules/puppet/manifests/init.pp |    6 ++++++ 3 files changed, 11 insertions(+), 0 deletions(-) create mode 100644 modules/puppet/files/papply.sh create mode 100644 modules/puppet/manifests/init.pp
    
  8. Apply the manifest manually once to install the papply script:
    [email protected] ~# puppet apply /etc/puppet/cookbook/manifests/site.pp --modulepath /etc/puppet/cookbook/modules
    Notice: Compiled catalog for testnode.example.com in environment production in 0.13 seconds
    Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply]/ensure: defined content as '{md5}d5c2cdd359306dd6e6441e6fb96e5ef7'
    Notice: Finished catalog run in 0.13 seconds
    
  9. Finally, test the script:
    [email protected] ~# papply
    Notice: Compiled catalog for testnode.example.com in environment production in 0.13 seconds
    Notice: Finished catalog run in 0.09 seconds
    

Now, whenever you need to run Puppet, you can simply run papply. In future, when we apply Puppet changes, I'll ask you to run papply instead of the full puppet apply command.

How to do it...

Follow these steps:

In your Puppet repo, create the directories needed for a Puppet module:
[email protected] ~$ cd puppet/modules
[email protected] modules$ mkdir -p puppet/{manifests,files}
Create the modules/puppet/files/papply.sh file with the following contents:
#!/bin/sh sudo puppet apply /etc/puppet/cookbook/manifests/site.pp \--modulepath=/etc/puppet/cookbook/modules $*
Create the modules/puppet/manifests/init.pp file with the following contents:
class puppet {
  file { '/usr/local/bin/papply':
    source => 'puppet:///modules/puppet/papply.sh',
    mode   => '0755',
  }
}
Modify
  1. your manifests/site.pp file as follows:
    node default {
      include base
      include puppet
    }
  2. Add the Puppet module to the Git repository and commit the change as follows:
    [email protected] puppet$ git add manifests/site.pp modules/puppet
    [email protected] puppet$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes to be committed:
      (use "git reset HEAD <file>..." to unstage)
    modified:   manifests/site.pp
    new file:   modules/puppet/files/papply.sh
    new file:   modules/puppet/manifests/init.pp
    [email protected] puppet$ git commit -m "adding puppet module to include papply"
    [master 7c2e3d5] adding puppet module to include papply
     3 files changed, 11 insertions(+)
     create mode 100644 modules/puppet/files/papply.sh
     create mode 100644 modules/puppet/manifests/init.pp
    
  3. Now remember to push the changes to the Git repository on git.example.com:
    [email protected] puppet$ git push origin master Counting objects: 14, done. Delta compression using up to 4 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (10/10), 894 bytes | 0 bytes/s, done. Total 10 (delta 0), reused 0 (delta 0) To [email protected]:repos/puppet.git 23e887c..7c2e3d5  master -> master
    
  4. Pull the latest version of the Git repository to your new node (testnode for me) as shown in the following command line:
    [email protected] ~# sudo -iu puppet
    [email protected] ~$ cd /etc/puppet/cookbook/[email protected] /etc/puppet/cookbook$ git pull origin master remote: Counting objects: 14, done. remote: Compressing objects: 100% (7/7), done. remote: Total 10 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (10/10), done. From git.example.com:repos/puppet * branch            master     -> FETCH_HEAD Updating 23e887c..7c2e3d5 Fast-forward manifests/site.pp                |    1 + modules/puppet/files/papply.sh   |    4 ++++ modules/puppet/manifests/init.pp |    6 ++++++ 3 files changed, 11 insertions(+), 0 deletions(-) create mode 100644 modules/puppet/files/papply.sh create mode 100644 modules/puppet/manifests/init.pp
    
  5. Apply the manifest manually once to install the papply script:
    [email protected] ~# puppet apply /etc/puppet/cookbook/manifests/site.pp --modulepath /etc/puppet/cookbook/modules
    Notice: Compiled catalog for testnode.example.com in environment production in 0.13 seconds
    Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply]/ensure: defined content as '{md5}d5c2cdd359306dd6e6441e6fb96e5ef7'
    Notice: Finished catalog run in 0.13 seconds
    
  6. Finally, test the script:
    [email protected] ~# papply
    Notice: Compiled catalog for testnode.example.com in environment production in 0.13 seconds
    Notice: Finished catalog run in 0.09 seconds
    

Now, whenever you need to run Puppet, you can simply run papply. In future, when we apply Puppet changes, I'll ask you to run papply instead of the full puppet apply command.

How it works...

As you've seen, to run Puppet on a machine and apply a specified manifest file, we use the puppet apply command:

puppet apply manifests/site.pp

When you're using modules (such as the Puppet module we just created), you also need to tell Puppet where to search for modules, using the modulepath argument:

puppet apply manifests/nodes.pp \--modulepath=/home/ubuntu/puppet/modules

In order to run Puppet with the root privileges it needs, we have to put sudo before everything:

sudo puppet apply manifests/nodes.pp \--modulepath=/home/ubuntu/puppet/modules

Finally, any

You can do a lot with the setup you already have: work on your Puppet manifests as a team, communicate changes via a central Git repository, and manually apply them on a machine using the papply script.

However, you still have to log into each machine to update the Git repo and rerun Puppet. It would be helpful to have each machine update itself and apply any changes automatically. Then all you need to do is to push a change to the repo, and it will go out to all your machines within a certain time.

The simplest way to do this is with a cron job that pulls updates from the repo at regular intervals and then runs Puppet if anything has changed.

Follow these steps:

  1. Copy the bootstrap.pp script to any node you wish to enroll. The bootstrap.pp manifest includes the private key used to access the Git repository, it should be protected in a production environment.
  2. Create the modules/puppet/files/pull-updates.sh file with the following contents:
    #!/bin/sh
    cd /etc/puppet/cookbook
    sudo –u puppet git pull && /usr/local/bin/papply
  3. Modify the modules/puppet/manifests/init.pp file and add the following snippet after the papply file definition:
    file { '/usr/local/bin/pull-updates':
      source => 'puppet:///modules/puppet/pull-updates.sh',
      mode   => '0755',
    }
    cron { 'run-puppet':
      ensure  => 'present',
      user    => 'puppet',
      command => '/usr/local/bin/pull-updates',
      minute  => '*/10',
      hour    => '*',
    }
  4. Commit the changes as before and push to the Git server as shown in the following command line:
    [email protected] puppet$ git add modules/puppet
    [email protected] puppet$ git commit -m "adding pull-updates"
    [master 7e9bac3] adding pull-updates
     2 files changed, 14 insertions(+)
     create mode 100644 modules/puppet/files/pull-updates.sh
    [email protected] puppet$ git push
    Counting objects: 14, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (7/7), done.
    Writing objects: 100% (8/8), 839 bytes | 0 bytes/s, done.
    Total 8 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
       7c2e3d5..7e9bac3  master -> master
    
  5. Issue a Git pull on the test node:
    [email protected] ~# cd /etc/puppet/cookbook/
    [email protected] /etc/puppet/cookbook# sudo –u puppet git pull
    remote: Counting objects: 14, done.
    remote: Compressing objects: 100% (7/7), done.
    remote: Total 8 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (8/8), done.
    From git.example.com:repos/puppet
       23e887c..7e9bac3  master     -> origin/master
    Updating 7c2e3d5..7e9bac3
    Fast-forward
     modules/puppet/files/pull-updates.sh |    3 +++
     modules/puppet/manifests/init.pp     |   11 +++++++++++
     2 files changed, 14 insertions(+), 0 deletions(-)
     create mode 100644 modules/puppet/files/pull-updates.sh
    
  6. Run Puppet on the test node:
    [email protected] /etc/puppet/cookbook# papply
    Notice: Compiled catalog for testnode.example.com in environment production in 0.17 seconds
    Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created
    Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-updates]/ensure: defined content as '{md5}04c023feb5d566a417b519ea51586398'
    Notice: Finished catalog run in 0.16 seconds
    
  7. Check that the pull-updates script works properly:
    [email protected] /etc/puppet/cookbook# pull-updates
    Already up-to-date.
    Notice: Compiled catalog for testnode.example.com in environment production in 0.15 seconds
    Notice: Finished catalog run in 0.14 seconds
    
  8. Verify the cron job was created successfully:
    [email protected] /etc/puppet/cookbook# crontab -l -u puppet
    # HEADER: This file was autogenerated at Tue Sep 09 02:31:00 -0400 2014 by puppet.
    # HEADER: While it can still be managed manually, it is definitely not recommended.
    # HEADER: Note particularly that the comments starting with 'Puppet Name' should
    # HEADER: not be deleted, as doing so could cause duplicate cron jobs.
    # Puppet Name: run-puppet
    */10 * * * * /usr/local/bin/pull-updates
    
Getting ready

You'll need the Git repo we set up in the Managing your manifests with Git and Creating a decentralized Puppet architecture recipes, and the papply script from the Writing a papply script recipe. You'll need to apply the bootstrap.pp manifest we created to install ssh keys to download the latest repository.

Follow these steps:

  1. Copy the bootstrap.pp script to any node you wish to enroll. The bootstrap.pp manifest includes the private key used to access the Git repository, it should be protected in a production environment.
  2. Create the modules/puppet/files/pull-updates.sh file with the following contents:
    #!/bin/sh
    cd /etc/puppet/cookbook
    sudo –u puppet git pull && /usr/local/bin/papply
  3. Modify the modules/puppet/manifests/init.pp file and add the following snippet after the papply file definition:
    file { '/usr/local/bin/pull-updates':
      source => 'puppet:///modules/puppet/pull-updates.sh',
      mode   => '0755',
    }
    cron { 'run-puppet':
      ensure  => 'present',
      user    => 'puppet',
      command => '/usr/local/bin/pull-updates',
      minute  => '*/10',
      hour    => '*',
    }
  4. Commit the changes as before and push to the Git server as shown in the following command line:
    [email protected] puppet$ git add modules/puppet
    [email protected] puppet$ git commit -m "adding pull-updates"
    [master 7e9bac3] adding pull-updates
     2 files changed, 14 insertions(+)
     create mode 100644 modules/puppet/files/pull-updates.sh
    [email protected] puppet$ git push
    Counting objects: 14, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (7/7), done.
    Writing objects: 100% (8/8), 839 bytes | 0 bytes/s, done.
    Total 8 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
       7c2e3d5..7e9bac3  master -> master
    
  5. Issue a Git pull on the test node:
    [email protected] ~# cd /etc/puppet/cookbook/
    [email protected] /etc/puppet/cookbook# sudo –u puppet git pull
    remote: Counting objects: 14, done.
    remote: Compressing objects: 100% (7/7), done.
    remote: Total 8 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (8/8), done.
    From git.example.com:repos/puppet
       23e887c..7e9bac3  master     -> origin/master
    Updating 7c2e3d5..7e9bac3
    Fast-forward
     modules/puppet/files/pull-updates.sh |    3 +++
     modules/puppet/manifests/init.pp     |   11 +++++++++++
     2 files changed, 14 insertions(+), 0 deletions(-)
     create mode 100644 modules/puppet/files/pull-updates.sh
    
  6. Run Puppet on the test node:
    [email protected] /etc/puppet/cookbook# papply
    Notice: Compiled catalog for testnode.example.com in environment production in 0.17 seconds
    Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created
    Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-updates]/ensure: defined content as '{md5}04c023feb5d566a417b519ea51586398'
    Notice: Finished catalog run in 0.16 seconds
    
  7. Check that the pull-updates script works properly:
    [email protected] /etc/puppet/cookbook# pull-updates
    Already up-to-date.
    Notice: Compiled catalog for testnode.example.com in environment production in 0.15 seconds
    Notice: Finished catalog run in 0.14 seconds
    
  8. Verify the cron job was created successfully:
    [email protected] /etc/puppet/cookbook# crontab -l -u puppet
    # HEADER: This file was autogenerated at Tue Sep 09 02:31:00 -0400 2014 by puppet.
    # HEADER: While it can still be managed manually, it is definitely not recommended.
    # HEADER: Note particularly that the comments starting with 'Puppet Name' should
    # HEADER: not be deleted, as doing so could cause duplicate cron jobs.
    # Puppet Name: run-puppet
    */10 * * * * /usr/local/bin/pull-updates
    
How to do it...

Follow

these steps:

  1. Copy the bootstrap.pp script to any node you wish to enroll. The bootstrap.pp manifest includes the private key used to access the Git repository, it should be protected in a production environment.
  2. Create the modules/puppet/files/pull-updates.sh file with the following contents:
    #!/bin/sh
    cd /etc/puppet/cookbook
    sudo –u puppet git pull && /usr/local/bin/papply
  3. Modify the modules/puppet/manifests/init.pp file and add the following snippet after the papply file definition:
    file { '/usr/local/bin/pull-updates':
      source => 'puppet:///modules/puppet/pull-updates.sh',
      mode   => '0755',
    }
    cron { 'run-puppet':
      ensure  => 'present',
      user    => 'puppet',
      command => '/usr/local/bin/pull-updates',
      minute  => '*/10',
      hour    => '*',
    }
  4. Commit the changes as before and push to the Git server as shown in the following command line:
    [email protected] puppet$ git add modules/puppet
    [email protected] puppet$ git commit -m "adding pull-updates"
    [master 7e9bac3] adding pull-updates
     2 files changed, 14 insertions(+)
     create mode 100644 modules/puppet/files/pull-updates.sh
    [email protected] puppet$ git push
    Counting objects: 14, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (7/7), done.
    Writing objects: 100% (8/8), 839 bytes | 0 bytes/s, done.
    Total 8 (delta 0), reused 0 (delta 0)
    To [email protected]:repos/puppet.git
       7c2e3d5..7e9bac3  master -> master
    
  5. Issue a Git pull on the test node:
    [email protected] ~# cd /etc/puppet/cookbook/
    [email protected] /etc/puppet/cookbook# sudo –u puppet git pull
    remote: Counting objects: 14, done.
    remote: Compressing objects: 100% (7/7), done.
    remote: Total 8 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (8/8), done.
    From git.example.com:repos/puppet
       23e887c..7e9bac3  master     -> origin/master
    Updating 7c2e3d5..7e9bac3
    Fast-forward
     modules/puppet/files/pull-updates.sh |    3 +++
     modules/puppet/manifests/init.pp     |   11 +++++++++++
     2 files changed, 14 insertions(+), 0 deletions(-)
     create mode 100644 modules/puppet/files/pull-updates.sh
    
  6. Run Puppet on the test node:
    [email protected] /etc/puppet/cookbook# papply
    Notice: Compiled catalog for testnode.example.com in environment production in 0.17 seconds
    Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created
    Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-updates]/ensure: defined content as '{md5}04c023feb5d566a417b519ea51586398'
    Notice: Finished catalog run in 0.16 seconds
    
  7. Check that the pull-updates script works properly:
    [email protected] /etc/puppet/cookbook# pull-updates
    Already up-to-date.
    Notice: Compiled catalog for testnode.example.com in environment production in 0.15 seconds
    Notice: Finished catalog run in 0.14 seconds
    
  8. Verify the cron job was created successfully:
    [email protected] /etc/puppet/cookbook# crontab -l -u puppet
    # HEADER: This file was autogenerated at Tue Sep 09 02:31:00 -0400 2014 by puppet.
    # HEADER: While it can still be managed manually, it is definitely not recommended.
    # HEADER: Note particularly that the comments starting with 'Puppet Name' should
    # HEADER: not be deleted, as doing so could cause duplicate cron jobs.
    # Puppet Name: run-puppet
    */10 * * * * /usr/local/bin/pull-updates
    
How it works...

When we created the bootstrap.pp manifest, we made sure that the Puppet user can checkout There's more...

Congratulations, you now have a fully-automated Puppet infrastructure! Once you have applied the bootstrap.pp manifest, run Puppet on the repository; the machine will be set up to pull any new changes and apply them automatically.

So, for example, if you wanted to add a new user account to all your machines, all you have to do is add the account in your working copy of the manifest, and commit and push the changes to the central Git repository. Within 10 minutes, it will automatically be applied to every machine that's running Puppet.

Previous versions of this book used Rakefiles to bootstrap Puppet. The problem with using Rake to configure a node is that you are running the commands from your laptop; you assume you already have ssh access to the machine. Most bootstrap processes work by issuing an easy to remember command from a node once it has been provisioned. In this section, we'll show how to use bash to bootstrap Puppet with a web server and a bootstrap script.

Getting ready

Install httpd on a centrally accessible server and create a password protected area to store the bootstrap script. In my example, I'll use the Git server I set up previously, git.example.com. Start by creating a directory in the root of your web server:

# cd /var/www/html # mkdir bootstrap

Now perform the following steps:

Add the following location definition to your apache configuration:
<Location /bootstrap>
AuthType basic
AuthName "Bootstrap"
AuthBasicProvider file
AuthUserFile /var/www/puppet.passwd
Require valid-user
</Location>
Reload your web server to ensure the location configuration is operating. Verify with curl that you cannot download from the bootstrap directory without authentication:
[[email protected] tmp]# curl http://git.example.com/bootstrap/
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
Create the password file you referenced in the apache configuration (/var/www/puppet.passwd):
[email protected]# cd /var/www
[email protected]# htpasswd –cb puppet.passwd bootstrap cookbook
Adding password for user bootstrap

Verify that the username and password permit access to the bootstrap directory as follows:
[[email protected] tmp]# curl --user bootstrap:cookbook http://git.example.com/bootstrap/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /bootstrap</title>
How to do it...

Now
How it works...

The There's more...

To support another operating system, we only need to create a new bash script. All Linux distributions will support bash scripting, Mac OS X does as well. Since we placed much of our logic into the bootstrap.pp manifest, the bootstrap script is quite minimal and easy to port to new operating systems.

A configuration management tool such as Puppet is best used when you have many machines to manage. If all the machines can reach a central location, using a centralized Puppet infrastructure might be a good solution. Unfortunately, Puppet doesn't scale well with a large number of nodes. If your deployment has less than 800 servers, a single Puppet master should be able to handle the load, assuming your catalogs are not complex (take less than 10 seconds to compile each catalog). If you have a larger number of nodes, I suggest a load balancing configuration described in Mastering Puppet, Thomas Uphill, Packt Publishing.

A Puppet master is a Puppet server that acts as an X509 certificate authority for Puppet and distributes catalogs (compiled manifests) to client nodes. Puppet ships with a built-in web server called WEBrick, which can handle a very small number of nodes. In this section, we will see how to use that built-in server to control a very small (less than 10) number of nodes.

The Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine).

Getting ready

The Puppet master process is started by running puppet master; most Linux distributions have start and stop scripts for the Puppet master in a separate package. To get started, we'll create a new debian server named puppet.example.com.

The Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine).

How to do it...

Install Puppet on the new server and then use Puppet to install the Puppet master package:
# puppet resource package puppetmaster ensure='installed' Notice: /Package[puppetmaster]/ensure: created package { 'puppetmaster': ensure => '3.7.0-1puppetlabs1', }
Now start the Puppet master service and ensure it will start at boot:
# puppet resource service puppetmaster ensure=true enable=true service { 'puppetmaster': ensure => 'running', enable => 'true', }

The Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine).

How it works...

The

Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine).

There's more...

When we ran puppet agent, Puppet looked for a host named puppet.example.com (since our test node is in the example.com domain); if it couldn't find that host, it would then look for a host named Puppet. We can specify the server to contact with the --server option to puppet agent. When we installed the Puppet master package and started the Puppet master service, Puppet created default SSL certificates based on our hostname. In the next section, we'll see how to create an SSL certificate that has multiple DNS names for our Puppet server.

By default, Puppet will create an SSL certificate for your Puppet master that contains the fully qualified domain name of the server only. Depending on how your network is configured, it can be useful for the server to be known by other names. In this recipe, we'll make a new certificate for our Puppet master that has multiple DNS names.

Getting ready

Install the Puppet master package if you haven't already done so. You will then need to start the Puppet master service at least once to create
How to do it...

The steps are as follows:

Stop the running Puppet master process with the following command:
# service puppetmaster stop
[ ok ] Stopping puppet master.
Delete (clean) the current server certificate:
# puppet cert clean puppet
Notice: Revoked certificate with serial 6
Notice: Removing file Puppet::SSL::Certificate puppet at '/var/lib/puppet/ssl/ca/signed/puppet.pem'
Notice: Removing file Puppet::SSL::Certificate puppet at '/var/lib/puppet/ssl/certs/puppet.pem'
Notice: Removing file Puppet::SSL::Key puppet at '/var/lib/puppet/ssl/private_keys/puppet.pem'
Create How it works...

When your puppet agents connect to the Puppet server, they look for a host called Puppet, they then look for a host called Puppet.[your domain]. If your clients are in different domains, then you need your Puppet master to reply to all the names correctly. By removing the existing certificate and generating a new one, you can have your Puppet master reply to multiple DNS names.

The WEBrick server we configured in the previous section is not capable of handling a large number of nodes. To deal with a large number of nodes, a scalable web server is required. Puppet is a ruby process, so we need a way to run a ruby process within a web server. Passenger is the solution to this problem. It allows us to run the Puppet master process within a web server (apache by default). Many distributions ship with a puppetmaster-passenger package that configures this for you. In this section, we'll use the package to configure Puppet to run within passenger.

The steps are as follows:

  1. Ensure the Puppet master site is enabled in your apache configuration. Depending on your distribution this may be at /etc/httpd/conf.d or /etc/apache2/sites-enabled. The configuration file should be created for you and contain the following information:
    PassengerHighPerformance on
    PassengerMaxPoolSize 12
    PassengerPoolIdleTime 1500
    # PassengerMaxRequests 1000
    PassengerStatThrottleRate 120
    RackAutoDetect Off
    RailsAutoDetect Off
    Listen 8140
    
  2. These lines are tuning settings for passenger. The file then instructs apache to listen on port 8140, the Puppet master port. Next a VirtualHost definition is created that loads the Puppet CA certificates and the Puppet master's certificate:
    <VirtualHost *:8140>
            SSLEngine on
            SSLProtocol             ALL -SSLv2 -SSLv3
            SSLCertificateFile      /var/lib/puppet/ssl/certs/puppet.pem
            SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/puppet.pem
            SSLCertificateChainFile /var/lib/puppet/ssl/certs/ca.pem
            SSLCACertificateFile    /var/lib/puppet/ssl/certs/ca.pem
            SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
            SSLVerifyClient optional
            SSLVerifyDepth  1
            SSLOptions +StdEnvVars +ExportCertData
    
  3. Next, a few important headers are set so that the passenger process has access to the SSL information sent by the client node:
    RequestHeader unset X-Forwarded-For
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
    
  4. Finally, the location of the passenger configuration file config.ru is given with the DocumentRoot location as follows:
           DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
            RackBaseURI /
    
  5. The config.ru file should exist at /usr/share/puppet/rack/puppetmasterd/ and should have the following content:
    $0 = "master"
    ARGV << "--rack"
    ARGV << "--confdir" << "/etc/puppet"
    ARGV << "--vardir"  << "/var/lib/puppet"
    require 'puppet/util/command_line'
    run Puppet::Util::CommandLine.new.execute
    
  6. With the passenger apache configuration file in place and the config.ru file correctly configured, start the apache server and verify that apache is listening on the Puppet master port (if you configured the standalone Puppet master previously, you must stop that process now using service puppetmaster stop):
    [email protected]:~ # service apache2 start
    [ ok ] Starting web server: apache2
    [email protected]:~ # lsof -i :8140
    COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
    apache2 9048     root    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9069 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9070 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    
Getting ready

Install the puppetmaster-passenger package:

# puppet resource package puppetmaster-passenger ensure=installed Notice: /Package[puppetmaster-passenger]/ensure: ensure changed 'purged' to 'present' package { 'puppetmaster-passenger': ensure => '3.7.0-1puppetlabs1', }

The steps are as follows:

  1. Ensure the Puppet master site is enabled in your apache configuration. Depending on your distribution this may be at /etc/httpd/conf.d or /etc/apache2/sites-enabled. The configuration file should be created for you and contain the following information:
    PassengerHighPerformance on
    PassengerMaxPoolSize 12
    PassengerPoolIdleTime 1500
    # PassengerMaxRequests 1000
    PassengerStatThrottleRate 120
    RackAutoDetect Off
    RailsAutoDetect Off
    Listen 8140
    
  2. These lines are tuning settings for passenger. The file then instructs apache to listen on port 8140, the Puppet master port. Next a VirtualHost definition is created that loads the Puppet CA certificates and the Puppet master's certificate:
    <VirtualHost *:8140>
            SSLEngine on
            SSLProtocol             ALL -SSLv2 -SSLv3
            SSLCertificateFile      /var/lib/puppet/ssl/certs/puppet.pem
            SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/puppet.pem
            SSLCertificateChainFile /var/lib/puppet/ssl/certs/ca.pem
            SSLCACertificateFile    /var/lib/puppet/ssl/certs/ca.pem
            SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
            SSLVerifyClient optional
            SSLVerifyDepth  1
            SSLOptions +StdEnvVars +ExportCertData
    
  3. Next, a few important headers are set so that the passenger process has access to the SSL information sent by the client node:
    RequestHeader unset X-Forwarded-For
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
    
  4. Finally, the location of the passenger configuration file config.ru is given with the DocumentRoot location as follows:
           DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
            RackBaseURI /
    
  5. The config.ru file should exist at /usr/share/puppet/rack/puppetmasterd/ and should have the following content:
    $0 = "master"
    ARGV << "--rack"
    ARGV << "--confdir" << "/etc/puppet"
    ARGV << "--vardir"  << "/var/lib/puppet"
    require 'puppet/util/command_line'
    run Puppet::Util::CommandLine.new.execute
    
  6. With the passenger apache configuration file in place and the config.ru file correctly configured, start the apache server and verify that apache is listening on the Puppet master port (if you configured the standalone Puppet master previously, you must stop that process now using service puppetmaster stop):
    [email protected]:~ # service apache2 start
    [ ok ] Starting web server: apache2
    [email protected]:~ # lsof -i :8140
    COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
    apache2 9048     root    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9069 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9070 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    
How to do it...

The steps are as follows:

Ensure the Puppet master site is enabled in your apache configuration. Depending on your distribution this may be at /etc/httpd/conf.d or /etc/apache2/sites-enabled. The configuration file should be created for you and contain the following information:
PassengerHighPerformance on
PassengerMaxPoolSize 12
PassengerPoolIdleTime 1500
# PassengerMaxRequests 1000
PassengerStatThrottleRate 120
RackAutoDetect Off
RailsAutoDetect Off
Listen 8140
These lines are tuning settings for passenger. The file then instructs apache to listen on port 8140, the Puppet master port. Next a VirtualHost definition is created
  1. that loads the Puppet CA certificates and the Puppet master's certificate:
    <VirtualHost *:8140>
            SSLEngine on
            SSLProtocol             ALL -SSLv2 -SSLv3
            SSLCertificateFile      /var/lib/puppet/ssl/certs/puppet.pem
            SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/puppet.pem
            SSLCertificateChainFile /var/lib/puppet/ssl/certs/ca.pem
            SSLCACertificateFile    /var/lib/puppet/ssl/certs/ca.pem
            SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
            SSLVerifyClient optional
            SSLVerifyDepth  1
            SSLOptions +StdEnvVars +ExportCertData
    
  2. Next, a few important headers are set so that the passenger process has access to the SSL information sent by the client node:
    RequestHeader unset X-Forwarded-For
    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
    
  3. Finally, the location of the passenger configuration file config.ru is given with the DocumentRoot location as follows:
           DocumentRoot /usr/share/puppet/rack/puppetmasterd/public/
            RackBaseURI /
    
  4. The config.ru file should exist at /usr/share/puppet/rack/puppetmasterd/ and should have the following content:
    $0 = "master"
    ARGV << "--rack"
    ARGV << "--confdir" << "/etc/puppet"
    ARGV << "--vardir"  << "/var/lib/puppet"
    require 'puppet/util/command_line'
    run Puppet::Util::CommandLine.new.execute
    
  5. With the passenger apache configuration file in place and the config.ru file correctly configured, start the apache server and verify that apache is listening on the Puppet master port (if you configured the standalone Puppet master previously, you must stop that process now using service puppetmaster stop):
    [email protected]:~ # service apache2 start
    [ ok ] Starting web server: apache2
    [email protected]:~ # lsof -i :8140
    COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
    apache2 9048     root    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9069 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    apache2 9070 www-data    8u  IPv6  16842      0t0  TCP *:8140 (LISTEN)
    
How it works...

The passenger configuration file uses the existing Puppet master certificates to listen on port 8140 and handles all the SSL communication between the server and the client. Once the certificate information has been dealt with, the connection is handed off to a ruby process started from passenger using the command line arguments from the config.ru file.

In this case, the $0 variable is set to master and the arguments variable is set to --rack --confdir /etc/puppet --vardir /var/lib/puppet; this is equivalent to running the following from the command line:

puppet master --rack --confdir /etc/puppet --vardir /var/lib/puppet There's more...

You can add additional configuration parameters to the config.ru file to further alter how Puppet

Environments in Puppet are directories holding different versions of your Puppet manifests. Environments prior to Version 3.6 of Puppet were not a default configuration for Puppet. In newer versions of Puppet, environments are configured by default.

Whenever a node connects to a Puppet master, it informs the Puppet master of its environment. By default, all nodes report to the production environment. This causes the Puppet master to look in the production environment for manifests. You may specify an alternate environment with the --environment setting when running puppet agent or by setting environment = newenvironment in /etc/puppet/puppet.conf in the [agent] section.

The steps are as follows:

  1. Create a production directory at /etc/puppet/environments that contains both a modules and manifests directory. Then create a site.pp which creates a file in /tmp as follows:
    [email protected]:~# cd /etc/puppet/environments/
    [email protected]:/etc/puppet/environments# mkdir -p production/{manifests,modules}
    [email protected]:/etc/puppet/environments# vim production/manifests/site.pp
    node default {
      file {'/tmp/production':
        content => "Hello World!\nThis is production\n",
      }
    }
    
  2. Run puppet agent on the master to connect to it and verify that the production code was delivered:
    [email protected]:~# puppet agent -vt
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Caching catalog for puppet
    Info: Applying configuration version '1410415538'
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/production]/ensure: defined content as '{md5}f7ad9261670b9da33a67a5126933044c'
    Notice: Finished catalog run in 0.04 seconds
    # cat /tmp/production
    Hello World!
    This is production
    
  3. Configure another environment devel. Create a new manifest in the devel environment:
    [email protected]:/etc/puppet/environments# mkdir -p devel/{manifests,modules}
    [email protected]:/etc/puppet/environments# vim devel/manifests/site.pp
    node default {
      file {'/tmp/devel':
        content => "Good-bye! Development\n",
      }
    }
    
  4. Apply the new environment by running the --environment devel puppet agent using the following command:
    [email protected]:/etc/puppet/environments# puppet agent -vt --environment devel
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Caching catalog for puppet
    Info: Applying configuration version '1410415890'
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/devel]/ensure: defined content as '{md5}b6313bb89bc1b7d97eae5aa94588eb68'
    Notice: Finished catalog run in 0.04 seconds
    [email protected]:/etc/puppet/environments# cat /tmp/devel
    Good-bye! Development
    
Getting ready

Set the environmentpath function of your installation by adding a line to the [main] section of /etc/puppet/puppet.conf as follows:

[main] ... environmentpath=/etc/puppet/environments

The steps are as follows:

  1. Create a production directory at /etc/puppet/environments that contains both a modules and manifests directory. Then create a site.pp which creates a file in /tmp as follows:
    [email protected]:~# cd /etc/puppet/environments/
    [email protected]:/etc/puppet/environments# mkdir -p production/{manifests,modules}
    [email protected]:/etc/puppet/environments# vim production/manifests/site.pp
    node default {
      file {'/tmp/production':
        content => "Hello World!\nThis is production\n",
      }
    }
    
  2. Run puppet agent on the master to connect to it and verify that the production code was delivered:
    [email protected]:~# puppet agent -vt
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Caching catalog for puppet
    Info: Applying configuration version '1410415538'
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/production]/ensure: defined content as '{md5}f7ad9261670b9da33a67a5126933044c'
    Notice: Finished catalog run in 0.04 seconds
    # cat /tmp/production
    Hello World!
    This is production
    
  3. Configure another environment devel. Create a new manifest in the devel environment:
    [email protected]:/etc/puppet/environments# mkdir -p devel/{manifests,modules}
    [email protected]:/etc/puppet/environments# vim devel/manifests/site.pp
    node default {
      file {'/tmp/devel':
        content => "Good-bye! Development\n",
      }
    }
    
  4. Apply the new environment by running the --environment devel puppet agent using the following command:
    [email protected]:/etc/puppet/environments# puppet agent -vt --environment devel
    Info: Retrieving pluginfacts
    Info: Retrieving plugin
    Info: Caching catalog for puppet
    Info: Applying configuration version '1410415890'
    Notice: /Stage[main]/Main/Node[default]/File[/tmp/devel]/ensure: defined content as '{md5}b6313bb89bc1b7d97eae5aa94588eb68'
    Notice: Finished catalog run in 0.04 seconds
    [email protected]:/etc/puppet/environments# cat /tmp/devel
    Good-bye! Development
    
How to do it...

The steps are as follows:

Create a production directory at /etc/puppet/environments that contains both a modules and manifests directory. Then create a site.pp which creates a file in /tmp as follows:
[email protected]:~# cd /etc/puppet/environments/
[email protected]:/etc/puppet/environments# mkdir -p production/{manifests,modules}
[email protected]:/etc/puppet/environments# vim production/manifests/site.pp
node default {
  file {'/tmp/production':
    content => "Hello World!\nThis is production\n",
  }
}
Run puppet agent on the master to connect to it and verify that the production code was delivered:
[email protected]:~# puppet agent -vt
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for puppet
Info: Applying configuration version '1410415538'
Notice: /Stage[main]/Main/Node[default]/File[/tmp/production]/ensure: defined content as '{md5}f7ad9261670b9da33a67a5126933044c'
Notice: Finished catalog run in 0.04 seconds
# cat /tmp/production
Hello World!
This is production
Configure There's more...

Each

PuppetDB is a database for Puppet that is used to store information about nodes connected to a Puppet master. PuppetDB is also a storage area for exported resources. Exported resources are resources that are defined on nodes but applied to other nodes. The simplest way to install PuppetDB is to use the PuppetDB module from Puppet labs. From this point on, we'll assume you are using the puppet.example.com machine and have a passenger-based configuration of Puppet.

Getting ready

Install the PuppetDB module in the production environment you created in the previous recipe. If you didn't create directory environments, don't worry, using puppet module install will install the module to the correct location for your installation with the following command:

[email protected]:~# puppet module install puppetlabs-puppetdb Notice: Preparing to install into /etc/puppet/environments/production/modules ... Notice: Downloading from https://forgeapi.puppetlabs.com ... Notice: Installing -- do not interrupt ... /etc/puppet/environments/production/modules └─┬ puppetlabs-puppetdb (v3.0.1) ├── puppetlabs-firewall (v1.1.3) ├── puppetlabs-inifile (v1.1.3) └─┬ puppetlabs-postgresql (v3.4.2) ├─┬ puppetlabs-apt (v1.6.0) │ └── puppetlabs-stdlib (v4.3.2) └── puppetlabs-concat (v1.1.0)
How to do it...

Now that our Puppet master has the PuppetDB module installed, we need to apply the PuppetDB module to our Puppet master, we can do this in the site manifest. Add the following to How it works...

The PuppetDB module is a great example of how a complex configuration task can be puppetized. Simply by adding the puppetdb class to our Puppet master node, Puppet installed and configured postgresql and puppetdb.

When we called the puppetdb::master::config class, we set the puppet_service_name variable to apache2, this is because we are running Puppet through passenger. Without this line our agent would try to start the puppetmaster process instead of apache2.

The agent then set up the configuration files for PuppetDB and configured Puppet to use PuppetDB. If There's more...

Now that PuppetDB is configured and we've had a successful agent run, PuppetDB will have data we can query:

[email protected]:~# puppet node status puppet puppet Currently active Last catalog: 2014-09-11T06:45:25.267Z Last facts: 2014-09-11T06:45:22.351Z

Hiera is an information repository for Puppet. Using Hiera you can have a hierarchical categorization of data about your nodes that is maintained outside of your manifests. This is very useful for sharing code and dealing with exceptions that will creep into any Puppet deployment.

Getting ready

Hiera should have already been installed as a dependency on your Puppet master. If it has not already, install it using Puppet:

[email protected]:~# puppet resource package hiera ensure=installed package { 'hiera': ensure => '1.3.4-1puppetlabs1', }
How to do it...

Hiera is configured from a yaml file, /etc/puppet/hiera.yaml. Create the file and add the following as a minimal configuration:
---
:hierarchy:
  - common
:backends:
  - yaml
:yaml:
  :datadir: '/etc/puppet/hieradata'
Create the common.yaml file referenced in the hierarchy:
[email protected]:/etc/puppet# mkdir hieradata
[email protected]:/etc/puppet# vim hieradata/common.yaml
---
message: 'Default Message'
Edit the site.pp file and add a notify resource based on the Hiera value:
node default {
  $message = hiera('message','unknown')
  notify {"Message is $message":}
}
Apply the manifest to a test node:
[email protected]:~$ sudo puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
...
Info: Caching catalog for cookbook-test
Info: Applying configuration version '1410504848'
Notice: Message is Default Message
Notice: /Stage[main]/Main/Node[default]/Notify[Message is Default Message]/message: defined 'message' as 'Message is Default Message'
Notice: Finished catalog run in 0.06 seconds
How it works...

Hiera uses
There's more...

Hiera can be used for automatic parameter lookup with parameterized classes. For example, if you have a class named cookbook::example with a parameter named publisher, you can include the following in a Hiera yaml file to automatically set this parameter:

cookbook::example::publisher: 'PacktPub'

Another often used fact is environment you may reference the environment of the client node using %{environment} as shown in the following hierarchy:

:hierarchy: hosts/%{hostname} os/%{operatingsystem} environment/%{environment} common

In our hierarchy defined in hiera.yaml, we created an entry based on the hostname fact; in this section, we'll create yaml files in the hosts subdirectory of Hiera data with information specific to a particular host.

Getting ready

Install and configure Hiera as in the last section and use the hierarchy defined in the previous recipe that includes a hosts/%{hostname} entry. How to do it...

The following are the steps:

Create a file at /etc/puppet/hieradata/hosts that is the hostname of your test node. For example if your host is named cookbook-test, then the file would be named cookbook-test.yaml.
Insert a specific message in this file:
message: 'This is the test node for the cookbook'
Run Puppet on two different test nodes to note the difference:
[email protected]:~$ sudo puppet agent -t
Info: Caching catalog for cookbook-test
Notice: Message is This is the test node for the cookbook
[[email protected] ~]# puppet agent -t
Info: Caching catalog for hiera-test.example.com
Notice: Message is Default Message
How it works...

Hiera

If you're using Hiera to store your configuration data, there's a gem available called hiera-gpg that adds an encryption backend to Hiera to allow you to protect values stored in Hiera.

In this example, we'll create a piece of encrypted data and retrieve it using hiera-gpg as follows:

  1. Create the secret.yaml file at /etc/puppet/secret with the following contents:
    top_secret: 'Val Kilmer'
    
  2. If you don't already have a GnuPG encryption key, follow the steps in the Using GnuPG to encrypt secrets recipe in Chapter 4, Working with Files and Packages.
  3. Encrypt the secret.yaml file to this key using the following command (replace the [email protected] with the e-mail address you specified when creating the key). This will create the secret.gpg file:
    [email protected]:/etc/puppet/secret# gpg -e -o secret.gpg -r [email protected] secret.yaml 
    [email protected]:/etc/puppet/secret# file secret.gpg
    secret.gpg: GPG encrypted data
    
  4. Remove the plaintext secret.yaml file:
    [email protected]:/etc/puppet/secret# rm secret.yaml
    
  5. Modify your default node in the site.pp file as follows:
    node default {
      $message = hiera('top_secret','Deja Vu')
      notify { "Message is $message": }
    }
    
  6. Now run Puppet on a node:
    [[email protected] ~]# puppet agent -t
    Info: Caching catalog for hiera-test.example.com
    Info: Applying configuration version '1410508276'
    Notice: Message is Deja Vu
    Notice: /Stage[main]/Main/Node[default]/Notify[Message is Deja Vu]/message: defined 'message' as 'Message is Deja Vu'
    Notice: Finished catalog run in 0.08 seconds
    
Getting ready

To set up hiera-gpg, follow these steps:

Install the ruby-dev package; it will be required to build the hiera-gpg gem as follows:
[email protected]:~# puppet resource package ruby-dev ensure=installed
Notice: /Package[ruby-dev]/ensure: ensure changed 'purged' to 'present'
package { 'ruby-dev':
  ensure => '1:1.9.3',
}
Install the hiera-gpg gem using the gem provider:
[email protected]:~# puppet resource package hiera-gpg ensure=installed provider=gem
Notice: /Package[hiera-gpg]/ensure: created
package { 'hiera-gpg':
  ensure => ['1.1.0'],
}
Modify your hiera.yaml file as follows:
    :hierarchy:
        - secret
        - common
    :backends:
        - yaml
        - gpg
    :yaml:
        :datadir: '/etc/puppet/hieradata'
    :gpg:
        :datadir: '/etc/puppet/secret'

In this example, we'll create a piece of encrypted data and retrieve it using hiera-gpg as follows:

  1. Create the secret.yaml file at /etc/puppet/secret with the following contents:
    top_secret: 'Val Kilmer'
    
  2. If you don't already have a GnuPG encryption key, follow the steps in the Using GnuPG to encrypt secrets recipe in Chapter 4, Working with Files and Packages.
  3. Encrypt the secret.yaml file to this key using the following command (replace the [email protected] with the e-mail address you specified when creating the key). This will create the secret.gpg file:
    [email protected]:/etc/puppet/secret# gpg -e -o secret.gpg -r [email protected] secret.yaml 
    [email protected]:/etc/puppet/secret# file secret.gpg
    secret.gpg: GPG encrypted data
    
  4. Remove the plaintext secret.yaml file:
    [email protected]:/etc/puppet/secret# rm secret.yaml
    
  5. Modify your default node in the site.pp file as follows:
    node default {
      $message = hiera('top_secret','Deja Vu')
      notify { "Message is $message": }
    }
    
  6. Now run Puppet on a node:
    [[email protected] ~]# puppet agent -t
    Info: Caching catalog for hiera-test.example.com
    Info: Applying configuration version '1410508276'
    Notice: Message is Deja Vu
    Notice: /Stage[main]/Main/Node[default]/Notify[Message is Deja Vu]/message: defined 'message' as 'Message is Deja Vu'
    Notice: Finished catalog run in 0.08 seconds
    
How to do it...

In this

example, we'll create a piece of encrypted data and retrieve it using hiera-gpg as follows:

  1. Create the secret.yaml file at /etc/puppet/secret with the following contents:
    top_secret: 'Val Kilmer'
    
  2. If you don't already have a GnuPG encryption key, follow the steps in the Using GnuPG to encrypt secrets recipe in Chapter 4, Working with Files and Packages.
  3. Encrypt the secret.yaml file to this key using the following command (replace the [email protected] with the e-mail address you specified when creating the key). This will create the secret.gpg file:
    [email protected]:/etc/puppet/secret# gpg -e -o secret.gpg -r [email protected] secret.yaml 
    [email protected]:/etc/puppet/secret# file secret.gpg
    secret.gpg: GPG encrypted data
    
  4. Remove the plaintext secret.yaml file:
    [email protected]:/etc/puppet/secret# rm secret.yaml
    
  5. Modify your default node in the site.pp file as follows:
    node default {
      $message = hiera('top_secret','Deja Vu')
      notify { "Message is $message": }
    }
    
  6. Now run Puppet on a node:
    [[email protected] ~]# puppet agent -t
    Info: Caching catalog for hiera-test.example.com
    Info: Applying configuration version '1410508276'
    Notice: Message is Deja Vu
    Notice: /Stage[main]/Main/Node[default]/Notify[Message is Deja Vu]/message: defined 'message' as 'Message is Deja Vu'
    Notice: Finished catalog run in 0.08 seconds
    
How it works...

When There's more...

You might also like to know about hiera-eyaml, another secret-data backend for Hiera that supports encryption of individual values within a Hiera data file. This could be handy if you need See also

The Using GnuPG to encrypt secrets recipe in
Getting ready

Install the msgpack gem onto your Puppet master and your nodes. Use Puppet to do the work for you with Puppet resource. You may need to install the ruby-dev or ruby-devel package on your nodes/server at this point:

[email protected]:~$ sudo puppet resource package msgpack ensure=installedprovider=gem Notice: /Package[msgpack]/ensure: created package { 'msgpack': ensure => ['0.5.8'], } How to do it...

Set the preferred_serialization_format to msgpack in the [agent] section of your nodes puppet.conf file:

[agent] preferred_serialization_format=msgpack How it works...

The master will be sent this option when the node begins communicating with the master. Any classes that support serialization with msgpack will be transmitted with msgpack.Serialization of the data between nodes and the master will in theory increase the speed at which nodes communicate by optimizing the data that is travelling between

It would be nice if we knew there was a syntax error in the manifest before we even committed it. You can have Puppet check the manifest using the puppet parser validate command:

This is especially useful because a mistake anywhere in the manifest will stop Puppet from running on any node, even on nodes that don't use that particular part of the manifest. So checking in a bad manifest can cause Puppet to stop applying updates to production for some time, until the problem is discovered, and this could potentially have serious consequences. The best way to avoid this is to automate the syntax check, by using a precommit hook in your version control repo.

Follow these steps:

  1. In your Puppet repo, create a new hooks directory:
    [email protected]:~/puppet$ mkdir hooks
    
  2. Create the file hooks/check_syntax.sh with the following contents (based on a script by Puppet Labs):
    #!/bin/sh
    
    syntax_errors=0
    error_msg=$(mktemp /tmp/error_msg.XXXXXX)
    
    if git rev-parse --quiet --verify HEAD > /dev/null
    then
        against=HEAD
    else
        # Initial commit: diff against an empty tree object
        against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
    fi
    
    # Get list of new/modified manifest and template files
      to check (in git index)
    for indexfile in 'git diff-index --diff-filter=AM --
      name-only --cached $against | egrep '\.(pp|erb)''
    do
        # Don't check empty files
        if [ 'git cat-file -s :0:$indexfile' -gt 0 ]
        then
            case $indexfile in
                *.pp )
                    # Check puppet manifest syntax
                    git cat-file blob :0:$indexfile | 
                      puppet parser validate > $error_msg ;;
                *.erb )
                    # Check ERB template syntax
                    git cat-file blob :0:$indexfile | 
                      erb -x -T - | ruby -c 2> $error_msg >
                        /dev/null ;;
            esac
            if [ "$?" -ne 0 ]
            then
                echo -n "$indexfile: "
                cat $error_msg
                syntax_errors='expr $syntax_errors + 1'
            fi
        fi
    done
    
    rm -f $error_msg
    
    if [ "$syntax_errors" -ne 0 ]
    then
        echo "Error: $syntax_errors syntax errors found,
          aborting commit."
        exit 1
    fi
  3. Set execute permission for the hook script with the following command:
    [email protected]:~/puppet$ chmod a+x hooks/check_syntax.sh
    
  4. Now either symlink or copy the script to the precommit hook in your hooks directory. If your Git repo is checked out in ~/puppet, then create the symlink at ~/puppet/hooks/pre-commit as follows:
    [email protected]:~/puppet$ ln -s ~/puppet/hooks/check_syntax.sh.git/hooks/pre-commit
    
How to do it...

Follow these steps:

In your Puppet repo, create a new hooks directory:
[email protected]:~/puppet$ mkdir hooks
Create the file hooks/check_syntax.sh with the following contents (based on a script by Puppet Labs):
#!/bin/sh

syntax_errors=0
error_msg=$(mktemp /tmp/error_msg.XXXXXX)

if git rev-parse --quiet --verify HEAD > /dev/null
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

# Get list of new/modified manifest and template files
  to check (in git index)
for indexfile in 'git diff-index --diff-filter=AM --
  name-only --cached $against | egrep '\.(pp|erb)''
do
    # Don't check empty files
    if [ 'git cat-file -s :0:$indexfile' -gt 0 ]
    then
        case $indexfile in
            *.pp )
                # Check puppet manifest syntax
                git cat-file blob :0:$indexfile | 
                  puppet parser validate > $error_msg ;;
            *.erb )
                # Check ERB template syntax
                git cat-file blob :0:$indexfile | 
                  erb -x -T - | ruby -c 2> $error_msg >
                    /dev/null ;;
        esac
        if [ "$?" -ne 0 ]
        then
            echo -n "$indexfile: "
            cat $error_msg
            syntax_errors='expr $syntax_errors + 1'
        fi
    fi
done

rm -f $error_msg

if [ "$syntax_errors" -ne 0 ]
then
    echo "Error: $syntax_errors syntax errors found,
      aborting commit."
    exit 1
fi
Set How it works...

The check_syntax.sh script will prevent you from committing any files with syntax errors when it is used as the pre-commit hook for Git:

[email protected]:~/puppet$ git commit -m "test commit" Error: Could not parse for environment production: Syntax error at '}' at line 3 Error: Try 'puppet help parser validate' for usage manifests/nodes.pp: Error: 1 syntax errors found, aborting commit.

If you

As we have already seen in the decentralized model, Git can be used to transfer files between machines using a combination of ssh and ssh keys. It can also be useful to have a Git hook do the same on each successful commit to the repository.

There exists a hook called post-commit that can be run after a successful commit to the repository. In this recipe, we'll create a hook that updates the code on our Puppet master with code from our Git repository on the Git server.

Perform the following steps:

  1. Now that the Git user can log in to the Puppet master as the Puppet user, modify the Git user's ssh configuration to use the newly created ssh key by default:
    [[email protected] ~]$ vim .ssh/config
    Host puppet.example.com
      IdentityFile ~/.ssh/puppet_rsa
    
  2. Add the Puppet master as a remote location for the Puppet repository on the Git server with the following command:
    [[email protected] puppet.git]$ git remote add puppetmaster [email protected]puppet.example.com:/etc/puppet/environments/puppet.git
    
  3. On the Puppet master, move the production directory out of the way and check out your Puppet repository:
    [email protected]:~# chown -R puppet:puppet /etc/puppet/environments
    [email protected]:~# sudo -iu puppet
    [email protected]:~$ cd /etc/puppet/environments/
    [email protected]:/etc/puppet/environments$ mv production production.orig
    [email protected]:/etc/puppet/environments$ git clone [email protected]:repos/puppet.git
    Cloning into 'puppet.git'...
    remote: Counting objects: 63, done.
    remote: Compressing objects: 100% (52/52), done.
    remote: Total 63 (delta 10), reused 0 (delta 0)
    Receiving objects: 100% (63/63), 9.51 KiB, done.
    Resolving deltas: 100% (10/10), done.
    
  4. Now we have a local bare repository on the Puppet server that we can push to, to remotely clone this into the production directory:
    [email protected]:/etc/puppet/environments$ git clone puppet.git production
    Cloning into 'production'...
    done.
    
  5. Now perform a Git push from the Git server to the Puppet master:
    [[email protected] ~]$ cd repos/puppet.git/
    [[email protected] puppet.git]$ git push puppetmaster
    Everything up-to-date
    
  6. Create a post-commit file in the hooks directory of the repository on the Git server with the following contents:
    [[email protected] puppet.git]$ vim hooks/post-commit
    #!/bin/sh
    git push puppetmaster
    ssh [email protected] "cd /etc/puppet/environments/production && git pull"
    [[email protected] puppet.git]$ chmod 755 hooks/post-commit
    
  7. Commit a change to the repository from your laptop and verify that the change is propagated to the Puppet master as follows:
    [email protected] puppet$ vim README
    [email protected] puppet$ git add README
    [email protected] puppet$ git commit -m "Adding README"
    [master 8148902] Adding README
     1 file changed, 4 deletions(-)
    [email protected] puppet$ git push
    X11 forwarding request failed on channel 0
    Counting objects: 5, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 371 bytes | 0 bytes/s, done.
    Total 3 (delta 1), reused 0 (delta 0)
    remote: To [email protected]:/etc/puppet/environments/puppet.git
    remote:    377ed44..8148902  master -> master
    remote: From /etc/puppet/environments/puppet
    remote:    377ed44..8148902  master     -> origin/master
    remote: Updating 377ed44..8148902
    remote: Fast-forward
    remote:  README |    4 ----
    remote:  1 file changed, 4 deletions(-)
    To [email protected]:repos/puppet.git
       377ed44..8148902  master -> master
    
Getting ready

Follow these steps to get started:

Create an ssh key that can access your Puppet user on your Puppet master and install this key into the Git user's account on git.example.com:
[[email protected] ~]$ ssh-keygen -f ~/.ssh/puppet_rsa
Generating public/private rsa key pair.
Your identification has been saved in /home/git/.ssh/puppet_rsa.
Your public key has been saved in /home/git/.ssh/puppet_rsa.pub.
Copy the public key into the authorized_keys file of the puppet user on your puppetmaster
[email protected]:~/.ssh$ cat puppet_rsa.pub >>authorized_keys
Modify the Puppet account to allow the Git user to log in as follows:
[email protected]:~# chsh puppet -s /bin/bash

Perform the following steps:

  1. Now that the Git user can log in to the Puppet master as the Puppet user, modify the Git user's ssh configuration to use the newly created ssh key by default:
    [[email protected] ~]$ vim .ssh/config
    Host puppet.example.com
      IdentityFile ~/.ssh/puppet_rsa
    
  2. Add the Puppet master as a remote location for the Puppet repository on the Git server with the following command:
    [[email protected] puppet.git]$ git remote add puppetmaster [email protected]:/etc/puppet/environments/puppet.git
    
  3. On the Puppet master, move the production directory out of the way and check out your Puppet repository:
    [email protected]:~# chown -R puppet:puppet /etc/puppet/environments
    [email protected]:~# sudo -iu puppet
    [email protected]:~$ cd /etc/puppet/environments/
    [email protected]:/etc/puppet/environments$ mv production production.orig
    [email protected]:/etc/puppet/environments$ git clone [email protected]:repos/puppet.git
    Cloning into 'puppet.git'...
    remote: Counting objects: 63, done.
    remote: Compressing objects: 100% (52/52), done.
    remote: Total 63 (delta 10), reused 0 (delta 0)
    Receiving objects: 100% (63/63), 9.51 KiB, done.
    Resolving deltas: 100% (10/10), done.
    
  4. Now we have a local bare repository on the Puppet server that we can push to, to remotely clone this into the production directory:
    [email protected]:/etc/puppet/environments$ git clone puppet.git production
    Cloning into 'production'...
    done.
    
  5. Now perform a Git push from the Git server to the Puppet master:
    [[email protected] ~]$ cd repos/puppet.git/
    [[email protected] puppet.git]$ git push puppetmaster
    Everything up-to-date
    
  6. Create a post-commit file in the hooks directory of the repository on the Git server with the following contents:
    [[email protected] puppet.git]$ vim hooks/post-commit
    #!/bin/sh
    git push puppetmaster
    ssh [email protected] "cd /etc/puppet/environments/production && git pull"
    [[email protected] puppet.git]$ chmod 755 hooks/post-commit
    
  7. Commit a change to the repository from your laptop and verify that the change is propagated to the Puppet master as follows:
    [email protected] puppet$ vim README
    [email protected] puppet$ git add README
    [email protected] puppet$ git commit -m "Adding README"
    [master 8148902] Adding README
     1 file changed, 4 deletions(-)
    [email protected] puppet$ git push
    X11 forwarding request failed on channel 0
    Counting objects: 5, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 371 bytes | 0 bytes/s, done.
    Total 3 (delta 1), reused 0 (delta 0)
    remote: To [email protected]:/etc/puppet/environments/puppet.git
    remote:    377ed44..8148902  master -> master
    remote: From /etc/puppet/environments/puppet
    remote:    377ed44..8148902  master     -> origin/master
    remote: Updating 377ed44..8148902
    remote: Fast-forward
    remote:  README |    4 ----
    remote:  1 file changed, 4 deletions(-)
    To [email protected]:repos/puppet.git
       377ed44..8148902  master -> master
    
How to do it...

Perform the following steps:

Now that the Git user can log in to the Puppet master as the Puppet user, modify the Git user's ssh configuration to use the newly created ssh key by default:
[[email protected] ~]$ vim .ssh/config
Host puppet.example.com
  IdentityFile ~/.ssh/puppet_rsa
Add the Puppet master as a remote location for the Puppet repository on the Git server with the following command:
[[email protected] puppet.git]$ git remote add puppetmaster [email protected]:/etc/puppet/environments/puppet.git
On the Puppet master, move the production directory out of the way and check out your Puppet repository:
[email protected]:~# chown -R puppet:puppet /etc/puppet/environments
[email protected]:~# sudo -iu puppet
[email protected]:~$ cd /etc/puppet/environments/
[email protected]:/etc/puppet/environments$ mv production production.orig
[email protected]:/etc/puppet/environments$ git clone [email protected]:repos/puppet.git
Cloning into 'puppet.git'...
remote: Counting objects: 63, done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 63 (delta 10), reused 0 (delta 0)
Receiving objects: 100% (63/63), 9.51 KiB, done.
Resolving deltas: 100% (10/10), done.
Now
  1. we have a local bare repository on the Puppet server that we can push to, to remotely clone this into the production directory:
    [email protected]:/etc/puppet/environments$ git clone puppet.git production
    Cloning into 'production'...
    done.
    
  2. Now perform a Git push from the Git server to the Puppet master:
    [[email protected] ~]$ cd repos/puppet.git/
    [[email protected] puppet.git]$ git push puppetmaster
    Everything up-to-date
    
  3. Create a post-commit file in the hooks directory of the repository on the Git server with the following contents:
    [[email protected] puppet.git]$ vim hooks/post-commit
    #!/bin/sh
    git push puppetmaster
    ssh [email protected] "cd /etc/puppet/environments/production && git pull"
    [[email protected] puppet.git]$ chmod 755 hooks/post-commit
    
  4. Commit a change to the repository from your laptop and verify that the change is propagated to the Puppet master as follows:
    [email protected] puppet$ vim README
    [email protected] puppet$ git add README
    [email protected] puppet$ git commit -m "Adding README"
    [master 8148902] Adding README
     1 file changed, 4 deletions(-)
    [email protected] puppet$ git push
    X11 forwarding request failed on channel 0
    Counting objects: 5, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 371 bytes | 0 bytes/s, done.
    Total 3 (delta 1), reused 0 (delta 0)
    remote: To [email protected]:/etc/puppet/environments/puppet.git
    remote:    377ed44..8148902  master -> master
    remote: From /etc/puppet/environments/puppet
    remote:    377ed44..8148902  master     -> origin/master
    remote: Updating 377ed44..8148902
    remote: Fast-forward
    remote:  README |    4 ----
    remote:  1 file changed, 4 deletions(-)
    To [email protected]:repos/puppet.git
       377ed44..8148902  master -> master
    
How it works...

We

Branches are a way of keeping several different tracks of development within a single source repository. Puppet environments are a lot like Git branches. You can have the same code with slight variations between branches, just as you can have different modules for different environments. In this section, we'll show how to use Git branches to define environments on the Puppet master.

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Now whenever we create a new branch, a corresponding directory is created in our environment's directory. A one-to-one mapping is established between environments and branches.

Getting ready

In the previous section, we created a production directory that was based on the master branch; we'll remove that directory now:

[email protected]:/etc/puppet/environments$ mv production production.master

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Now whenever we create a new branch, a corresponding directory is created in our environment's directory. A one-to-one mapping is established between environments and branches.

How to do it...

Modify the post-receive hook to accept a branch variable. The hook will use this variable to create a directory on the Puppet master as follows:

#!/bin/sh read oldrev newrev refname branch=${refname#*\/*\/} git push puppetmaster $branch ssh [email protected] "if [ ! -d /etc/puppet/environments/$branch ]; then git clone /etc/puppet/environments/puppet.git /etc/puppet/environments/$branch; fi; cd /etc/puppet/environments/$branch; git checkout $branch; git pull"

Modify your README file again and push to the repository on git.example.com:

[email protected] puppet$ git add README [email protected] puppet$ git commit -m "Adding README" [master 539d9f8] Adding README 1 file changed, 1 insertion(+) [email protected] puppet$ git push Counting objects: 5, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 374 bytes | 0 bytes/s, done. Total 3 (delta 1), reused 0 (delta 0) remote: To [email protected]:/etc/puppet/environments/puppet.git remote: 0d6b49f..539d9f8 master -> master remote: Cloning into '/etc/puppet/environments/master'... remote: done. remote: Already on 'master' remote: Already up-to-date. To [email protected]:repos/puppet.git 0d6b49f..539d9f8 master -> master

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Now whenever we create a new branch, a corresponding directory is created in our environment's directory. A one-to-one mapping is established between environments and branches.

How it works...

The

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Now whenever we create a new branch, a corresponding directory is created in our environment's directory. A one-to-one mapping is established between environments and branches.

There's more...

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Create the production branch as shown in the following command line:
[email protected] puppet$ git branch production
[email protected] puppet$ git checkout production
Switched to branch 'production'
Update the production branch and push to the Git server as follows:
[email protected] puppet$ vim README
[email protected] puppet$ git add README
[email protected] puppet$ git commit -m "Production Branch"
[email protected] puppet$ git push origin production
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 372 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: To [email protected]:/etc/puppet/environments/puppet.git
remote:    11db6e5..832f6a9  production -> production
remote: Cloning into '/etc/puppet/environments/production'...
remote: done.
remote: Switched to a new branch 'production'
remote: Branch production set up to track remote branch production from origin.
remote: Already up-to-date.
To [email protected]:repos/puppet.git
   11db6e5..832f6a9  production -> production

Now
 
How to do it…

Here are the steps to refactor using arrays of resources:

Identify a class in your manifest where you have several instances of the same kind of resource, for example, packages:
  package { 'sudo' : ensure => installed }
  package { 'unzip' : ensure => installed }
  package { 'locate' : ensure => installed }
  package { 'lsof' : ensure => installed }
  package { 'cron' : ensure => installed }
  package { 'rubygems' : ensure => installed }
Group them together and replace them with a single package resource using an array:
  package
  {
    [ 'cron',
    'locate',
    'lsof',
    'rubygems',
    'sudo',
    'unzip' ]:
    ensure => installed,
  }
How it works…

Most of Puppet's See also

The Iterating over multiple items recipe in

A Puppet module is a group of related resources, usually grouped to configure a specific service. Within a module, you may define multiple resources; resource defaults allow you to specify the default attribute values for a resource. In this example, we'll show you how to specify a resource default for the File type.

To show you how to use resource defaults, we'll create an apache module. Within this module we will specify that the default owner and group are the apache user as follows:

  1. Create an apache module and create a resource default for the File type:
      class apache {
        File {
          owner => 'apache',
          group => 'apache',
          mode => 0644,
        }
      }
  2. Create html files within the /var/www/html directory:
      file {'/var/www/html/index.html':
        content => "<html><body><h1><a
          href='cookbook.html'>Cookbook!
          </a></h1></body></html>\n",
      }
      file {'/var/www/html/cookbook.html':
        content =>
          "<html><body><h2>PacktPub</h2></body></html>\n",
      }
    
  3. Add this class to your default node definition, or use puppet apply to apply the module to your node. I will use the method we configured in the previous chapter, pushing our code to the Git repository and using a Git hook to have the code deployed to the Puppet master as follows:
    [email protected] ~/puppet $ git pull origin production
    From git.example.com:repos/puppet
     * branch            production -> FETCH_HEAD
    Already up-to-date.
    [email protected] ~/puppet $ cd modules
    [email protected] ~/puppet/modules $ mkdir -p apache/manifests
    [email protected] ~/puppet/modules $ vim apache/manifests/init.pp
    [email protected] ~/puppet/modules $ cd ..
    [email protected] ~/puppet $ vim manifests/site.pp 
    [email protected] ~/puppet $ git status
    On branch production
    Changes not staged for commit:
    modified:   manifests/site.pp
    Untracked files:
    modules/apache/
    [email protected] ~/puppet $ git add manifests/site.pp modules/apache
    [email protected] ~/puppet $ git commit -m 'adding apache module'
    [production d639a86] adding apache module
     2 files changed, 14 insertions(+)
     create mode 100644 modules/apache/manifests/init.pp
    [email protected] ~/puppet $ git push origin production
    Counting objects: 13, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (8/8), 885 bytes | 0 bytes/s, done.
    Total 8 (delta 0), reused 0 (delta 0)
    remote: To [email protected]:/etc/puppet/environments/puppet.git
    remote:    832f6a9..d639a86  production -> production
    remote: Already on 'production'
    remote: From /etc/puppet/environments/puppet
    remote:    832f6a9..d639a86  production -> origin/production
    remote: Updating 832f6a9..d639a86
    remote: Fast-forward
    remote:  manifests/site.pp                |    1 +
    remote:  modules/apache/manifests/init.pp |   13 +++++++++++++
    remote:  2 files changed, 14 insertions(+)
    remote:  create mode 100644 modules/apache/manifests/init.pp
    To [email protected]:repos/puppet.git
       832f6a9..d639a86  production -> production
    
  4. Apply the module to a node or run Puppet:
    Notice: /Stage[main]/Apache/File[/var/www/html/cookbook.html]/ensure: defined content as '{md5}493473fb5bde778ca93d034900348c5d'
    Notice: /Stage[main]/Apache/File[/var/www/html/index.html]/ensure: defined content as '{md5}184f22c181c5632b86ebf9a0370685b3'
    Notice: Finished catalog run in 2.00 seconds
    [[email protected] ~]# ls -l /var/www/html
    total 8
    -rw-r--r--. 1 apache apache 44 Sep 15 12:00 cookbook.html
    -rw-r--r--. 1 apache apache 73 Sep 15 12:00 index.html
    
How to do it...

To show you how to use

resource defaults, we'll create an apache module. Within this module we will specify that the default owner and group are the apache user as follows:

  1. Create an apache module and create a resource default for the File type:
      class apache {
        File {
          owner => 'apache',
          group => 'apache',
          mode => 0644,
        }
      }
  2. Create html files within the /var/www/html directory:
      file {'/var/www/html/index.html':
        content => "<html><body><h1><a
          href='cookbook.html'>Cookbook!
          </a></h1></body></html>\n",
      }
      file {'/var/www/html/cookbook.html':
        content =>
          "<html><body><h2>PacktPub</h2></body></html>\n",
      }
    
  3. Add this class to your default node definition, or use puppet apply to apply the module to your node. I will use the method we configured in the previous chapter, pushing our code to the Git repository and using a Git hook to have the code deployed to the Puppet master as follows:
    [email protected] ~/puppet $ git pull origin production
    From git.example.com:repos/puppet
     * branch            production -> FETCH_HEAD
    Already up-to-date.
    [email protected] ~/puppet $ cd modules
    [email protected] ~/puppet/modules $ mkdir -p apache/manifests
    [email protected] ~/puppet/modules $ vim apache/manifests/init.pp
    [email protected] ~/puppet/modules $ cd ..
    [email protected] ~/puppet $ vim manifests/site.pp 
    [email protected] ~/puppet $ git status
    On branch production
    Changes not staged for commit:
    modified:   manifests/site.pp
    Untracked files:
    modules/apache/
    [email protected] ~/puppet $ git add manifests/site.pp modules/apache
    [email protected] ~/puppet $ git commit -m 'adding apache module'
    [production d639a86] adding apache module
     2 files changed, 14 insertions(+)
     create mode 100644 modules/apache/manifests/init.pp
    [email protected] ~/puppet $ git push origin production
    Counting objects: 13, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (8/8), 885 bytes | 0 bytes/s, done.
    Total 8 (delta 0), reused 0 (delta 0)
    remote: To [email protected]:/etc/puppet/environments/puppet.git
    remote:    832f6a9..d639a86  production -> production
    remote: Already on 'production'
    remote: From /etc/puppet/environments/puppet
    remote:    832f6a9..d639a86  production -> origin/production
    remote: Updating 832f6a9..d639a86
    remote: Fast-forward
    remote:  manifests/site.pp                |    1 +
    remote:  modules/apache/manifests/init.pp |   13 +++++++++++++
    remote:  2 files changed, 14 insertions(+)
    remote:  create mode 100644 modules/apache/manifests/init.pp
    To [email protected]:repos/puppet.git
       832f6a9..d639a86  production -> production
    
  4. Apply the module to a node or run Puppet:
    Notice: /Stage[main]/Apache/File[/var/www/html/cookbook.html]/ensure: defined content as '{md5}493473fb5bde778ca93d034900348c5d'
    Notice: /Stage[main]/Apache/File[/var/www/html/index.html]/ensure: defined content as '{md5}184f22c181c5632b86ebf9a0370685b3'
    Notice: Finished catalog run in 2.00 seconds
    [[email protected] ~]# ls -l /var/www/html
    total 8
    -rw-r--r--. 1 apache apache 44 Sep 15 12:00 cookbook.html
    -rw-r--r--. 1 apache apache 73 Sep 15 12:00 index.html
    
How it works...

The resource default we defined specifies the owner, group, and mode for all file resources within this class (also known as within this scope). Unless you specifically override a resource default, the value for an attribute will be taken from the default. There's more...

You can specify resource defaults for any

In the previous example, we saw how to reduce redundant code by grouping identical resources into arrays. However, this technique is limited to resources where all the parameters are the same. When you have a set of resources that have some parameters in common, you need to use a defined type to group them together.

How to do it…

The following steps will show you how to create a definition:

Add the following code to your manifest:
  define tmpfile() {
    file { "/tmp/${name}": content => "Hello, world\n",
    }
  }
  tmpfile { ['a', 'b', 'c']: }
Run Puppet:
[[email protected] ~]# vim tmp.pp
[[email protected] ~]# puppet apply tmp.pp 
Notice: Compiled catalog for hiera-test.example.com in environment production in 0.11 seconds
Notice: /Stage[main]/Main/Tmpfile[a]/File[/tmp/a]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
Notice: /Stage[main]/Main/Tmpfile[b]/File[/tmp/b]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
Notice: /Stage[main]/Main/Tmpfile[c]/File[/tmp/c]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
Notice: Finished catalog run in 0.09 seconds
[[email protected] ~]# cat /tmp/{a,b,c}
Hello, world
Hello, world
Hello, world