Computers in the future may have as few as 1,000 vacuum tubes and weigh only 1.5 tons. | ||
--Popular Mechanics, 1949 |
In this chapter, we will cover:
Installing Puppet
Creating a manifest
Managing your manifests with Git
Creating a decentralized Puppet architecture
Writing a papply script
Running Puppet from cron
Deploying changes with Rake
Bootstrapping Puppet with Rake
Automatic syntax checking with Git hooks
Some of the recipes in this book represent best practices as agreed upon by the Puppet community. Others are tips and tricks which will make it easier for you to work with Puppet, or introduce you to features that you may not have been previously aware of. Some recipes are shortcuts that I wouldn't recommend you use as standard operating procedure, but may be useful in emergencies. Finally, there are some experimental recipes that you may like to try, but are only useful or applicable in very large-scale infrastructures or otherwise unusual circumstances.
My hope is that, by reading through and thinking about the recipes presented here, you will gain a deeper and broader understanding of how Puppet works and how you can use it to improve your infrastructure. Only you can decide whether a particular recipe is appropriate for you and your organization, but I hope this collection will inspire you to experiment, find out more, and most of all have fun using Puppet!
Because Linux distributions, such as Ubuntu, Red Hat, and CentOS differ in the specific details of package names, configuration file paths, and many other things, I have decided that for reasons of space and clarity the best approach for this book is to pick one distribution (Ubuntu 12.04 Precise) and stick to that. However, Puppet runs on most popular operating systems, so you should have very little trouble adapting the recipes to your own favored OS and distribution.
At the time of writing, Puppet 3.2 is the latest stable version available, and consequently I have chosen that as the reference version of Puppet used in the book. The syntax of Puppet commands changes often, so be aware that while older versions of Puppet are still perfectly usable, they may not support all of the features and syntax described in this book.
If you already have a working Puppet installation, you can skip this section. If not, or if you want to upgrade or re-install Puppet, we'll go through the installation process step by step.
I'm using an Amazon EC2 cloud instance to demonstrate setting up Puppet, though you may prefer to use a physical server, a Linux workstation, or a virtual machine such as Vagrant, VMWare, or VirtualBox (with Internet access). I'll log in as the ubuntu
user and use sudo
to run commands that need root
privileges (the default setup on Ubuntu).
Note
On EC2 Ubuntu images, the ubuntu
user is already set up with the sudo
permissions to run any commands as root
. If you're using a different Linux distribution or you're not on EC2, you'll need to configure this yourself in the /etc/sudoers
file.
To prepare the machine for Puppet, we need to set its hostname.
Set a suitable hostname for your server (ignore any warning from
sudo
):ubuntu@domU-12-31-39-09-51-23:~$ sudo hostname cookbook ubuntu@domU-12-31-39-09-51-23:~$ sudo su -c 'echo cookbook >/etc/hostname' sudo: unable to resolve host cookbook
Log out and log back in to check the hostname is now correctly set:
ubuntu@cookbook:~$
Find out the local IP address of the server:
ubuntu@cookbook:~$ ip addr show |grep eth0 inet 10.96.247.132/23 brd 10.96.247.255 scope global eth0
Copy the IP address of your server (here it's
10.96.247.132
) and add this to the/etc/hosts
file so that it looks something like this (use your own hostname and domain):10.96.247.132 cookbook cookbook.example.com
Puppet packages for most Linux distributions, including Ubuntu, are available from Puppet Labs. Here's how to install the version for Ubuntu 12.04 Precise:
Download the Puppet Labs
repo
package:ubuntu@cookbook:~$ wget http://apt.puppetlabs.com/puppetlabs- release-precise.deb
Install the
repo
package:ubuntu@cookbook:~$ sudo dpkg -i puppetlabs-release-precise.deb Selecting previously unselected package puppetlabs-release. (Reading database ... 33975 files and directories currently installed.) Unpacking puppetlabs-release (from puppetlabs-release- precise.deb) Setting up puppetlabs-release (1.0-5)
Update your APT configuration:
ubuntu@cookbook:~$ sudo apt-get update
Note
If you're not using Ubuntu 12.04 Precise, you can find out how to add the Puppet Labs repos package to your system here:
http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html
Install Puppet:
ubuntu@cookbook:~$ sudo apt-get -y install puppet
Note
If you're on Mac, you can download and install suitable DMG images from Puppet Labs available at:
https://downloads.puppetlabs.com/mac/
If you're using Windows, you can download MSI packages from the Puppet Labs website available at:
Run the following command to check that Puppet is properly installed:
ubuntu@cookbook:~$ puppet --version 3.2.2
If the version of Puppet you've installed is not exactly the same, it doesn't matter; you'll get whatever is the latest version made available by Puppet Labs. So long as your version is at least 3.0, you should have no trouble running the examples in this book.
If you have an older version of Puppet, you may find that some things don't work or work differently to the way you'd expect. I recommend that you upgrade to Puppet 3.x or later if at all possible.
Now that Puppet is set up, you can use it to make some configuration changes by creating a manifest. We'll see how to do this in the next section.
If you already have some Puppet code (known as a Puppet manifest
), you can skip this section and go on to the next. If not, we'll see how to create and apply a simple manifest.
Follow these steps:
First, let's create a suitable directory structure to keep the manifest code in:
ubuntu@cookbook:~$ mkdir puppet ubuntu@cookbook:~$ cd puppet ubuntu@cookbook:~/puppet$ mkdir manifests
Within your
puppet
directory, create the filemanifests/site.pp
with the following contents:import 'nodes.pp'
Create the file
manifests/nodes.pp
with the following contents (use your machine's hostname in place ofcookbook
):node 'cookbook' { file { '/tmp/hello': content => "Hello, world\n", } }
Test your manifest with the
puppet apply
command. This will tell Puppet to read the manifest, compare it to the state of the machine, and make any necessary changes to that state:ubuntu@cookbook:~/puppet$ sudo puppet apply manifests/site.pp Notice: /Stage[main]//Node[cookbook]/File[/tmp/hello]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850' Notice: Finished catalog run in 0.06 seconds
To see if Puppet did what we expected (create the file
/tmp/hello
with the contents Hello, world), run the following command:ubuntu@cookbook:~/puppet$ cat /tmp/hello Hello, world
It's a great idea to put your Puppet manifests in a version control system such as Git or Subversion (I recommend Git) and give all Puppet-managed machines a checkout from your repository. This gives you several advantages:
You can undo changes and revert to any previous version of your manifest
You can experiment with new features using a
branch
If several people need to make changes to the manifests, they can make them independently, in their own working copies, and then merge their changes later
You can use the
git log
feature to see what was changed, and when (and by whom)
In this section we'll import your existing manifest files into Git. If you have created a puppet
directory in the previous section, use that, otherwise use your existing manifest directory.
I'm going to use the popular GitHub service as my Git server. You don't have to do this, it's easy to run your own Git server but it does simplify things. If you already use Git and have a suitable server, feel free to use that instead.
Note
Note that GitHub currently only offers free repository hosting for public repositories (that is, everyone will be able to see and read your Puppet manifests). This isn't a good idea if your manifest contains secret data such as passwords. It's fine for playing and experimenting with the recipes in this book, but for production use, consider a private GitHub repo instead.
Here's what you need to do to prepare for importing your manifest:
First, you'll need Git installed on your machine:
ubuntu@cookbook:~/puppet$ sudo apt-get install git
Next, you'll need a GitHub account (free for open-source projects, or you'll need to pay a small fee to create private repositories) and a repository. Follow the instructions at github.com to create and initialize your repository (from now on, just "repo" for short). Make sure you tick the box that says, Initialize this repository with a README.
Authorize your SSH key for read/write access to the repo (see the GitHub site for instructions on how to do this).
You're now ready to add your existing manifests to the Git repo. We're going to clone the repo, and then move your manifest files into it, as follows:
First, move your
puppet
directory to a different name:mv puppet puppet.import
Clone the repo onto your machine into a directory named
puppet
(use your own repo URL, as shown on GitHub):ubuntu@cookbook:~$ git clone git@github.com:bitfield/cookbook.git puppet Cloning into 'puppet'... remote: Counting objects: 3, done. remote: Total 3 (delta 0), reused 0 (delta 0) Receiving objects: 100% (3/3), done.
Move everything from
puppet.import
topuppet
:ubuntu@cookbook:~$ mv puppet.import/* puppet/
Add and commit the new files to the repo, setting your Git identity details if necessary:
ubuntu@cookbook:~$ cd puppet ubuntu@cookbook:~/puppet$ git status # On branch master # Untracked files: # (use "git add <file>..." to include in what will be committed) # # manifests/ nothing added to commit but untracked files present (use "git add" to track) ubuntu@cookbook:~/puppet$ git add manifests/ ubuntu@cookbook:~/puppet$ git config --global user.name "John Arundel" ubuntu@cookbook:~/puppet$ git config --global user.email "john@bitfieldconsulting.com" ubuntu@cookbook:~/puppet$ git commit -m "Importing" [master a063a5b] Importing Committer: John Arundel <john@bitfieldconsulting.com> 2 files changed, 6 insertions(+) create mode 100644 manifests/nodes.pp create mode 100644 manifests/site.pp
Finally, push your changes back to GitHub:
ubuntu@cookbook:~/puppet$ git push -u origin master Counting objects: 6, done. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 457 bytes, done. Total 5 (delta 0), reused 0 (delta 0) To git@github.com:bitfield/cookbook.git 6d6aa51..a063a5b master -> master
Git tracks changes to files, and stores a complete history of all changes. The history of the repo is made up of commits. A commit represents the state of the repo at a particular point in time, which you create with the git commit
command and annotate with a message.
You've added your Puppet manifest files to the repo and created your first commit. This updates the history of the repo, but only in your local working copy. To synchronize the changes with GitHub's copy, the git push
command pushes all changes made since the last sync.
Now that you have a central Git repo for your Puppet manifests, you can check out multiple copies of it in different places and work on them, before committing your changes. For example, if you're working in a team, each member can have her own local copy of the repo and synchronize changes with the others via GitHub.
Now that you've taken control of your manifests with Git, you can use it as a simple, scalable way to distribute manifest files to lots of machines. We'll see how to do this in the next section.
Some systems work best when they're decentralized. The Mafia is a good example, although, of course, there is no Mafia.
A common way to use Puppet is to run a Puppet Master server, which Puppet clients can then connect to and receive their manifests. However, you don't need a Puppet Master to use Puppet. You can run the puppet apply
command directly on a manifest file to have Puppet apply it:
ubuntu@cookbook:~/puppet$ puppet apply manifests/site.pp Notice: Finished catalog run in 0.08 seconds
In other words, if you can arrange to distribute a suitable manifest file to a client machine, you can have Puppet execute it directly without the need for a central Puppet Master. This removes the performance bottleneck of a single master server, and also eliminates a single point of failure. It also avoids having to sign and exchange the SSL certificates when provisioning a new client machine.
There are many ways you could deliver the manifest file to the client, but Git (or any version control system) does most of the work for you. You can edit your manifests in a local working copy, commit them to Git, and push them to a central repo, and from there they can be automatically distributed to the client machines.
If your Puppet manifests aren't already in Git, follow the steps in Managing your manifests with Git.
You'll need a second machine to check out a copy of your Puppet repo. If you're using EC2 instances, create another instance, and call it something like cookbook2
.
Follow these steps:
Check out your GitHub repo on the new machine:
ubuntu@cookbook2:~$ git clone git@github.com:bitfield/cookbook.git puppet Cloning into 'puppet'... remote: Counting objects: 8, done. remote: Compressing objects: 100% (5/5), done. remote: Total 8 (delta 0), reused 5 (delta 0) Receiving objects: 100% (8/8), done.
Modify your
manifests/nodes.pp
file, as follows:node 'cookbook', 'cookbook2' { file { '/tmp/hello': content => "Hello, world\n", } }
Run the following command:
ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850' Notice: Finished catalog run in 0.05 seconds
We've created a new working copy of the Puppet repo on the new machine:
ubuntu@cookbook2:~$ git clone git@github.com:bitfield/cookbook.git puppet
However, before we can run Puppet, we have to create a node declaration for the cookbook2
node:
node 'cookbook', 'cookbook2' { ... }
Now, we apply the manifest:
ubuntu@cookbook2:~/puppet$ sudo puppet apply manifests/site.pp
Puppet finds the node declaration for cookbook2
and applies the same manifest that we used before on cookbook
:
Notice: /Stage[main]//Node[cookbook2]/File[/tmp/hello]/ensure: defined content as '{md5}a7966bf58e23583c9a5a4059383ff850'
Having scaled your Puppet infrastructure from one machine to the other, you can now extend it to as many as you like! All you need to do is check out the Git repo on a machine, and run puppet apply
.
This is a great way to add Puppet management to your existing machines without lots of complicated setup, or using an extra machine to serve as a Puppet Master. Many of my clients have switched to using a Git-based infrastructure because it's simpler, easier to scale, and easier to maintain.
A refinement which you might like to consider is having each machine automatically pull changes from GitHub and apply them with Puppet. Then all you need to do is push a change to GitHub, and it will roll out to all your Puppet-managed machines within a certain time. We'll see how to do this in the following sections.
We'd like to make it as quick and easy as possible to apply Puppet on a machine, so I usually write a little script that wraps the puppet apply
command with the parameters it needs. And to deploy the script where it's needed, what better tool than Puppet itself?
Follow these steps:
In your Puppet repo, create the directories needed for a
puppet
module:ubuntu@cookbook:~/puppet$ mkdir modules ubuntu@cookbook:~/puppet$ mkdir modules/puppet ubuntu@cookbook:~/puppet$ mkdir modules/puppet/manifests ubuntu@cookbook:~/puppet$ mkdir modules/puppet/files
Create the file
modules/puppet/files/papply.sh
with the following contents (change the path/home/ubuntu/puppet
to where your Puppet repo is located). Thesudo puppet apply
command should all be on one line:#!/bin/sh sudo puppet apply /home/ubuntu/puppet/manifests/site.pp --modulepath=/home/ubuntu/puppet/modules/ $*
Create the file
modules/puppet/manifests/init.pp
with the following contents:class puppet { file { '/usr/local/bin/papply': source => 'puppet:///modules/puppet/papply.sh', mode => '0755', } }
Modify your
manifests/nodes.pp
file as follows:node 'cookbook' { include puppet }
Apply your changes:
ubuntu@cookbook:~/puppet$ sudo puppet apply manifests/site.pp --modulepath=/home/ubuntu/puppet/modules Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply] /ensure: defined content as '{md5} 171896840d39664c00909eb8cf47a53c' Notice: Finished catalog run in 0.07 seconds
Test that the script works:
ubuntu@cookbook:~/puppet$ papply Notice: Finished catalog run in 0.07 seconds
Now whenever you need to run Puppet, you can simply run papply
. In future, when we apply Puppet changes, I'll ask you to run papply
instead of the full puppet apply
command.
As you've seen, to run Puppet on a machine and apply a specified manifest file, we use the puppet
apply
command:
puppet apply manifests/site.pp
When you're using modules (such as the puppet module we just created) you also need to tell Puppet where to search for modules, using the modulepath
argument:
puppet apply manifests/nodes.pp -- modulepath=/home/ubuntu/puppet/modules
In order to run Puppet with the root privileges it needs, we have to put sudo
before everything:
sudo puppet apply manifests/nodes.pp -- modulepath=/home/ubuntu/puppet/modules
Finally, any additional arguments passed to papply
will be passed through to Puppet itself, by adding the $*
parameter:
sudo puppet apply manifests/nodes.pp -- modulepath=/home/ubuntu/puppet/modules $*
That's a lot of typing, so putting this in a script makes sense. We've added a Puppet file
resource that will deploy the script to /usr/local/bin
and make it executable:
file { '/usr/local/bin/papply': source => 'puppet:///modules/puppet/papply.sh', mode => '0755', }
Finally, we include the puppet
module in our node declaration for cookbook
:
node 'cookbook' { include puppet }
You can do the same for any other nodes managed by Puppet.
You can do a lot with the setup you already have: work on your Puppet manifests as a team, communicate changes via GitHub, and manually apply them on a machine using the papply
script.
However, you still have to log into each machine to update the Git repo and re-run Puppet. It would be helpful to have each machine update itself and apply any changes automatically. Then all you need to do is to push a change to the repo, and it will go out to all your machines within a certain time.
The simplest way to do this is with a cron job that pulls updates from the repo at regular intervals and then runs Puppet if anything has changed.
You'll need the Git repo we set up in Managing your manifests with Git and Creating a decentralized Puppet architecture, and the papply
script from Writing a papply script.
You'll also need to create an SSH key that each machine can use to pull changes from the Git repo. To create this, follow these steps:
Run the following command to generate the keyfile:
ubuntu@cookbook:~/puppet$ ssh-keygen -f ubuntu Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in ubuntu. Your public key has been saved in ubuntu.pub. The key fingerprint is: ae:80:48:1c:14:51:d6:b1:73:4f:60:e2:cf:3d:ce:f1 ubuntu@cookbook The key's randomart image is: +--[ RSA 2048]----+ | ++o.o.o | | + | | + | | = + | | o oS= | | o + | | o E | | | | | +-----------------+
Print the contents of the
ubuntu.pub
file:ubuntu@cookbook:~/puppet$ cat ubuntu.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8EsdLAZHIg1nnMzJuIQ5jEcFL1WI5AVhml6Z3Gw4zc4xw6F1Citomc+3DexcaD+y3VrD3WEOGcXweCsxJF0EGyJoc4RbPAJaP3D4V/+9FQVZcH90GukasvtIrfJYy2KFfRBROKtrfckMbBlWF7U2U+FwaalMOtgLzZeECSDU4eYuheN3UVcyg9Zx87zrLYU5EK1JH2WVoZd3UmdH73/rwPJWtSEQ3xs9A2wMr0lJsCF4CcFCVwrAIoEf5WzIoHbhWyZaVyPR4gHUHd3wNIzC0rmoRiYwE5uYvVBObLN10uZhn7zGPWHEc5tYU7DMbz61iTe4NLtauwJkZxmXUiPJh ubuntu@cookbook
Copy this and add it to your GitHub repo as a deploy key (refer to the GitHub site for instructions on how to do this). This will authorize the key to clone the Puppet repo from GitHub.
Move the public key file into your
puppet
module:ubuntu@cookbook:~/puppet$ mv ubuntu.pub modules/puppet/files/ubuntu.pub
Keep the private key file somewhere separate from your Puppet repo (you'll distribute this via some other channel to machines which need to check out the repo).
Create the file
modules/puppet/files/pull-updates.sh
with the following contents:#!/bin/sh cd /home/ubuntu/puppet git pull && /usr/local/bin/papply
Modify the file
modules/puppet/manifests/init.pp
to look like this:class puppet { file { '/usr/local/bin/papply': source => 'puppet:///modules/puppet/papply.sh', mode => '0755', } file { '/usr/local/bin/pull-updates': source => 'puppet:///modules/puppet/pull-updates.sh', mode => '0755', } file { '/home/ubuntu/.ssh/id_rsa': source => 'puppet:///modules/puppet/ubuntu.priv', owner => 'ubuntu', mode => '0600', } cron { 'run-puppet': ensure => 'present', user => 'ubuntu', command => '/usr/local/bin/pull-updates', minute => '*/10', hour => '*', } }
Run Puppet:
ubuntu@cookbook:~/puppet$ papply Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull- updates]/ensure: defined content as '{md5}20cfc6cf2a40155d4055d475a109137d' Notice: /Stage[main]/Puppet/File[/home/ubuntu/.ssh/id_rsa]/ensure: defined content as '{md5}db19f750104d3bf4e2603136553c6f3e' Notice: Finished catalog run in 0.27 seconds
Test that the new SSH key is authorized to GitHub correctly:
ubuntu@cookbook:~/puppet$ ssh git@github.com PTY allocation request failed on channel 0 Hi bitfield/cookbook! You've successfully authenticated, but GitHub does not provide shell access. Connection to github.com closed.
Check that the
pull-updates
script works properly:ubuntu@cookbook:~/puppet$ pull-updates Already up-to-date. Notice: Finished catalog run in 0.16 seconds
Up to now, you've been using your own SSH credentials to access GitHub from the managed machine (using SSH agent forwarding), but that won't work if we want the machine to be able to pull updates unattended, while you're not logged in. So we've created a new SSH keypair and added the public part of it as a deploy key on GitHub, which gives repo access to anyone who has the private half of the key.
We've added this private key as the ubuntu
user's default SSH key:
file { '/home/ubuntu/.ssh/id_rsa': source => 'puppet:///modules/puppet/ubuntu.priv', owner => 'ubuntu', mode => '0600', }
This enables the ubuntu
user to run git pull
in the puppet
directory. We've also added the pull-updates
script, which does this and runs Puppet if any changes were pulled:
#!/bin/sh cd /home/ubuntu/puppet git pull && papply
We deploy this script to the box with Puppet:
file { '/usr/local/bin/pull-updates': source => 'puppet:///modules/puppet/pull-updates.sh', mode => '0755', }
Finally, we've created a cron job that runs pull-updates
at regular intervals (every 10 minutes, but feel free to change this if you need to):
cron { 'run-puppet': ensure => 'present', command => '/usr/local/bin/pull-updates', minute => '*/10', hour => '*', }
Congratulations, you now have a fully-automated Puppet infrastructure! Once you have checked out the repo on a new machine and applied the manifest, the machine will be set up to pull any new changes and apply them automatically.
So, for example, if you wanted to add a new user account to all your machines, all you have to do is add the account in your working copy of the manifest, and commit and push the changes to GitHub. Within 10 minutes it will automatically be applied to every machine that's running Puppet.
That's very handy, but sometimes we'd like to be able to apply the changes to a specific machine right away, without waiting for them to be picked up by the cron job. We can do this using the Rake tool, and we'll see how to do that in the next section.
Rake is a useful tool from the Ruby world which you can use to help automate your Puppet workflow. Although there are lots of other ways to run commands on remote servers, this happens to be the one I use, and it's easily extensible to whatever you need it to do.
The first helpful thing we can have Rake perform for us is log into the remote machine and run the pull-updates
script to apply any new Puppet manifest changes. This is fairly simple to do, as you'll see in the following sections.
You may already have Rake installed (try running rake
), but if not, here's how to install it:
Run the following command:
sudo apt-get install rake
In your Puppet repo, create the file,
Rakefile
with the following contents. Replacessh...
with the correctssh
command line for you to log into your server:SSH = 'ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu' desc "Run Puppet on ENV['CLIENT']" task :apply do client = ENV['CLIENT'] sh "git push" sh "#{SSH} #{client} pull-updates" end
Add, commit, and push the change to the Git repo:
ubuntu@cookbook:~/puppet$ git add Rakefile ubuntu@cookbook:~/puppet$ git commit -m "adding Rakefile" [master 63bb0c1] adding Rakefile 1 file changed, 8 insertions(+) create mode 100644 Rakefile ubuntu@cookbook:~/puppet$ git push Counting objects: 31, done. Compressing objects: 100% (22/22), done. Writing objects: 100% (28/28), 4.67 KiB, done. Total 28 (delta 1), reused 0 (delta 0) To git@github.com:bitfield/cookbook.git a063a5b..63bb0c1 master -> master
On your own computer, check out a working copy of the Puppet repo, if you haven't already got one (replace the Git URL with the URL to your own repo):
[john@Susie:~/git]$ git clone git@github.com:bitfield/cookbook.git Cloning into 'cookbook'... remote: Counting objects: 36, done. remote: Compressing objects: 100% (26/26), done. remote: Total 36 (delta 1), reused 33 (delta 1) Receiving objects: 100% (36/36), 5.28 KiB, done. Resolving deltas: 100% (1/1), done.
Run the following command, replacing
cookbook
with the address of your server:[john@Susie:~/git]$ cd cookbook [john@Susie:~/git/cookbook(master)]$ rake CLIENT= cookbook apply (in /Users/john/git/cookbook) git push Everything up-to-date ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu cookbook pull-updates Already up-to-date. Notice: Finished catalog run in 0.18 seconds Connection to cookbook closed.
What you'd do manually to update your server is log into it using SSH, and then run pull-updates
. The Rakefile simply automates this for you. First, we set up the correct SSH command line:
SSH = 'ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu'
The arguments to ssh
are as follows:
-A
: Forward your SSH key to the remote server, so that you can use it for further authentication-i KEYFILE
: Set the SSH private key to use (in this case, it's the Amazon AWS keyfile that I'm using. You may not need this argument if you're already set up for SSH access to the server with your default key.)-l ubuntu
: Log in as userubuntu
(the standard arrangement on EC2 servers; you may not need this argument if you log into servers using the same username as on your local machine.)
We then define a Rake task named apply
:
desc "Run Puppet on ENV['CLIENT']" task :apply do end
The desc
is just a helpful description, which you'll see if you run the command, rake -T
, which lists available tasks:
$ rake –T (in /Users/john/git/cookbook) rake apply # Run puppet on ENV['CLIENT']
The code between task
and end
will be run when you run rake apply
. Here's what it does:
client = ENV['CLIENT']
This captures the value of the CLIENT
environment variable, which tells the script the address of the remote servers to connect to.
The next line is as follows:
sh "git push"
sh
simply runs a command in your local shell, in this case to make sure any changes to the Puppet repo have been pushed to GitHub. If they weren't, they wouldn't be picked up on the remote machine.
sh "#{SSH} #{client} pull-updates"
This is the line which actually connects to the client, using the ssh
command line we defined at the start of the script, and the client address. Having logged in, it will then run the pull-updates
command as the remote user.
Since we already set up the pull-updates
script to do everything necessary to get the latest changes from GitHub and apply Puppet, that's all we need to do.
You can now make and apply Puppet changes to your remote servers without ever explicitly logging into them. Once you've installed Puppet on a machine, checked out a copy of the manifest repo, and run Puppet for the first time, you can then do all the administration for that machine remotely.
If you're as lazy as I am, you're already asking, "Couldn't we use a Rake task to do the initial Puppet install and checkout, as well as just applying changes?"
We certainly can, and we'll see how to do that in the next section.
To make a newly provisioned machine part of our Puppet infrastructure, we just need to run a few commands on it, so let's make this process even easier by adding a new bootstrap task to the Rakefile
.
To get ready for the recipe, do the following:
Add the following line to the top of your
Rakefile
:REPO = 'git@github.com:bitfield/cookbook.git'
Add the following task anywhere in the
Rakefile
:desc "Bootstrap Puppet on ENV['CLIENT'] with hostname ENV['HOSTNAME']" task :bootstrap do client = ENV['CLIENT'] hostname = ENV['HOSTNAME'] || client commands = <<BOOTSTRAP sudo hostname #{hostname} && \ sudo su - c 'echo #{hostname} >/etc/hostname' && \ wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && \ sudo dpkg -i puppetlabs-release-precise.deb && \ sudo apt-get update && sudo apt-get -y install git puppet && \ git clone #{REPO} puppet && \ sudo puppet apply --modulepath=/home/ubuntu/puppet /modules /home/ubuntu/puppet/manifests/site.pp BOOTSTRAP sh "#{SSH} #{client} '#{commands}'" end
You'll need a freshly provisioned server (one that you can log in to, but that doesn't have Puppet installed or any other config changes made on it). If you're using EC2, create a new EC2 instance. Get the public instance address from the AWS control panel; it'll be something like:
ec2-107-22-22-159.compute-1.amazonaws.com
Here are the steps to bootstrap the new server using Rake:
Add a node declaration to your
nodes.pp
file for the hostname you'll be using on the new server. For example, if you wanted to call itcookbook-test
, you could usenode 'cookbook-test' { include puppet }
Run the following command in the Puppet repo on your own machine (substitute the address of the new server as the value of
CLIENT
, and the hostname you want to use as the value ofHOSTNAME
). The command should all be on one line:$ rake CLIENT=ec2-107-22-22-159.compute-1.amazonaws.com HOSTNAME=cookbook-test bootstrap
You'll see output something like the following:
(in /Users/john/git/cookbook) ssh -A -i ~/git/bitfield/bitfield.pem -l ubuntu ec2-107-22-22-159.compute-1.amazonaws.com 'sudo hostname cookbook-test && sudo su -c 'echo cookbook-test >/etc/hostname' && wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && sudo dpkg -i puppetlabs-release-precise.deb && sudo apt-get update && sudo apt-get -y install git puppet && git clone git@github.com:bitfield/cookbook.git puppet && sudo puppet apply --modulepath=/home/ubuntu/puppet/modules /home/ubuntu/puppet/manifests/site.pp' The authenticity of host 'ec2-107-22-22-159.compute-1.amazonaws.com (107.22.22.159)' can't be established. RSA key fingerprint is 23:c5:06:ad:58:f3:8d:e5:75:bd:94:6e:1e:a0:a3:a4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ec2-107-22-22-159.compute-1.amazonaws.com,107.22.22.159' (RSA) to the list of known hosts. sudo: unable to resolve host cookbook-test --2013-03-15 15:53:44-- http://apt.puppetlabs.com/puppetlabs-release-precise.deb Resolving apt.puppetlabs.com (apt.puppetlabs.com)... 96.126.116.126, 2600:3c00::f03c:91ff:fe93:711a Connecting to apt.puppetlabs.com (apt.puppetlabs.com)|96.126.116.126|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 3392 (3.3K) [application/x-debian-package] Saving to: `puppetlabs-release-precise.deb' 0K 100% 302M=0s 2013-03-15 15:53:44 (302 MB/s) - `puppetlabs-release-precise.deb' saved [3392/3392] Selecting previously unselected package puppetlabs-release. (Reading database ... 25370 files and directories currently installed.) Unpacking puppetlabs-release (from puppetlabs-release-precise.deb) ... Setting up puppetlabs-release (1.0-5) ... Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-3.2.0-29-virtual Ign http://us-east-1.ec2.archive.ubuntu.com precise InRelease [ ... apt output redacted ... ] Setting up hiera (1.1.2-1puppetlabs1) ... Setting up puppet-common (3.2.2-1puppetlabs1) ... Setting up puppet (3.2.2-1puppetlabs1) ... * Starting puppet agent puppet not configured to start, please edit /etc/default/puppet to enable ...done. Processing triggers for libc-bin ... ldconfig deferred processing now taking place Cloning into 'puppet'... Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts. Notice: /Stage[main]/Puppet/Cron[run-puppet]/ensure: created Notice: /Stage[main]/Puppet/File[/usr/local/bin/pull-updates]/ensure: defined content as '{md5}20cfc6cf2a40155d4055d475a109137d' Notice: /Stage[main]/Puppet/File[/usr/local/bin/papply]/ensure: defined content as '{md5}171896840d39664c00909eb8cf47a53c' Notice: /Stage[main]/Puppet/File[/home/ubuntu/.ssh/id_rsa]/ensure: defined content as '{md5}db19f750104d3bf4e2603136553c6f3e' Notice: Finished catalog run in 0.11 seconds
Here's a line by line breakdown of what the Rake task does. In order to make the machine ready to run Puppet, we need to set its hostname to the name you've chosen:
sudo hostname #{hostname} sudo echo #{hostname} >/etc/hostname
Next, we download and install the Puppet Labs repo package, and install Puppet and Git:
wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb sudo dpkg -i puppetlabs-release-precise.deb sudo apt-get update && sudo apt-get -y install git puppet
We need to disable the SSH StrictHostKeyChecking
option to avoid being prompted when the script clones the Git repo:
echo -e \"Host github.com\n\tStrictHostKeyChecking no\n\" >> ~/.ssh/config
We check out the repo:
git clone #{REPO} puppet
And finally, run Puppet:
sudo puppet apply --modulepath=/home/ubuntu/puppet/modules /home/ubuntu/puppet/manifests/site.pp
The new machine will now pull and apply Puppet changes automatically, without you ever having to log into it interactively. You can use this Rake task to bring lots of new servers under Puppet control quickly.
It would be nice if we knew there was a syntax error in the manifest before we even committed it. You can have Puppet check the manifest using the puppet parser validate
command:
ubuntu@cookbook:~/puppet$ puppet parser validate manifests/nodes.pp Error: Could not parse for environment production: Syntax error at end of file; expected '}' at /home/ubuntu/puppet/manifests/nodes.pp:3 Error: Try 'puppet help parser validate' for usage
This is especially useful because a mistake anywhere in the manifest will stop Puppet from running on any node, even on nodes that don't use that particular part of the manifest. So checking in a bad manifest can cause Puppet to stop applying updates to production for some time, until the problem is discovered, and this could potentially have serious consequences. The best way to avoid this is to automate the syntax check, by using a pre-commit hook in your version control repo.
Follow these steps:
In your Puppet repo, create a new
hooks
directory:ubuntu@cookbook:~/puppet$ mkdir hooks
Create the file
hooks/check_syntax.sh
with the following contents (based on a script by Puppet Labs):#!/bin/sh syntax_errors=0 error_msg=$(mktemp /tmp/error_msg.XXXXXX) if git rev-parse --quiet --verify HEAD > /dev/null then against=HEAD else # Initial commit: diff against an empty tree object against=4b825dc642cb6eb9a060e54bf8d69288fbee4904 fi # Get list of new/modified manifest and template files to check (in git index) for indexfile in `git diff-index --diff-filter=AM -- name-only --cached $against | egrep '\.(pp|erb)'` do # Don't check empty files if [ `git cat-file -s :0:$indexfile` -gt 0 ] then case $indexfile in *.pp ) # Check puppet manifest syntax git cat-file blob :0:$indexfile | puppet parser validate > $error_msg ;; *.erb ) # Check ERB template syntax git cat-file blob :0:$indexfile | erb -x -T - | ruby -c 2> $error_msg > /dev/null ;; esac if [ "$?" -ne 0 ] then echo -n "$indexfile: " cat $error_msg syntax_errors=`expr $syntax_errors + 1` fi fi done rm -f $error_msg if [ "$syntax_errors" -ne 0 ] then echo "Error: $syntax_errors syntax errors found, aborting commit." exit 1 fi
Set execute permission for the
hook
script with the following command:ubuntu@cookbook:~/puppet$ chmod a+x .hooks/check_syntax.sh
Add the following task to your Rakefile:
desc "Add syntax check hook to your git repo" task :add_check do here = File.dirname(__FILE__) sh "ln -s #{here}/hooks/check_syntax.sh #{here}/.git/hooks/pre-commit" puts "Puppet syntax check hook added" end
Run the following command:
ubuntu@cookbook:~/puppet$ rake add_check ln -s /home/ubuntu/puppet/hooks/check_syntax.sh /home/ubuntu/puppet/.git/hooks/pre-commit Puppet syntax check hook added
The check_syntax.sh
script will prevent you from committing any files with syntax errors:
ubuntu@cookbook:~/puppet$ git commit -m "test commit" Error: Could not parse for environment production: Syntax error at '}' at line 3 Error: Try 'puppet help parser validate' for usage manifests/nodes.pp: Error: 1 syntax errors found, aborting commit.
If you add the hooks
directory to your Git repo, anyone who has a checkout can run the rake add_check
task and get this syntax checking behavior.