Chef Infrastructure

Exclusive offer: get 50% off this eBook here
Chef Infrastructure Automation Cookbook

Chef Infrastructure Automation Cookbook — Save 50%

Over 80 delicious recipes to automate your cloud and server infrastructure with Chef with this book and ebook

$26.99    $13.50
by Matthias Marschall | September 2013 | Cookbooks Networking & Telephony Open Source

This article by Matthias Marschall author of Chef Infrastructure Automation Cookbook helps you to get started with Chef. It explains some key concepts such as cookbooks, roles, and environments and shows you how to use some basic tools such as Git, Knife, Chef Shell, Vagrant, and Berkshelf.

"What made Manhattan Manhattan was the underground infrastructure, that engineering marvel." - Andrew Cuomo

This article will cover the basics of Chef, including common terminology, workflow practices, and various tools surrounding Chef. We will explore version control using Git, walk through working with community cookbooks, and running those cookbooks on your own servers to configure them the way you need them.

(For more resources related to this topic, see here.)

First, let's talk about the terminology used in the Chef universe.

A cookbook is a collection of recipes – codifying the actual resources, which should be installed and configured on your node – and the files and configuration templates needed.

Once you've written your cookbooks, you need a way to deploy them to the nodes you want to provision. Chef offers multiple ways for this task. The most widely used way is to use a central Chef Server. You can either run your own or sign up for Opscode's Hosted Chef.

The Chef Server is the central registry where each node needs to get registered. The Chef Server distributes the cookbooks to the nodes based on their configuration settings.

Knife is Chef's command-line tool called to interact with the Chef Server. You use it for uploading cookbooks and managing other aspects of Chef.

On your nodes, you need to install Chef Client – the part that retrieves the cookbooks from the Chef Server and executes them on the node.

In this article, we'll see the basic infrastructure components of your Chef setup at work and learn how to use the basic tools. Let's get started with having a look at how to use Git as a version control system for your cookbooks.

Using version control

Do you manually back up every file before you change it? And do you invent creative filename extensions like _me and _you when you try to collaborate on a file? If you answer yes to any of the preceding questions, it's time to rethink your process.

A version control system (VCS) helps you stay sane when dealing with important files and collaborating on them.

Using version control is a fundamental part of any infrastructure automation. There are multiple solutions (some free, some paid) for managing source version control including Git, SVN, Mercurial, and Perforce. Due to its popularity among the Chef community, we will be using Git. However, you could easily use any other version control system with Chef.

Getting ready

You'll need Git installed on your box. Either use your operating system's package manager (such as Apt on Ubuntu or Homebrew on OS X), or simply download the installer from www.git-scm.org.

Git is a distributed version control system. This means that you don't necessarily need a central host for storing your repositories. But in practice, using GitHub as your central repository has proven to be very helpful. In this article, I'll assume that you're using GitHub. Therefore, you need to go to github.com and create a (free) account to follow the instructions given in this article. Make sure that you upload your SSH key following the instructions at https://help.github.com/articles/generating-ssh-keys, so that you're able to use the SSH protocol to interact with your GitHub account.

As soon as you've created your GitHub account, you should create your repository by visiting https://github.com/new and using chef-repo as the repository name.

How to do it...

Before you can write any cookbooks, you need to set up your initial Git repository on your development box. Opscode provides an empty Chef repository to get you started. Let's see how you can set up your own Chef repository with Git using Opscode's skeleton.

  1. Download Opscode's skeleton Chef repository as a tarball:

    mma@laptop $ wget http://github.com/opscode/chef-repo/tarball/
    master

    ...TRUNCATED OUTPUT...
    2013-07-05 20:54:24 (125 MB/s) - 'master' saved [9302/9302]

  2. Extract the downloaded tarball:

    mma@laptop $ tar zvf master

  3. Rename the directory. Replace 2c42c6a with whatever your downloaded tarball contained in its name:

    mma@laptop $ mv opscode-chef-repo-2c42c6a/ chef-repo

  4. Change into your newly created Chef repository:

    mma@laptop $ cd chef-repo/

  5. Initialize a fresh Git repository:

    mma@laptop:~/chef-repo $ git init .
    Initialized empty Git repository in /Users/mma/work/chef-repo/.
    git/

  6. Connect your local repository to your remote repository on github.com. Make sure to replace mmarschall with your own GitHub username:

    mma@laptop:~/chef-repo $ git remote add origin git@github.
    com:mmarschall/chef-repo.git

  7. Add and commit Opscode's default directory structure:

    mma@laptop:~/chef-repo $ git add .
    mma@laptop:~/chef-repo $ git commit -m "initial commit"

    [master (root-commit) 6148b20] initial commit
    10 files changed, 339 insertions(+), 0 deletions(-)
    create mode 100644 .gitignore
    ...TRUNCATED OUTPUT...
    create mode 100644 roles/README.md

  8. Push your initialized repository to GitHub. This makes it available to all your co-workers to collaborate on it.

    mma@laptop:~/chef-repo $ git push -u origin master
    ...TRUNCATED OUTPUT...
    To git@github.com:mmarschall/chef-repo.git
    * [new branch] master -> master

How it works...

You've downloaded a tarball containing Opscode's skeleton repository. Then, you've initialized your chef-repo and connected it to your own repository on GitHub.

After that, you've added all the files from the tarball to your repository and committed them. This makes Git track your files and the changes you make later.

As a last step, you've pushed your repository to GitHub, so that your co-workers can use your code too.

There's more...

Let's assume you're working on the same chef-repo repository together with your co-workers. They cloned your repository, added a new cookbook called other_cookbook, committed their changes locally, and pushed their changes back to GitHub. Now it's time for you to get the new cookbook down to your own laptop.

Pull your co-workers, changes from GitHub. This will merge their changes into your local copy of the repository.

mma@laptop:~/chef-repo $ git pull From github.com:mmarschall/chef-repo * branch master -> FETCH_HEAD ...TRUNCATED OUTPUT... create mode 100644 cookbooks/other_cookbook/recipes/default.rb

In the case of any conflicting changes, Git will help you merge and resolve them.

Installing Chef on your workstation

If you want to use Chef, you'll need to install it on your local workstation first. You'll have to develop your configurations locally and use Chef to distribute them to your Chef Server.

Opscode provides a fully packaged version, which does not have any external prerequisites. This fully packaged Chef is called the Omnibus Installer. We'll see how to use it in this section.

Getting ready

Make sure you've curl installed on your box by following the instructions available at http://curl.haxx.se/download.html.

How to do it...

Let's see how to install Chef on your local workstation using Opscode's Omnibus Chef installer:

  1. In your local shell, run the following command:

    mma@laptop:~/chef-repo $ curl -L https://www.opscode.com/chef/
    install.sh | sudo bash
    Downloading Chef...
    ...TRUNCATED OUTPUT...
    Thank you for installing Chef!

  2. Add the newly installed Ruby to your path:

    mma@laptop:~ $ echo 'export PATH="/opt/chef/embedded/bin:$PATH"'
    >> ~/.bash_profile && source ~/.bash_profile

How it works...

The Omnibus Installer will download Ruby and all the required Ruby gems into /opt/chef/embedded. By adding the /opt/chef/embedded/bin directory to your .bash_profile, the Chef command-line tools will be available in your shell.

There's more...

If you already have Ruby installed in your box, you can simply install the Chef Ruby gem by running mma@laptop:~ $ gem install chef.

Using the Hosted Chef platform

If you want to get started with Chef right away (without the need to install your own Chef Server) or want a third party to give you an Service Level Agreement (SLA) for your Chef Server, you can sign up for Hosted Chef by Opscode. Opscode operates Chef as a cloud service. It's quick to set up and gives you full control, using users and groups to control the access permissions to your Chef setup. We'll configure Knife, Chef's command-line tool to interact with Hosted Chef, so that you can start managing your nodes.

Getting ready

Before being able to use Hosted Chef, you need to sign up for the service. There is a free account for up to five nodes.

Visit http://www.opscode.com/hosted-chef and register for a free trial or the free account.

I registered as the user webops with an organization short-name of awo.

After registering your account, it is time to prepare your organization to be used with your chef-repo repository.

How to do it...

Carry out the following steps to interact with the Hosted Chef:

  1. Navigate to http://manage.opscode.com/organizations. After logging in, you can start downloading your validation keys and configuration file.
  2. Select your organization to be able to see its contents using the web UI.

  3. Regenerate the validation key for your organization and save it as <your-organization-short-name>.pem in the .chef directory inside your chef-repo repository.

  4. Generate the Knife config and put the downloaded knife.rb into the .chef directory inside your chef-repo directory as well. Make sure you replace webops with the username you chose for Hosted Chef and awo with the short-name you chose for your organization:

    current_dir = File.dirname(__FILE__)
    log_level :info
    log_location STDOUT
    node_name "webops"
    client_key "#{current_dir}/webops.pem"
    validation_client_name "awo-validator"
    validation_key "#{current_dir}/awo-validator.pem"
    chef_server_url "https://api.opscode.com/organizations/
    awo"
    cache_type 'BasicFile'
    cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
    cookbook_path ["#{current_dir}/../cookbooks"]

  5. Use Knife to verify that you can connect to your hosted Chef organization. It should only have your validator client so far. Instead of awo, you'll see your organization's short-name:

    mma@laptop:~/chef-repo $ knife client list
    awo-validator

How it works...

Hosted Chef uses two private keys (called validators): one for the organization and the other for every user. You need to tell Knife where it can find these two keys in your knife.rb file.

The following two lines of code in your knife.rb file tells Knife about which organization to use and where to find its private key:

validation_client_name "awo-validator"
validation_key "#{current_dir}/awo-validator.pem"

The following line of code in your knife.rb file tells Knife about where to find your users' private key:

client_key "#{current_dir}/webops.pem"

And the following line of code in your knife.rb file tells Knife that you're using Hosted Chef. You will find your organization name as the last part of the URL:

chef_server_url "https://api.opscode.com/organizations/awo"

Using the knife.rb file and your two validators Knife can now connect to your organization hosted by Opscode.

You do not need your own, self-hosted Chef Server, nor do you need to use Chef Solo in this setup.

There's more...

This setup is good for you if you do not want to worry about running, scaling, and updating your own Chef Server and if you're happy with saving all your configuration data in the cloud (under Opscode's control).

If you need to have all your configuration data within your own network boundaries, you might sign up for Private Chef, which is a fully supported and enterprise-ready version of Chef Server.

If you don't need any advanced enterprise features like role-based access control or multi-tenancy, then the open source version of Chef Server might be just right for you.

Summary

In this article, we learned about key concepts such as cookbooks, roles, and environments and how to use some basic tools such as Git, Knife, Chef Shell, Vagrant, and Berkshelf.

Resources for Article:


Further resources on this subject:


Chef Infrastructure Automation Cookbook Over 80 delicious recipes to automate your cloud and server infrastructure with Chef with this book and ebook
Published: August 2013
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

About the Author :


Matthias Marschall

Matthias Marschall is a software engineer "Made in Germany". His four children make sure that he feels comfortable in lively environments, and stays in control of chaotic situations. A lean and agile engineering lead, he's passionate about continuous delivery, infrastructure automation, and all things DevOps.

In recent years, Matthias has helped build several web-based businesses, first with Java and then with Ruby on Rails. He quickly grew into system administration, writing his own configuration management tool before moving his whole infrastructure to Chef in its early days.

In 2008, he started a blog (http://www.agileweboperations.com) together with Dan Ackerson. There they shared their ideas about DevOps since the early days of the continually emerging movement. You can find him on Twitter as @mmarschall.

Matthias is a CTO at gutefrage.net GmbH, helping run Germany's biggest Q&A site among other high-traffic sites. He holds a Master's degree in Computer Science (Dipl.-Inf. (FH)) and teaches courses on Agile Software Development at the University of Augsburg.

When not writing or coding, Matthias enjoys drawing cartoons and playing Go. He lives near Munich, Germany.

Books From Packt


Raspberry Pi Home Automation with Arduino
Raspberry Pi Home Automation with Arduino

Netduino Home Automation Projects
Netduino Home Automation Projects

 Learn OpenOffice.org Spreadsheet Macro Programming: OOoBasic and Calc automation
Learn OpenOffice.org Spreadsheet Macro Programming: OOoBasic and Calc automation

BMC Control-M 7: A Journey from Traditional Batch Scheduling to Workload Automation
BMC Control-M 7: A Journey from Traditional Batch Scheduling to Workload Automation

Instant Sikuli Test Automation [Instant]
Instant Sikuli Test Automation [Instant]

Windows Server 2012 Automation with PowerShell Cookbook
Windows Server 2012 Automation with PowerShell Cookbook

Instant AutoIt Scripting [Instant]
Instant AutoIt Scripting [Instant]

Oracle SOA Infrastructure Implementation Certification Handbook (1Z0-451)
Oracle SOA Infrastructure Implementation Certification Handbook (1Z0-451)


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software