Home Cloud & Networking Mastering Chef Provisioning

Mastering Chef Provisioning

By Earl Waud
books-svg-icon Book
eBook $35.99 $24.99
Print $43.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $35.99 $24.99
Print $43.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Setting Up a Development Environment on Your Workstation
About this book
This book will show you the best practices to describe your entire infrastructure as code. With the help of this book you can expand your knowledge of Chef because and implement robust and scalable automation solutions. You can automate and document every aspect of your network, from the hardware to software, middleware, and all your containers. You will become familiar with the Chef’s Chef Provisioning tool. You will be able to make a perfect model system where everything is represented as code beneath your fingertips. Make the best possible use of your resources, and deliver infrastructure as code, making it as versionable, testable and repeatable as application software
Publication date:
June 2016
Publisher
Packt
Pages
262
ISBN
9781785888915

 

Chapter 1. Setting Up a Development Environment on Your Workstation

In today's fast-paced IT world, it is a requirement to deliver infrastructure at warp speed. There's really only one way to achieve this requirement, and that is through automation.

One of the best paths to infrastructure automation is via Chef. Using Chef, you can turn your infrastructure into code. With infrastructure as code, you can automate the way you build, deploy, and manage all of it. Using Chef, your infrastructure becomes very consistent, very duplicable, and version-controlled. Using Chef, you can easily test your infrastructure setup and configuration. With Chef, you can become an IT superhero!

In this chapter, we're going to detail the setup and configuration of a complete development system or workstation. The main focus will be centered on deploying the Chef Development Kit (ChefDK) and preparing the various components of a development environment. The chapter also includes suggestions for additional tools to round out the DevOps toolbox. However, before we dive into ChefDK, we will fly through a high-level review of the what-and-how of Chef so that we are all on the same page.

"If you wish to make an apple pie truly from scratch, you must first invent the universe." – Carl Sagan

Here is what you will find in this chapter:

  • Filling in the gaps in your Chef knowledge base

  • Exploring ChefDK

  • Installing ChefDK

  • Setting up a standard Chef repo

  • What else do you need?

  • Using chef-apply

 

Re-introducing Chef


If you are reading this book, then you've probably been working with Chef for some time now, and you know the many benefits it brings. You may also know that, in the past, it was a somewhat daunting task to set up a new Chef workstation. You had to download and install Chef, then download and install a variety of community tools, and make sure that all the versions were compatible and configured correctly.

Today, a lot of the work is done for you via the ChefDK. Once you install it, you have a basic workstation setup and are ready to create, modify, and test Chef code.

What version of the Chef tools do I use?

To make it easy to follow along with the contents, let's go over the versions of the tools that will be used throughout this book.

In March 2016, chef.io announced the release of ChefDK 0.12.0, which includes Chef client 12.8.1. As this is the latest version at the time of writing, I will be using it as the version for this book. All of the examples shown will be based on this version of the ChefDK and Chef client. This is very exciting because a lot of really exciting changes are in this release, including Policies and the transition from Resource Providers to Custom Resources.

Currently, there are ChefDK installers available for Mac OS, Windows OS, and Linux OS (RHEL, Debian, and Ubuntu). In the examples within this book, the workstation used will be Mac OS X, so the ChefDK version will be the Mac OS version.

With the many choices available to use for your Chef Server mode, including Hosted Chef Server, Private Chef Server, Open Source Chef Server, and Chef-Solo, it would be difficult to show examples for each mode. Therefore, the majority of examples you'll see in this book will be based on using the Hosted Chef Server mode. Later in this chapter, I will briefly review the installation and setup of a Private Chef Server onto an Ubuntu server, for readers who want to use Chef Server on-premise.

Which OS do I use for my workstation? Everyone has their own, nearly religious, choice for the best workstation platform. However, it would make this book way too long to provide examples for the major OSes alone. Therefore, to keep the focus on Chef content and not on the differences between workstation implementations, I will be using a Mac OS X (Yosemite version 10.10) workstation for the examples. I may at times show additional examples on a Windows or Ubuntu workstation where the differences are significant and worth the extra detail. And for the nodes used in the examples, a variety of OSes will be represented in the hope of having some overlap with the reader's real environment.

References

 

Filling in the gaps in your Chef knowledge base


First things first. Let's have a quick, high-level review of Chef. A Chef 101 class, if you will. The plan here is to make sure that all readers are on the same page with me so that the rest of the book will be easier to follow and benefit from. If you are already a Chef ninja, you can probably skip right on past this section and start exploring the ChefDK in the next section. Otherwise, let's audit our 101 class now.

Major pieces of a Chef puzzle

Chef is a ruby framework, and its main purpose is to facilitate the automation, reuse, and documentation of server configurations. Chef allows you to treat server configuration just like any other type of software code.

There are three major components of this framework for almost all corporate-level Chef deployments—the workstation, the Chef Server, and the nodes. Let's talk a little about each of these components:

  • Workstations: The Chef workstation is, as the name would suggest, where the real "work" is done. This is where developers will create their infrastructure code. It's also where they will test their creations. The workstation is where developers will interact with their source code control systems. It is where genius is born. It is where the "desired state" of nodes is defined via code.

  • Chef servers: The Chef server is a database-backed web API server with a browsable user interface. It is like the matrix housing the hive mind of the Chef universe. The "work" created in the workstation is uploaded into the Chef server so that it can be used to automate the configuration of the many nodes. It stands ready to provide the desired state information to the Chef client on a node so that it can bring that node into alignment or convergence with the desired state of configuration.

  • Nodes: Nodes are the ultimate targets of the "work" that's been created on the workstation and uploaded to the Chef server. A node is where the automation occurs via the Chef client. Nodes are transformed into the desired state configuration. Commonly, nodes are servers, either virtual or physical, but a node can be anything that needs to be configured, such as a Docker container, or a network device such as a switch or router. Every node has a unique name, such as the FQDN of a server.

The Chef client

The Chef client is the tool that is deployed to all nodes and used to configure or reconfigure the node to its desired state. The Chef client is what executes the automation. It is the magician that, through the magic of Chef, transforms an ordinary server into the desired state node that the developer defined back on his workstation.

The Ohai Chef tool

Ohai is the tool that gathers information about a node. Information such as platform details, operating system data, and processor information is made available to the Chef client so that the latter can have the know-how to bring the node to the desired state. Ohai is executed at the beginning of a Chef client run to gather the state of the node. At the end of a Chef client run, all of the data gathered by Ohai (usually) is shared as node data with the Chef server, and the shared data is available for searches done against the Chef server. Ohai can be extended via plugins, and we'll take a look at that in a later chapter.

Recipes and cookbooks

Recipes are the building blocks used to define the desired states. Recipes are files of ruby code that define the commands to be run on nodes. They are like blueprints used to "build" a node. Recipes are collections of Chef resources. We will explain more about Chef resources shortly.

Cookbooks are collections of related recipes, templates, files, and custom resources. They provide organization and versioning for recipes. Each unique version of a cookbook represents unique sets of functionality, such as bug fixes or added features.

Cookbooks define a scenario, such as everything needed to install and configure apt-docker or Sublime text, and they contain all the elements needed to support the defined scenario.

Recipes and cookbooks provide modularity and let you easily reuse code.

Chef run lists

A "run list" is, as the name would suggest, a list of, and the sequence for, the recipes, cookbooks, and policies (spoiler alert!) to be applied to a node. A run list contains all of the information required to configure a node to a desired state. That is, a Chef run list describes the desired final state of the node. It is important to note that, if the node's state already matches what the resources in the run list describe, then no action will be taken to change the node's state.

Chef roles

Roles are functional groupings of recipes and cookbooks used to describe the full blueprint needed for a node to become everything it is intended to be. Roles are reusable configurations, and they can be applied to multiple nodes to make functionally identical servers, such as a farm of web servers.

Chef resources

Chef resources are statements of configuration policy. They are defined in recipes and take actions through the Chef client to put the node into the desired state. Chef resources have some types: Package, Template, Service, and so on. They have a name and parameters. Also, Chef resources can send notifications to other resources.

Chef resources define what we want to happen to a node. They don't say how to do it. The how to do it is left to the providers which are platform-specific. That is to say, the way you install a package will be different depending on the OS, and the provider determines the correct way to do it—the "how". The Chef resource simply defines the "what," such as "install ntp".

  • Package: This contains software or applications, such as apache, ntp, and cron, and the action to be performed on that software or application, such as "install".

  • Template: These are files with place-holders for attributes that are transformed into configuration files for package installation and execution.

  • Service: This is the installed executable of the package and the actions that the executable can perform, such as start, stop, or restart. Service also defines whether the software or application is launched at node startup.

More things to consider

Apart from the three types of Chef resources we saw earlier, let's look at some other important aspects of recipes and cookbooks.

  • Attributes: These are variables used in recipes and templates. Generally speaking, a recipe will represent the pattern used in configuration. The attributes provide the specific details for the recipes. For example, the recipe will say "configure the port" and the attribute will say "which port?". Attributes can be provided in a large variety of places, such as cookbooks, roles, and environments. As such, there is a necessity for an order of precedence. There is a complete description of attribute precedence on the chef.io site. You can find it at https://docs.chef.io/attributes.html.

  • Order matters: When creating recipes, the resources need to be listed in order. First, the package, then the template, and finally the service; when creating run lists, the order of policies, roles, cookbooks, and recipes is the order in which they are evaluated. Any recipe that appears in a run list (or as a dependency of a policy, role, or cookbook) more than once will only be evaluated the first time.

  • Convergence and idempotence: A Chef client run converges the node into the desired state. What this means is that only things that do not equal the desired state on the node are modified during the Chef client run. If, for example, the recipe says that the desired state of the node is to have the ntp demon running, the ntp package is already installed on the node, and the ntp demon is already running, then Chef client will take no action to install or start ntp.

Chef resources are idempotent. That is, applying them more than once results in the same outcome every time. If no inputs related to the resource have changed, then applying that resource won't change anything after the first application. In fact, if none of the inputs have changed, the corresponding commands don't even get run.

This is actually one of the most important concepts and features of Chef.

Data bags

Data bags are containers for information that is not tied to a specific node. Data bags are the global variables of a Chef server. They can be used in recipes and can be searched like node data via the Chef server. One common use case for data bags is for user and group information. Items in a data bag can be encrypted. This allows secret information to be stored in them, for example, passwords.

Environments

Environments allow you to define specific cookbook versions that are applied to a given set of nodes. They permit you to model the stages of your infrastructure workflow, that is, Development, Test, Stage, and Production. By identifying specific nodes as Development, and other nodes as Production, for example, you can apply different versions of your cookbooks to the nodes based on their environment membership.

The Chef supermarket

The supermarket is a site that provides shared cookbooks. There is a public supermarket that contains community-created and -maintained cookbooks. This site is hosted by Chef and is available at https://supermarket.chef.io. In addition to the public supermarket, anyone can create and manage a private supermarket that can host cookbooks intended for the private consumption of you and your organization.

Tip

It is always best practice to do a thorough code review of any cookbooks obtained from the public community supermarket before using them in your production environments.

Chef Development Kit

The ChefDK contains everything you need to start working with Chef on a workstation. It provides all the tools that a developer needs to create and modify cookbooks and upload them to a Chef server. We are going to go into a lot of detail regarding the ChefDK later in this chapter.

You can jump ahead to the ChefDK sections now, or you can read on and learn how to set up your very own private Chef server.

Setting up an on-premise (private) Chef Server

Many companies will have strong reasons to keep their infrastructure configuration data within the firewalls of their datacenters, and Chef server has a mode for that. It's called on-premise or private Chef server.

There are some real benefits to using an in-house solution, including control. With a private installation, you have full control of your system, and for some that reason is enough. But there are additional considerations such as performance. Since a private Chef server will likely be physically closer to the node's network, Chef client runs will be faster. For example, if Chef is used in a load-based on-demand server deployment solution, then the extra speed the private Chef server can provide when converging your new Nodes can make all the difference in meeting the load demand in time.

With all the reasons to use an on-premise Chef server, there is a downside: you are responsible for the deployment, configuration, and maintenance of your Chef servers.

To get you started, let's take a look at the deployment and initial configuration of a standalone private Chef server now. We'll go through the steps to get a new Chef server set up on an Ubuntu 14.04 server.

The first step is downloading the required Chef server installer version. Visit the Chef server page at this link https://downloads.chef.io/chef-server/ to find and download the version you will use for your installation.

Next, you will want to transfer the downloaded installer package to your soon to be Chef server.

Once the installer is available on your server, you will want to install the package. The command to do this is:

sudo dpkg -i chef-server-core_12.2.0-1_amd64.deb

Once the Chef server package has been installed onto your server, the first thing to do is start the Chef server services. The command to do this is:

sudo chef-server-ctl reconfigure

The Chef server startup does a lot of work and as such you will see a lot of activity displayed as the setup progresses. When the startup command finally finishes successfully (a few minutes later; you might want to get a cup of coffee), the result will be a running Chef server.

Tip

Do you get an error in the nginx.rb file during the reconfigure?

The private Chef server has a dependency on the nginx cookbook, and in some cases you may experience an error when this dependency is resolved. The error may look something like this:

"common_name nil currently does not overwrite the value of common_name."

What this error message is probably indicating is that there is no valid host entry in the /etc/hosts file. You should check that file, and if there isn't a valid entry, create one.

Are we there yet? Well, we do have a Chef server up-and-running, but it is not much good to us yet. If you browse to your new Chef server, you will see something like this:

That is a good start, but there is still work to be done. Next up is creating your first Chef user on the Chef server with the user-create command. That user will be the initial Chef admin for the server. The format of the user-create command is:

sudo chef-server-ctl user-create user_name first_name last_name email password --filename FILE_NAME

Here is an example of what that command might look like:

sudo chef-server-ctl user-create earlwaud Earl Waud earl@sandiegoearl.com MyPassword99 --filename /home/earlwaud/earlwaud.pem

Here is what it looks like when you issue the command on your Chef server (along with the user-show command):

The filename parameter used in the user-create command provides the location at which to store the user's private key. The contents of this key file should be securely stored so that they can be provided to the user whose account has been created.

The next step is to create the first organization in your Chef server. The format of that command is as follows:

sudo chef-server-ctl org-create short_name "full_organization_name" --association_user user_name --filename ORGANIZATION-validator.pem

Here is the command I used to create the organization on my example Chef server:

sudo chef-server-ctl org-create sdearl "San Diego Earl" --association_user earlwaud --filename /home/earlwaud/sdearl-validator.pem

You can see that, in the example, the key file was saved at /home/earlwaud/sdearl-validator.pem.

If the filename parameter is not provided in the org-create command, the validator key will be displayed to stdout.

Either way, it is vital that the key information be captured and securely stored since it is not saved in the Chef server database and cannot be recovered if lost. The only option available when an organization's private key is completely lost is to reset the validation key.

Next, we need to install a package that will allow management of the Chef server via a web interface. The package is named opscode-manage. The command to install the package is:

sudo chef-server-ctl install opscode-manage

Now we need to restart the Chef server services by issuing the reconfigure command again:

sudo chef-server-ctl reconfigure

And finally, we need to start the opscode-manage services. This is done by issuing the command:

sudo opscode-manage-ctl reconfigure

If everything went as planned, you should now be able to browse to the new Chef server, and log in with your newly created admin user. Of course, there is still more work to do, such as integrating your new Chef server with your corporate domain to allow domain user access control. Still, this is a good start, right?

References

 

Exploring the Chef Development Kit


The Chef Development Kit, or ChefDK, is the omnibus installer used to set up a Chef development environment on a workstation. It includes the Chef client itself, an early version of the new Chef tool, an embedded version of Ruby, RubyGems, OpenSSL, and command-line utilities such as Knife, Ohai, and Chef Zero. It also includes community tools, such as Test Kitchen, Foodcritic, Berkshelf, Chef Vault, Rubocop, and ChefSpec.

Included in the ChefDK installer is the new Chef command-line tool. It is the new go-to way to generate cookbooks, recipes, templates, custom resources, and more. It also downloads RubyGems for the chef-client environment, and it will verify the ChefDK installation and configuration.

The Chef client is a key component of everything Chef. The Chef client is not only used on nodes for converging configuration data, but it is also used on workstations and the Chef server.

The Chef Knife

Knife is a command-line tool that supplies the interface between your chef-repo and your Chef server. It allows developers to upload their work on the workstation to the Chef server, from where it can be distributed to the nodes during their Chef client runs. Knife also allows you to obtain cookbooks from the supermarket, and it provides a mechanism to bootstrap nodes.

Community tools

Some of the most commonly used community tools are installed as part of the ChefDK. Test Kitchen is installed, so you can test the results of your recipes and cookbook development on deployed test nodes. Berkshelf is installed to manage your cookbooks and their dependencies. Foodcritic is included in the ChefDK installation. Foodcritic will help you check your cookbooks for common errors and omissions.

Another valuable community tool installed with the ChefDK is Rubocop. You can use it to give you cookbooks some "style" and make sure that you're following the same conventions used by other developers in your organization. There is also ChefSpec, which is based on the ruby tool RSpec. ChefSpec will allow you to unit-test your cookbooks and recipes. We will have to exercise a lot of these community tools in later chapters, so please stay tuned.

References

 

Installing Chef Development Kit


In this section, you will learn how easy it is to get your workstation up and running using the Chef Development Kit. First, you will download ChefDK. Next, you will install ChefDK; and finally, you will validate the installation with some quick version checks.

Downloading ChefDK

You are going to download Chef Development Kit by opening your favorite browser and visiting the Chef.io ChefDK downloads page found at this URL: https://downloads.chef.io/chef-dk/.

  1. Select the platform you are going to install the Chef Development Kit on and then click on the download button that corresponds to the OS Version you are using. In my case, I am selecting the Mac OS X 10.10 version of the ChefDK installer.

  2. As expected, clicking on the download button will transfer the ChefDK installer to your local workstation.

  3. Expand the Downloads folder and click on the ChefDK disk image file to mount the image on your desktop.

Installing ChefDK

To initiate the installer for the Chef Development Kit, double-click on the mounted ChefDK image icon.

  1. The ChefDK image will open, exposing the chefdk package. Double-click on the package icon to begin the installation process.

  2. You will see the Chef Development Kit installer introduction screen. Click on the Continue button to advance the installation.

  3. You will be presented with license information for the Chef Development Kit, which is the commonly used Apache License. I would suggest that you read through the entire license document and perhaps even consult with your lawyer to make sure that you are able to accept this license agreement. Then, click on the Continue button to advance the installation.

  4. Once you've carefully read and understood the license agreement, click on the Agree button to advance the installation.

  5. At this point, you can begin to customize the installation. The options available here are to install for all users or only the currently logged in user. In the example, I am installing for all users of this computer (even though I am the only user).

  6. Next, you can choose the install location. For consistency with regard to the examples in this book, I have left the install location at the default. Click on the Install button when you are ready to begin the actual installation.

  7. As always, you will have to provide credentials to allow the system to install new software. Enter your username and password and click on the Install Software button to continue.

The installer will now go about its business of installing the super fantastic Chef Development Kit, including the corresponding versions of community tools such as Test Kitchen, Food Critic, and Rubocup.

Finally, you will get the installation summary screen congratulating you on your successful installation and keen DevOps Kung-Fu.

The captain has turned off the seat-belt sign, so it is now OK to unmount the ChefDK image, and move about the installation.

Tip

Installing ChefDK on Ubuntu

It is even easier to install the ChefDK on an Ubuntu system. Once you have downloaded the installer, you simply issue a dpkg command as follows:

sudo dpkg -i ~/Downloads/chefdk_0.12.0-1_amd64.deb

That's it. You should see the "Thank you for installing Chef Development Kit" message.

Verifying ChefDK

The ChefDK installer is a superhero for DevOps aficionados, saving them time and energy by installing the major necessities for a Chef workstation. Still, let's double-check the work done by the installer to make sure that we are really ready to go.

First, let's just try a simple command to see if things look right. Open a terminal window and issue the command chef --help. If the install was successful, you should see information about using Chef.

Next, let's use the new Chef tool to do a deeper validation of the install. Issue the following command in your terminal session:

chef verify

You should see the Chef tool validate each of the major packages installed during the ChefDK installation. Note that, depending on the state of your (OS X) workstation at the time you run this command, you may receive an error message regarding the need to install the command-line tools for gcc. It may look something like this:

If you get a message such as the preceding one, go ahead and install the command-line tools and then run the chef verify command again. This time, you should get a clean run of the command, and the output should be very similar to that shown in the preceding screenshot.

Now, at this point, things are looking pretty good, and you should be feeling fine. So, let's do one last validation check just to be 100% certain. Let's check the versions of some of the expected tools installed via the ChefDK installer.

Issue the following commands:

chef --version
ohai --version
foodcritic --version
rubocop --version
rspec --version

Your results should look something like this:

Hey wait… what happened to rspec? The Version 0.12 ChefDK installer does not seem to have successfully installed rspec, which is a requirement for the Chef-specific tool ChefSpec. That's not good. What went wrong? Everything seemed to have been installed based on the verify command we just used. What gives?

Well, rspec is installed, there is just one more step that we need to do. By the way, I think this is something that should be done by the ChefDK installer, but for whatever reason it's not, and it falls on our shoulders to handle it.

So what do we have to do? We need to set up the environment used in our terminal window for our Chef workstation development. How do we do that?

Fortunately, Chef has a command for that. It is the chef shell-init command. You can run this command interactively every time you open a new terminal window, or you can update your profile so that it is run automatically. I would recommend updating your profile, but let me show you both ways so you can make the call.

To make the interactive change, you would issue a command in the following format:

eval "$(chef shell-init SHELL_NAME)"

The actual call will be the following for the bash shell:

eval "$(chef shell-init bash)"

If you want to make the change more permanent, then you need to issue a command in the following format:

echo 'eval "$(chef shell-init SHELL_NAME)"' >> ~/.YOUR_SHELL_RC_FILE

If you are using Mac, the commands will be:

echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
source ~/.bash_profile

If you are using Ubuntu, the commands will be:

echo 'eval "$(chef shell-init bash)"' >> ~/.bashrc
source ~/.bashrc

This echo command will add the eval command to your bash startup so that each run of your bash session will already have the desired environment ready to go for your Chef development work.

It should look something like this:

By the way, when you install the ChefDK onto a Windows system, the installation creates a Chef Development Kit icon on your desktop. When you execute from icon, a Chef-specific PowerShell session is launched. It will automatically run the chef shell-init command and set up the full environment needed for Chef. So there is no need to do this last step on a Windows workstation.

Tip

Chef Development Kit on a Windows workstation

When you launch Chef Development Kit from your Windows workstation, remember to right-click the icon and choose "Run as administrator." This allows PowerShell to use the correct permissions to set up the Chef environment.

Well, there you have it: the successful installation of the ChefDK. Your workstation is ready to do some real Chef work now. You should feel pretty awesome!

References

 

Setting up a standard Chef repo


Ever Chef workstation has at least one Chef repository. The Chef repo is where all working copies of the cookbooks, recipes, templates, and so on live. It is where you do your development work. It is from where you check your work into your source code repository, and it is from where you upload your work to the Chef server.

Let's take a look at how to set up a new empty Chef repo on your new ChefDK-installed workstation.

Using the Chef tool for Chef repo

Another benefit of the new Chef command-line tool is that it will create a standard chef repo for you. You no longer need to download or clone a starter repo from GitHub. You can use the chef generate command to create your new repo. Let's take a look at the chef generate repo command's help:

chef generate repo -help

Based on the description of the default values for the chef generate repo command, we can simply issue the following command on our workstation:

chef generate repo ~/chef-repo

That was easy! So what do we get? Well, let's have a look:

tree -a chef-repo
chef-repo
├── .chef-repo.txt
├── .git
│   ├── HEAD
│   ├── branches
│   ├── config
│   ├── description
│   ├── hooks
│   │   ├── applypatch-msg.sample
│   │   ├── commit-msg.sample
│   │   ├── post-update.sample
│   │   ├── pre-applypatch.sample
│   │   ├── pre-commit.sample
│   │   ├── pre-push.sample
│   │   ├── pre-rebase.sample
│   │   ├── prepare-commit-msg.sample
│   │   └── update.sample
│   ├── info
│   │   └── exclude
│   ├── objects
│   │   ├── info
│   │   └── pack
│   └── refs
│       ├── heads
│       └── tags
├── .gitignore
├── LICENSE
├── README.md
├── chefignore
├── cookbooks
│   ├── README.md
│   └── example
│       ├── README.md
│       ├── attributes
│       │   └── default.rb
│       ├── metadata.rb
│       └── recipes
│           └── default.rb
├── data_bags
│   ├── README.md
│   └── example
│       └── example_item.json
├── environments
│   ├── README.md
│   └── example.json
└── roles
    ├── README.md
    └── example.json

This list of files looks pretty good but, as you may have noted, an essential folder is missing: the .chef folder, which needs to contain the private key files and the knife.rb file, used to communicate with the Chef server. Let's take a look at creating or obtaining these files. One way is to use the starter kit. It is an OK way if this is a new Chef server. But, if it's not, be sure to read the following Starter Kit info; it could save your job!

The repo starter kit

You still need the .chef files for your new workstation to securely communicate with your Chef server. These files are your user private key file, the organization's validator private key file, and the knife.rb file. These files need to be placed in a .chef folder in your chef-repo.

When setting up a new organization in your Chef server, there is an easy way to get the files: with the repo starter kit.

Warning!!! Only download the starter kit if you are the Chef server admin and are the first (or only) person setting up your repo for a brand new organization. Currently, the act of downloading the starter kit will reset both your user private key and the organization validator key.

Note

Let me say this again. Downloading the Starter Kit will reset the private key values for both your user and your organization.

Downloading the starter kit on a mature organization will essentially break the Internet.

With this warning, if you are still going to use the starter kit, proceed at your own risk.

To get the starter kit, you will want to browse to your Chef server and log in. Once there, click on Administration and then select your Organization. Now click on Starter Kit and you will see the Download Starter Kit button. When you click the button, you will get the warning shown earlier. Click on the Proceed button to transfer a zip archive to your workstation, then unzip it into your chef-repo folder, and you're done.

Files for your .chef folder – a safer way

Really, the only files you need from the starter kit are the .chef folder files:

  • The organization's validator private key file

  • Your user private key file

  • A knife.rb file

The organization's key file can be provided by an administrator of the Chef server. The organization's validator key is generated when the organization is created. When the command-line tool is used to create the org, the location of the validator key file may be provided in the command. Otherwise, it will be displayed to stdout. The Chef server admin will have captured the validator key info and will be able to provide it to you now.

The user private key is much like the organization's private key in that it is created when the user is created and must be captured and securely stored at that time as it is not saved to the Chef server database. A user can reset their private key through the use of the Reset Key feature available when viewing the user profile on a Chef server. A user resetting their private key will have no negative impact on other users or the organization, so it is relatively safe to do.

Finally, you need the knife.rb file. This file is the easy one. It can be downloaded from the Chef server any time by logging in and browsing to the organization. From there, you can click on any of the Generate Knife Config links in the UI. This will create the knife.rb file and download it to your workstation. Another option is to use the knife configure command. This will prompt you through the process of creating your knife.rb file.

Put all three of these files into the .chef folder of your new chef-repo. Make sure that you save the two key files using the names in the knife.rb file (or conversely, update the contents of the knife.rb file to match the file names used to save your key files).

Growing your own tree

OS X doesn't come with a version of the tree command installed so, if you like that tool and use a Mac, I've found a clever way to have your cake and eat it too by creating your own tree script. This script was used to create the "tree" output shown in the preceding screenshot. This idea was shared by users Ahmed Masud and JakeGould on http://superuser.com. It goes like this:

Self-signed certificates

When you are using an On Premise Chef server, you need to do one more thing to finish setting up your chef-repo. If the Chef server you are connecting to has a self-signed certificate, then you will need to add the server's certificate to your chef-repo to allow your workstation to trust your Chef server. An easy way to check if this applies to you is to run the ssl check command. It goes like this:

knife ssl check

If the Chef Server you are communicating with has a self-signed certificate, you will see a message like this:

As you can see in the error message generated by the ssl check command, the Chef server's certificate needs to be obtained and placed into a trusted_certs folder in your chef-repo/.chef folder. Again, you will need to contact your Chef server administrator to get a copy of the certificate file. Once you have it, just place it in the trusted certs folder; when you run your ssl check command again, you should get a message back that says something like:

knife ssl check
Connecting to host chefserver:443
Successfully verified certificates from 'chefserver'

With that successful knife command completed, you have verified that you have successfully set up your chef-repo and are ready to start creating your infrastructure as code.

References

 

What else do you need?


You will need just a few more things to round out your workstation toolbox. First, you will want to make sure that you have your favorite text editor and have it configured to integrate with Chef. Next, since you are creating infrastructure code, you really must use a source code control system. I don't want to give anything away on that topic yet but … Git Git Git! Then, you will want to add a hypervisor to your workstation to deploy test nodes. Finally, you'll want to add Vagrant to manage those test nodes. So, let's take a look at these finishing touches for your workstation.

Using your favorite editor

Everyone has their favorite text editor. For a lot of people that editor is Sublime Text, but there are many other choices available. A lot of old schoolers still prefer to use a flavor of the vi editor. Many OS X users swear by Mac-only TextMate. And now there's a new kid on the block from the makers of GitHub: Atom 1.0. Sublime Text and Atom are both available for multiple platforms, so whatever OS is your poison, they have a version for you.

Tip

I would recommend that, if you don't already use an editor that has a "project space" feature,you switch to one that does. This feature allows you to quickly and easily switch to and edit different files within a project without the delay of the "save current then open new" workflow needed in an editor that does not have the project space feature.

The project space feature is one of the many reasons I usually recommend Sublime Text, but this feature is also present in other editors such as the new Atom 1.0.

To keep things current, I will use the Atom 1.0 editor in the remainder of this book's examples.

Once you have decided on the editor you plan to use for your Chef workstation and have that editor installed and working, there is one more step. You will need to set the configuration to allow you to integrate your editor with the Chef command-line tools. The best way to do that is to edit your .chef/knife.rb file and add a line to set the editor you want to use with your knife commands. (You could also add an export EDITOR= command to your shell's .bashrc or .bash_profile file.)

Here are some sample knife.rb entries:

Sublime Text:

knife[:editor] = '/Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl -w'

Atom:

knife[:editor] = '/Applications/Atom.app/Contents/MacOS/Atom -w'

On Mac, the Atom editor has a built-in feature to create symbolic links in the /usr/local/bin folder. With the Atom editor running, open the Atom menu and select Install Shell Commands from the submenu. This will create two new links for you. If you do this before updating your knife.rb file, the knife[:editor] entry simply becomes this:

knife[:editor] = 'atom -w'

When you're using a Windows OS for your workstation, you will have to take greater care in formatting your knife[:editor] line to escape the slash characters in the path. A convenient way to do that is to use single quotes around the full value and use double quotes for the path. For Sublime Text, it might look something like this:

knife[:editor] = '"C:\Program Files\Sublime Text 2\sublime_text.exe" --wait'

You can test your editor setting by issuing a knife command that requires using the editor. For example, try the command:

knife client create editor-tester

If your editor integration is configured correctly, you will see your editor open up and show you some code. Save the code and exit the editor, and you should see the private key value print out. (This key is not important at this time because we are just testing the editor, and will immediately delete the client.)

If your editor opens, you are done. If you get an error that says Please set EDITOR environment variable, then your knife[:editor] value is incorrect. There is probably a typo in the path or some other error. Double-check the value and try again until you successfully launch the editor with your knife client create command.

Once you have validated the integration with your editor, you can delete the test client with the following command:

knife client delete editor-tester

Now you are ready to edit your infrastructure code! When you are creating amazing infrastructure code, you are going to want to make sure that it gets preserved and to ensure that happens you going to want to use a source control tool. Say hello to my little friend, Git.

Version control systems

Git, initially designed and developed by Linus Torvalds for Linux kernel development back in 2005, is now the most widely used version control system for developers everywhere. It's the number-one choice due to its speed, data integrity, and its distributed workflow model. Git is certainly the primary choice as the version control system to be used when developing infrastructure code with Chef.

I will be emphasizing the use of Git throughout the examples in the rest of the book. Git is included with Mac OS X, but it will need to be installed on Windows and Ubuntu workstations. I recommend you to use the GitHub desktop installer where available (Windows and OS X) and allow it to deploy the command-line tools as part of the installation. This installation of Git is easy and will be left to the reader to execute.

Virtualization hypervisor

There are several virtualization choices you can use for your Chef testing. The overall goal is to have a system that you can use to deploy a temporary Chef node to test your infrastructure code. One of the best ways to accomplish this is to install a local virtualization tool on your workstation. Among the choices for a local hypervisor are:

  • VirtualBox

  • Docker

  • VMware Workstation or Fusion

One of the best choices for this kind of Chef testing is VirtualBox. The reasons include its functionality, its ease of use, and its free price tag. Either of the VMware solutions will cost you not only for the hypervisor layer, but also for the integration plugins we'll use with Vagrant to manage our test nodes.

Visit the VirtualBox downloads page at https://www.virtualbox.org/wiki/Downloads to download the installer. Once downloaded, it is an easy matter to install it, so I'll leave that to the reader to take care of.

Vagrant

Last, but certainly not least, is Vagrant. Vagrant is another item I consider to be one of the key pieces of the Chef testing puzzle. It is the "glue" between Chef and the hypervisor that deploys the test nodes during testing. You will want to download and install Vagrant on your workstation. You can find the downloaders at https://www.vagrantup.com/downloads.html. Vagrant is also an easy install, so I'll leave that to the reader to handle.

I'll go into using Vagrant with VitualBox (and Docker) in some detail in a later chapter.

References

 

What is chef-apply?


One often overlooked tool that is installed with the ChefDK is chef-apply. This tool basically allows you to execute chef to converge a single recipe on your local machine. There is no Chef server involved. Everything is local.

Note that chef-apply is not a tool for the deployment of production nodes. It's just a quick and easy way to use Chef to configure your local system.

Why is that useful?

One use case for chef-apply is to learn what a recipe does. You can use chef-apply to run a recipe in a "test only" mode by using the -W or --why-run parameter. This will converge the recipe, showing you everything that would happen with the recipe without making actual changes to your system.

You can also execute some one-liner scripts. You can easily use chef-apply in a scripting-like manner to install software. For example, if you need to use Git on your system and you find it's not installed. Just issue the command:

sudo chef-apply -e "package 'git'"

This will check to see if Git is currently installed. If it is, then the command will just inform you that that is the case and exit. If Git is not installed, then it will go about using the correct installer (aka Provider) to obtain and install the package.

Testing this on my Ubuntu workstation, you can see that Git was not installed initially. Then, by using the chef-apply command, the git package is installed. Then to confirm that nothing happens if you try to use chef-apply to install a package that is already installed, I ran the same chef-apply again to show what happens:

How about creating a "workstation" recipe to set up all the "missing" items from a new ChefDK workstations setup? You create the recipe that defines the state of having your editor installed and integrated with Chef, defines the state of having git installed, and defines the state of having your custom OS X "tree" script created in /usr/local/bin. You create this recipe once, and you (and anyone in your organization) can quickly complete their workstation setup by using chef-apply with your workstation recipe.

References

 

Summary


In this chapter, we've taken a quick stroll down memory lane and reviewed some Chef basics so that those concepts are fresh and easy to mentally access during the rest of the journey through this book. Next up, we took a deeper dive into the Chef Development Kit and learned how to download, install, and configure it for our Chef workstation. We've exposed the small gap in the ChefDK installer when used on Mac OS X and Linux systems, which is the setup of the shell environment and learned the simple way (provided by the new Chef tool) to fill that gap. We also explored the setup and configuration of your very own on-premise Chef server, including creating your admin user and the first organization. After this, we took a look at setting up our Chef repo, giving a BIG WARNING about the starter kit. And then we considered what was still missing from a well-rounded Chef workstation. That will be our favorite editor, a version control system, a virtualization layer to simplify our testing flow, and Vagrant to glue things together. Finally, as an added bonus, we talked about the ChefDK tool chef-apply.

Wow! That was a lot of ground for a single chapter, but I hope that you were able to stick with me till the end. This chapter is a great foundation for the Chef knowledge transfer coming your way in the chapters ahead.

So without any further delay, let's charge ahead into Chapter 2, Knife Tooling and Plugins, and carve things up with some Swiss Army knife skills.

About the Author
  • Earl Waud

    Earl Waud is a virtualization development professional with more than 20 year's experience developing customer-facing, enterprise-grade software for VM, and works with container provisioning, management, and automation. Earl has a proven track record of delivering on-time solutions that significantly impact business results; solutions that align engineering strategies with organizational vision. Recently, Earl has been creating AWS-based container solutions for enterprises using Docker, Kubernetes, Artifactory, Xray, and Twistlock. Currently, Earl is a senior software engineer with Intuit Inc. Other books by Earl include Mastering Chef Provisioning and Chef: Powerful Infrastructure Automation. Earl can be found online at SanDiegoEarl (dot) com.

    Browse publications by this author
Latest Reviews (2 reviews total)
Testi completi e pienamente fruibili
Still learning from the resource, but this far I am not disappointed.
Mastering Chef Provisioning
Unlock this book and the full library FREE for 7 days
Start now