In today's fast-paced IT world, it is a requirement to deliver infrastructure at warp speed. There's really only one way to achieve this requirement, and that is through automation.
One of the best paths to infrastructure automation is via Chef. Using Chef, you can turn your infrastructure into code. With infrastructure as code, you can automate the way you build, deploy, and manage all of it. Using Chef, your infrastructure becomes very consistent, very duplicable, and version-controlled. Using Chef, you can easily test your infrastructure setup and configuration. With Chef, you can become an IT superhero!
In this chapter, we're going to detail the setup and configuration of a complete development system or workstation. The main focus will be centered on deploying the Chef Development Kit (ChefDK) and preparing the various components of a development environment. The chapter also includes suggestions for additional tools to round out the DevOps toolbox. However, before we dive into ChefDK, we will fly through a high-level review of the what-and-how of Chef so that we are all on the same page.
"If you wish to make an apple pie truly from scratch, you must first invent the universe." – Carl Sagan
Here is what you will find in this chapter:
Filling in the gaps in your Chef knowledge base
Setting up a standard Chef repo
What else do you need?
If you are reading this book, then you've probably been working with Chef for some time now, and you know the many benefits it brings. You may also know that, in the past, it was a somewhat daunting task to set up a new Chef workstation. You had to download and install Chef, then download and install a variety of community tools, and make sure that all the versions were compatible and configured correctly.
Today, a lot of the work is done for you via the ChefDK. Once you install it, you have a basic workstation setup and are ready to create, modify, and test Chef code.
In March 2016, chef.io announced the release of ChefDK 0.12.0, which includes Chef client 12.8.1. As this is the latest version at the time of writing, I will be using it as the version for this book. All of the examples shown will be based on this version of the ChefDK and Chef client. This is very exciting because a lot of really exciting changes are in this release, including Policies and the transition from Resource Providers to Custom Resources.
Currently, there are ChefDK installers available for Mac OS, Windows OS, and Linux OS (RHEL, Debian, and Ubuntu). In the examples within this book, the workstation used will be Mac OS X, so the ChefDK version will be the Mac OS version.
With the many choices available to use for your Chef Server mode, including Hosted Chef Server, Private Chef Server, Open Source Chef Server, and Chef-Solo, it would be difficult to show examples for each mode. Therefore, the majority of examples you'll see in this book will be based on using the Hosted Chef Server mode. Later in this chapter, I will briefly review the installation and setup of a Private Chef Server onto an Ubuntu server, for readers who want to use Chef Server on-premise.
Which OS do I use for my workstation? Everyone has their own, nearly religious, choice for the best workstation platform. However, it would make this book way too long to provide examples for the major OSes alone. Therefore, to keep the focus on Chef content and not on the differences between workstation implementations, I will be using a Mac OS X (Yosemite version 10.10) workstation for the examples. I may at times show additional examples on a Windows or Ubuntu workstation where the differences are significant and worth the extra detail. And for the nodes used in the examples, a variety of OSes will be represented in the hope of having some overlap with the reader's real environment.
First things first. Let's have a quick, high-level review of Chef. A Chef 101 class, if you will. The plan here is to make sure that all readers are on the same page with me so that the rest of the book will be easier to follow and benefit from. If you are already a Chef ninja, you can probably skip right on past this section and start exploring the ChefDK in the next section. Otherwise, let's audit our 101 class now.
Chef is a ruby framework, and its main purpose is to facilitate the automation, reuse, and documentation of server configurations. Chef allows you to treat server configuration just like any other type of software code.
There are three major components of this framework for almost all corporate-level Chef deployments—the workstation, the Chef Server, and the nodes. Let's talk a little about each of these components:
Workstations: The Chef workstation is, as the name would suggest, where the real "work" is done. This is where developers will create their infrastructure code. It's also where they will test their creations. The workstation is where developers will interact with their source code control systems. It is where genius is born. It is where the "desired state" of nodes is defined via code.
Chef servers: The Chef server is a database-backed web API server with a browsable user interface. It is like the matrix housing the hive mind of the Chef universe. The "work" created in the workstation is uploaded into the Chef server so that it can be used to automate the configuration of the many nodes. It stands ready to provide the desired state information to the Chef client on a node so that it can bring that node into alignment or convergence with the desired state of configuration.
Nodes: Nodes are the ultimate targets of the "work" that's been created on the workstation and uploaded to the Chef server. A node is where the automation occurs via the Chef client. Nodes are transformed into the desired state configuration. Commonly, nodes are servers, either virtual or physical, but a node can be anything that needs to be configured, such as a Docker container, or a network device such as a switch or router. Every node has a unique name, such as the FQDN of a server.
The Chef client is the tool that is deployed to all nodes and used to configure or reconfigure the node to its desired state. The Chef client is what executes the automation. It is the magician that, through the magic of Chef, transforms an ordinary server into the desired state node that the developer defined back on his workstation.
Ohai is the tool that gathers information about a node. Information such as platform details, operating system data, and processor information is made available to the Chef client so that the latter can have the know-how to bring the node to the desired state. Ohai is executed at the beginning of a Chef client run to gather the state of the node. At the end of a Chef client run, all of the data gathered by Ohai (usually) is shared as node data with the Chef server, and the shared data is available for searches done against the Chef server. Ohai can be extended via plugins, and we'll take a look at that in a later chapter.
Recipes are the building blocks used to define the desired states. Recipes are files of ruby code that define the commands to be run on nodes. They are like blueprints used to "build" a node. Recipes are collections of Chef resources. We will explain more about Chef resources shortly.
Cookbooks are collections of related recipes, templates, files, and custom resources. They provide organization and versioning for recipes. Each unique version of a cookbook represents unique sets of functionality, such as bug fixes or added features.
Cookbooks define a scenario, such as everything needed to install and configure apt-docker or Sublime text, and they contain all the elements needed to support the defined scenario.
Recipes and cookbooks provide modularity and let you easily reuse code.
A "run list" is, as the name would suggest, a list of, and the sequence for, the recipes, cookbooks, and policies (spoiler alert!) to be applied to a node. A run list contains all of the information required to configure a node to a desired state. That is, a Chef run list describes the desired final state of the node. It is important to note that, if the node's state already matches what the resources in the run list describe, then no action will be taken to change the node's state.
Roles are functional groupings of recipes and cookbooks used to describe the full blueprint needed for a node to become everything it is intended to be. Roles are reusable configurations, and they can be applied to multiple nodes to make functionally identical servers, such as a farm of web servers.
Chef resources are statements of configuration policy. They are defined in recipes and take actions through the Chef client to put the node into the desired state. Chef resources have some types: Package, Template, Service, and so on. They have a name and parameters. Also, Chef resources can send notifications to other resources.
Chef resources define what we want to happen to a node. They don't say how to do it. The how to do it is left to the providers which are platform-specific. That is to say, the way you install a package will be different depending on the OS, and the provider determines the correct way to do it—the "how". The Chef resource simply defines the "what," such as "install ntp".
Service: This is the installed executable of the package and the actions that the executable can perform, such as start, stop, or restart. Service also defines whether the software or application is launched at node startup.
Apart from the three types of Chef resources we saw earlier, let's look at some other important aspects of recipes and cookbooks.
Attributes: These are variables used in recipes and templates. Generally speaking, a recipe will represent the pattern used in configuration. The attributes provide the specific details for the recipes. For example, the recipe will say "configure the port" and the attribute will say "which port?". Attributes can be provided in a large variety of places, such as cookbooks, roles, and environments. As such, there is a necessity for an order of precedence. There is a complete description of attribute precedence on the chef.io site. You can find it at https://docs.chef.io/attributes.html.
Order matters: When creating recipes, the resources need to be listed in order. First, the package, then the template, and finally the service; when creating run lists, the order of policies, roles, cookbooks, and recipes is the order in which they are evaluated. Any recipe that appears in a run list (or as a dependency of a policy, role, or cookbook) more than once will only be evaluated the first time.
Convergence and idempotence: A Chef client run converges the node into the desired state. What this means is that only things that do not equal the desired state on the node are modified during the Chef client run. If, for example, the recipe says that the desired state of the node is to have the ntp demon running, the
ntppackage is already installed on the node, and the ntp demon is already running, then Chef client will take no action to install or start ntp.
Chef resources are idempotent. That is, applying them more than once results in the same outcome every time. If no inputs related to the resource have changed, then applying that resource won't change anything after the first application. In fact, if none of the inputs have changed, the corresponding commands don't even get run.
This is actually one of the most important concepts and features of Chef.
Data bags are containers for information that is not tied to a specific node. Data bags are the global variables of a Chef server. They can be used in recipes and can be searched like node data via the Chef server. One common use case for data bags is for user and group information. Items in a data bag can be encrypted. This allows secret information to be stored in them, for example, passwords.
Environments allow you to define specific cookbook versions that are applied to a given set of nodes. They permit you to model the stages of your infrastructure workflow, that is, Development, Test, Stage, and Production. By identifying specific nodes as Development, and other nodes as Production, for example, you can apply different versions of your cookbooks to the nodes based on their environment membership.
The supermarket is a site that provides shared cookbooks. There is a public supermarket that contains community-created and -maintained cookbooks. This site is hosted by Chef and is available at https://supermarket.chef.io. In addition to the public supermarket, anyone can create and manage a private supermarket that can host cookbooks intended for the private consumption of you and your organization.
The ChefDK contains everything you need to start working with Chef on a workstation. It provides all the tools that a developer needs to create and modify cookbooks and upload them to a Chef server. We are going to go into a lot of detail regarding the ChefDK later in this chapter.
You can jump ahead to the ChefDK sections now, or you can read on and learn how to set up your very own private Chef server.
Many companies will have strong reasons to keep their infrastructure configuration data within the firewalls of their datacenters, and Chef server has a mode for that. It's called on-premise or private Chef server.
There are some real benefits to using an in-house solution, including control. With a private installation, you have full control of your system, and for some that reason is enough. But there are additional considerations such as performance. Since a private Chef server will likely be physically closer to the node's network, Chef client runs will be faster. For example, if Chef is used in a load-based on-demand server deployment solution, then the extra speed the private Chef server can provide when converging your new Nodes can make all the difference in meeting the load demand in time.
With all the reasons to use an on-premise Chef server, there is a downside: you are responsible for the deployment, configuration, and maintenance of your Chef servers.
To get you started, let's take a look at the deployment and initial configuration of a standalone private Chef server now. We'll go through the steps to get a new Chef server set up on an Ubuntu 14.04 server.
The first step is downloading the required Chef server installer version. Visit the Chef server page at this link https://downloads.chef.io/chef-server/ to find and download the version you will use for your installation.
Next, you will want to transfer the downloaded installer package to your soon to be Chef server.
Once the installer is available on your server, you will want to install the package. The command to do this is:
sudo dpkg -i chef-server-core_12.2.0-1_amd64.deb
Once the Chef server package has been installed onto your server, the first thing to do is start the Chef server services. The command to do this is:
sudo chef-server-ctl reconfigure
The Chef server startup does a lot of work and as such you will see a lot of activity displayed as the setup progresses. When the startup command finally finishes successfully (a few minutes later; you might want to get a cup of coffee), the result will be a running Chef server.
Do you get an error in the nginx.rb file during the reconfigure?
The private Chef server has a dependency on the nginx cookbook, and in some cases you may experience an error when this dependency is resolved. The error may look something like this:
"common_name nil currently does not overwrite the value of common_name."
What this error message is probably indicating is that there is no valid host entry in the
/etc/hosts file. You should check that file, and if there isn't a valid entry, create one.
That is a good start, but there is still work to be done. Next up is creating your first Chef user on the Chef server with the
user-create command. That user will be the initial Chef admin for the server. The format of the
user-create command is:
sudo chef-server-ctl user-create user_name first_name last_name email password --filename FILE_NAME
sudo chef-server-ctl user-create earlwaud Earl Waud firstname.lastname@example.org MyPassword99 --filename /home/earlwaud/earlwaud.pem
Here is what it looks like when you issue the command on your Chef server (along with the
The filename parameter used in the
user-create command provides the location at which to store the user's private key. The contents of this key file should be securely stored so that they can be provided to the user whose account has been created.
The next step is to create the first organization in your Chef server. The format of that command is as follows:
sudo chef-server-ctl org-create short_name "full_organization_name" --association_user user_name --filename ORGANIZATION-validator.pem
Here is the command I used to create the organization on my example Chef server:
sudo chef-server-ctl org-create sdearl "San Diego Earl" --association_user earlwaud --filename /home/earlwaud/sdearl-validator.pem
You can see that, in the example, the key file was saved at
If the filename parameter is not provided in the
org-create command, the validator key will be displayed to
Either way, it is vital that the key information be captured and securely stored since it is not saved in the Chef server database and cannot be recovered if lost. The only option available when an organization's private key is completely lost is to reset the validation key.
sudo chef-server-ctl install opscode-manage
Now we need to restart the Chef server services by issuing the
reconfigure command again:
sudo chef-server-ctl reconfigure
And finally, we need to start the opscode-manage services. This is done by issuing the command:
sudo opscode-manage-ctl reconfigure
If everything went as planned, you should now be able to browse to the new Chef server, and log in with your newly created admin user. Of course, there is still more work to do, such as integrating your new Chef server with your corporate domain to allow domain user access control. Still, this is a good start, right?
Explore the Chef community Supermarket at this link: https://supermarket.chef.io
Download Chef server from https://downloads.chef.io/chef-server/
Learn more about installing Chef server at https://docs.chef.io/release/server_12-2/install_server.html
Learn more about attributes and attribute precedence at https://docs.chef.io/attributes.html
Read more about the "common name" error at https://github.com/chef-cookbooks/chef-server/issues/108
The Chef Development Kit, or ChefDK, is the omnibus installer used to set up a Chef development environment on a workstation. It includes the Chef client itself, an early version of the new Chef tool, an embedded version of Ruby, RubyGems, OpenSSL, and command-line utilities such as Knife, Ohai, and Chef Zero. It also includes community tools, such as Test Kitchen, Foodcritic, Berkshelf, Chef Vault, Rubocop, and ChefSpec.
Included in the ChefDK installer is the new Chef command-line tool. It is the new go-to way to generate cookbooks, recipes, templates, custom resources, and more. It also downloads RubyGems for the chef-client environment, and it will verify the ChefDK installation and configuration.
Knife is a command-line tool that supplies the interface between your chef-repo and your Chef server. It allows developers to upload their work on the workstation to the Chef server, from where it can be distributed to the nodes during their Chef client runs. Knife also allows you to obtain cookbooks from the supermarket, and it provides a mechanism to bootstrap nodes.
Some of the most commonly used community tools are installed as part of the ChefDK. Test Kitchen is installed, so you can test the results of your recipes and cookbook development on deployed test nodes. Berkshelf is installed to manage your cookbooks and their dependencies. Foodcritic is included in the ChefDK installation. Foodcritic will help you check your cookbooks for common errors and omissions.
Another valuable community tool installed with the ChefDK is Rubocop. You can use it to give you cookbooks some "style" and make sure that you're following the same conventions used by other developers in your organization. There is also ChefSpec, which is based on the ruby tool RSpec. ChefSpec will allow you to unit-test your cookbooks and recipes. We will have to exercise a lot of these community tools in later chapters, so please stay tuned.
Learn more about the Chef Development Kit at https://docs.chef.io/chef_dk.html
Learn more about Berkshelf in this video-recorded interview at https://www.chef.io/blog/chefconf-talks/the-berkshelf-way-jamie-winsor/
Learn more about Foodcritic at this link http://www.foodcritic.io/
Learn about the Knife command line tool at this link: https://docs.chef.io/knife.html
And download an excellent printable Knife reference guide at this link https://github.com/chef/quick-reference/blob/master/qr_knife_web.png
In this section, you will learn how easy it is to get your workstation up and running using the Chef Development Kit. First, you will download ChefDK. Next, you will install ChefDK; and finally, you will validate the installation with some quick version checks.
You are going to download Chef Development Kit by opening your favorite browser and visiting the Chef.io ChefDK downloads page found at this URL: https://downloads.chef.io/chef-dk/.
Select the platform you are going to install the Chef Development Kit on and then click on the download button that corresponds to the OS Version you are using. In my case, I am selecting the Mac OS X 10.10 version of the ChefDK installer.
As expected, clicking on the download button will transfer the ChefDK installer to your local workstation.
Downloadsfolder and click on the ChefDK disk image file to mount the image on your desktop.
The ChefDK image will open, exposing the chefdk package. Double-click on the package icon to begin the installation process.
You will see the Chef Development Kit installer introduction screen. Click on the Continue button to advance the installation.
You will be presented with license information for the Chef Development Kit, which is the commonly used Apache License. I would suggest that you read through the entire license document and perhaps even consult with your lawyer to make sure that you are able to accept this license agreement. Then, click on the Continue button to advance the installation.
Once you've carefully read and understood the license agreement, click on the Agree button to advance the installation.
At this point, you can begin to customize the installation. The options available here are to install for all users or only the currently logged in user. In the example, I am installing for all users of this computer (even though I am the only user).
Next, you can choose the install location. For consistency with regard to the examples in this book, I have left the install location at the default. Click on the Install button when you are ready to begin the actual installation.
As always, you will have to provide credentials to allow the system to install new software. Enter your username and password and click on the Install Software button to continue.
The installer will now go about its business of installing the super fantastic Chef Development Kit, including the corresponding versions of community tools such as Test Kitchen, Food Critic, and Rubocup.
Finally, you will get the installation summary screen congratulating you on your successful installation and keen DevOps Kung-Fu.
The captain has turned off the seat-belt sign, so it is now OK to unmount the ChefDK image, and move about the installation.
Installing ChefDK on Ubuntu
sudo dpkg -i ~/Downloads/chefdk_0.12.0-1_amd64.deb
That's it. You should see the "Thank you for installing Chef Development Kit" message.
The ChefDK installer is a superhero for DevOps aficionados, saving them time and energy by installing the major necessities for a Chef workstation. Still, let's double-check the work done by the installer to make sure that we are really ready to go.
First, let's just try a simple command to see if things look right. Open a terminal window and issue the command
chef --help. If the install was successful, you should see information about using Chef.
Next, let's use the new Chef tool to do a deeper validation of the install. Issue the following command in your terminal session:
You should see the Chef tool validate each of the major packages installed during the ChefDK installation. Note that, depending on the state of your (OS X) workstation at the time you run this command, you may receive an error message regarding the need to install the command-line tools for gcc. It may look something like this:
If you get a message such as the preceding one, go ahead and install the command-line tools and then run the
chef verify command again. This time, you should get a clean run of the command, and the output should be very similar to that shown in the preceding screenshot.
Now, at this point, things are looking pretty good, and you should be feeling fine. So, let's do one last validation check just to be 100% certain. Let's check the versions of some of the expected tools installed via the ChefDK installer.
Issue the following commands:
chef --version ohai --version foodcritic --version rubocop --version rspec --version
Hey wait… what happened to rspec? The Version 0.12 ChefDK installer does not seem to have successfully installed rspec, which is a requirement for the Chef-specific tool ChefSpec. That's not good. What went wrong? Everything seemed to have been installed based on the verify command we just used. What gives?
Well, rspec is installed, there is just one more step that we need to do. By the way, I think this is something that should be done by the ChefDK installer, but for whatever reason it's not, and it falls on our shoulders to handle it.
So what do we have to do? We need to set up the environment used in our terminal window for our Chef workstation development. How do we do that?
Fortunately, Chef has a command for that. It is the
chef shell-init command. You can run this command interactively every time you open a new terminal window, or you can update your profile so that it is run automatically. I would recommend updating your profile, but let me show you both ways so you can make the call.
To make the interactive change, you would issue a command in the following format:
eval "$(chef shell-init SHELL_NAME)"
The actual call will be the following for the bash shell:
eval "$(chef shell-init bash)"
If you want to make the change more permanent, then you need to issue a command in the following format:
echo 'eval "$(chef shell-init SHELL_NAME)"' >> ~/.YOUR_SHELL_RC_FILE
If you are using Mac, the commands will be:
echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile source ~/.bash_profile
echo 'eval "$(chef shell-init bash)"' >> ~/.bashrc source ~/.bashrc
echo command will add the eval command to your bash startup so that each run of your bash session will already have the desired environment ready to go for your Chef development work.
It should look something like this:
By the way, when you install the ChefDK onto a Windows system, the installation creates a Chef Development Kit icon on your desktop. When you execute from icon, a Chef-specific PowerShell session is launched. It will automatically run the chef
shell-init command and set up the full environment needed for Chef. So there is no need to do this last step on a Windows workstation.
Chef Development Kit on a Windows workstation
When you launch Chef Development Kit from your Windows workstation, remember to right-click the icon and choose "Run as administrator." This allows PowerShell to use the correct permissions to set up the Chef environment.
Visit this link to download the installer for the ChefDK or Chef server at https://www.chef.io/chef/get-chef/.
Here is Chef.io's information on installing the Chef Development Kit: https://docs.chef.io/install_dk.html. The Chef DK change log can be found at https://github.com/chef/chef-dk/blob/master/CHANGELOG.md.
Ever Chef workstation has at least one Chef repository. The Chef repo is where all working copies of the cookbooks, recipes, templates, and so on live. It is where you do your development work. It is from where you check your work into your source code repository, and it is from where you upload your work to the Chef server.
Another benefit of the new Chef command-line tool is that it will create a standard chef repo for you. You no longer need to download or clone a starter repo from GitHub. You can use the
chef generate command to create your new repo. Let's take a look at the
chef generate repo command's help:
chef generate repo -help
Based on the description of the default values for the chef generate repo command, we can simply issue the following command on our workstation:
chef generate repo ~/chef-repo
That was easy! So what do we get? Well, let's have a look:
tree -a chef-repo chef-repo ├── .chef-repo.txt ├── .git │ ├── HEAD │ ├── branches │ ├── config │ ├── description │ ├── hooks │ │ ├── applypatch-msg.sample │ │ ├── commit-msg.sample │ │ ├── post-update.sample │ │ ├── pre-applypatch.sample │ │ ├── pre-commit.sample │ │ ├── pre-push.sample │ │ ├── pre-rebase.sample │ │ ├── prepare-commit-msg.sample │ │ └── update.sample │ ├── info │ │ └── exclude │ ├── objects │ │ ├── info │ │ └── pack │ └── refs │ ├── heads │ └── tags ├── .gitignore ├── LICENSE ├── README.md ├── chefignore ├── cookbooks │ ├── README.md │ └── example │ ├── README.md │ ├── attributes │ │ └── default.rb │ ├── metadata.rb │ └── recipes │ └── default.rb ├── data_bags │ ├── README.md │ └── example │ └── example_item.json ├── environments │ ├── README.md │ └── example.json └── roles ├── README.md └── example.json
This list of files looks pretty good but, as you may have noted, an essential folder is missing: the
.chef folder, which needs to contain the private key files and the
knife.rb file, used to communicate with the Chef server. Let's take a look at creating or obtaining these files. One way is to use the starter kit. It is an OK way if this is a new Chef server. But, if it's not, be sure to read the following Starter Kit info; it could save your job!
You still need the
.chef files for your new workstation to securely communicate with your Chef server. These files are your user private key file, the organization's validator private key file, and the
knife.rb file. These files need to be placed in a
.chef folder in your chef-repo.
When setting up a new organization in your Chef server, there is an easy way to get the files: with the repo starter kit.
Warning!!! Only download the starter kit if you are the Chef server admin and are the first (or only) person setting up your repo for a brand new organization. Currently, the act of downloading the starter kit will reset both your user private key and the organization validator key.
Let me say this again. Downloading the Starter Kit will reset the private key values for both your user and your organization.
With this warning, if you are still going to use the starter kit, proceed at your own risk.
To get the starter kit, you will want to browse to your Chef server and log in. Once there, click on Administration and then select your Organization. Now click on Starter Kit and you will see the Download Starter Kit button. When you click the button, you will get the warning shown earlier. Click on the Proceed button to transfer a zip archive to your workstation, then unzip it into your chef-repo folder, and you're done.
The organization's validator private key file
Your user private key file
The organization's key file can be provided by an administrator of the Chef server. The organization's validator key is generated when the organization is created. When the command-line tool is used to create the org, the location of the validator key file may be provided in the command. Otherwise, it will be displayed to
stdout. The Chef server admin will have captured the validator key info and will be able to provide it to you now.
The user private key is much like the organization's private key in that it is created when the user is created and must be captured and securely stored at that time as it is not saved to the Chef server database. A user can reset their private key through the use of the Reset Key feature available when viewing the user profile on a Chef server. A user resetting their private key will have no negative impact on other users or the organization, so it is relatively safe to do.
Finally, you need the
knife.rb file. This file is the easy one. It can be downloaded from the Chef server any time by logging in and browsing to the organization. From there, you can click on any of the Generate Knife Config links in the UI. This will create the
knife.rb file and download it to your workstation. Another option is to use the
knife configure command. This will prompt you through the process of creating your knife.rb file.
Put all three of these files into the
.chef folder of your new chef-repo. Make sure that you save the two key files using the names in the
knife.rb file (or conversely, update the contents of the
knife.rb file to match the file names used to save your key files).
OS X doesn't come with a version of the tree command installed so, if you like that tool and use a Mac, I've found a clever way to have your cake and eat it too by creating your own tree script. This script was used to create the "tree" output shown in the preceding screenshot. This idea was shared by users Ahmed Masud and JakeGould on http://superuser.com. It goes like this:
When you are using an On Premise Chef server, you need to do one more thing to finish setting up your chef-repo. If the Chef server you are connecting to has a self-signed certificate, then you will need to add the server's certificate to your chef-repo to allow your workstation to trust your Chef server. An easy way to check if this applies to you is to run the
ssl check command. It goes like this:
knife ssl check
If the Chef Server you are communicating with has a self-signed certificate, you will see a message like this:
As you can see in the error message generated by the
ssl check command, the Chef server's certificate needs to be obtained and placed into a
trusted_certs folder in your
chef-repo/.chef folder. Again, you will need to contact your Chef server administrator to get a copy of the certificate file. Once you have it, just place it in the trusted
certs folder; when you run your
ssl check command again, you should get a message back that says something like:
knife ssl check Connecting to host chefserver:443 Successfully verified certificates from 'chefserver'
With that successful knife command completed, you have verified that you have successfully set up your chef-repo and are ready to start creating your infrastructure as code.
Learn about setting up your Chef repo at the link https://docs.chef.io/chef_repo.html.
Learn more about setting up a chef-repo at the link https://learn.chef.io/manage-a-web-app/ubuntu/get-set-up/.
OS X doesn't come with a version of tree installed so, if you like that tool and use Mac, there is a clever way to have your cake and eat it too by creating your own a tree script. You can find an example shared by Ahmed Masud and JakeGould at the link http://superuser.com/questions/359723/mac-os-x-equivalent-of-the-ubuntu-tree-command.
Keep up-to-date on the Starter Kit resetting the organization's validator key at the link https://feedback.chef.io/forums/301644-chef-product-feedback/suggestions/9618459-starter-kit-should-not-reset-validator-and-user-ke.
You will need just a few more things to round out your workstation toolbox. First, you will want to make sure that you have your favorite text editor and have it configured to integrate with Chef. Next, since you are creating infrastructure code, you really must use a source code control system. I don't want to give anything away on that topic yet but … Git Git Git! Then, you will want to add a hypervisor to your workstation to deploy test nodes. Finally, you'll want to add Vagrant to manage those test nodes. So, let's take a look at these finishing touches for your workstation.
Everyone has their favorite text editor. For a lot of people that editor is Sublime Text, but there are many other choices available. A lot of old schoolers still prefer to use a flavor of the vi editor. Many OS X users swear by Mac-only TextMate. And now there's a new kid on the block from the makers of GitHub:
Atom 1.0. Sublime Text and Atom are both available for multiple platforms, so whatever OS is your poison, they have a version for you.
I would recommend that, if you don't already use an editor that has a "project space" feature,you switch to one that does. This feature allows you to quickly and easily switch to and edit different files within a project without the delay of the "save current then open new" workflow needed in an editor that does not have the project space feature.
The project space feature is one of the many reasons I usually recommend Sublime Text, but this feature is also present in other editors such as the new Atom 1.0.
To keep things current, I will use the Atom 1.0 editor in the remainder of this book's examples.
Once you have decided on the editor you plan to use for your Chef workstation and have that editor installed and working, there is one more step. You will need to set the configuration to allow you to integrate your editor with the Chef command-line tools. The best way to do that is to edit your
.chef/knife.rb file and add a line to set the editor you want to use with your knife commands. (You could also add an
export EDITOR= command to your shell's
Here are some sample
knife[:editor] = '/Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl -w'
knife[:editor] = '/Applications/Atom.app/Contents/MacOS/Atom -w'
On Mac, the Atom editor has a built-in feature to create symbolic links in the
/usr/local/bin folder. With the Atom editor running, open the Atom menu and select Install Shell Commands from the submenu. This will create two new links for you. If you do this before updating your
knife.rb file, the
knife[:editor] entry simply becomes this:
knife[:editor] = 'atom -w'
When you're using a Windows OS for your workstation, you will have to take greater care in formatting your
knife[:editor] line to escape the slash characters in the path. A convenient way to do that is to use single quotes around the full value and use double quotes for the path. For Sublime Text, it might look something like this:
knife[:editor] = '"C:\Program Files\Sublime Text 2\sublime_text.exe" --wait'
You can test your editor setting by issuing a knife command that requires using the editor. For example, try the command:
knife client create editor-tester
If your editor integration is configured correctly, you will see your editor open up and show you some code. Save the code and exit the editor, and you should see the private key value print out. (This key is not important at this time because we are just testing the editor, and will immediately delete the client.)
If your editor opens, you are done. If you get an error that says Please set EDITOR environment variable, then your knife[:editor] value is incorrect. There is probably a typo in the path or some other error. Double-check the value and try again until you successfully launch the editor with your knife client create command.
Once you have validated the integration with your editor, you can delete the test client with the following command:
knife client delete editor-tester
Now you are ready to edit your infrastructure code! When you are creating amazing infrastructure code, you are going to want to make sure that it gets preserved and to ensure that happens you going to want to use a source control tool. Say hello to my little friend, Git.
Git, initially designed and developed by Linus Torvalds for Linux kernel development back in 2005, is now the most widely used version control system for developers everywhere. It's the number-one choice due to its speed, data integrity, and its distributed workflow model. Git is certainly the primary choice as the version control system to be used when developing infrastructure code with Chef.
I will be emphasizing the use of Git throughout the examples in the rest of the book. Git is included with Mac OS X, but it will need to be installed on Windows and Ubuntu workstations. I recommend you to use the GitHub desktop installer where available (Windows and OS X) and allow it to deploy the command-line tools as part of the installation. This installation of Git is easy and will be left to the reader to execute.
There are several virtualization choices you can use for your Chef testing. The overall goal is to have a system that you can use to deploy a temporary Chef node to test your infrastructure code. One of the best ways to accomplish this is to install a local virtualization tool on your workstation. Among the choices for a local hypervisor are:
VMware Workstation or Fusion
One of the best choices for this kind of Chef testing is VirtualBox. The reasons include its functionality, its ease of use, and its free price tag. Either of the VMware solutions will cost you not only for the hypervisor layer, but also for the integration plugins we'll use with Vagrant to manage our test nodes.
Visit the VirtualBox downloads page at https://www.virtualbox.org/wiki/Downloads to download the installer. Once downloaded, it is an easy matter to install it, so I'll leave that to the reader to take care of.
Last, but certainly not least, is Vagrant. Vagrant is another item I consider to be one of the key pieces of the Chef testing puzzle. It is the "glue" between Chef and the hypervisor that deploys the test nodes during testing. You will want to download and install Vagrant on your workstation. You can find the downloaders at https://www.vagrantup.com/downloads.html. Vagrant is also an easy install, so I'll leave that to the reader to handle.
I'll go into using Vagrant with VitualBox (and Docker) in some detail in a later chapter.
You can download Sublime Text 2 from http://www.sublimetext.com/
If you want to try the long-running beta of Sublime Text 3, it can be found at http://www.sublimetext.com/3
If you are looking to try Atom 1.0, you can get it at https://atom.io/
The OS X editor, TextMate, can be had at https://macromates.com
Learn more about GitHub at https://github.com/
Learn about and download VirtualBox from https://www.virtualbox.org/
Learn more about Docker at https://www.docker.com/
To get VMware Workstation go to https://www.vmware.com/products/workstation
Or for Mac OS X, get VMware Fusion at https://www.vmware.com/products/fusion
One often overlooked tool that is installed with the ChefDK is
chef-apply. This tool basically allows you to execute chef to converge a single recipe on your local machine. There is no Chef server involved. Everything is local.
Note that chef-apply is not a tool for the deployment of production nodes. It's just a quick and easy way to use Chef to configure your local system.
One use case for chef-apply is to learn what a recipe does. You can use
chef-apply to run a recipe in a "test only" mode by using the
-W or --why-run parameter. This will converge the recipe, showing you everything that would happen with the recipe without making actual changes to your system.
You can also execute some one-liner scripts. You can easily use
chef-apply in a scripting-like manner to install software. For example, if you need to use Git on your system and you find it's not installed. Just issue the command:
sudo chef-apply -e "package 'git'"
This will check to see if Git is currently installed. If it is, then the command will just inform you that that is the case and exit. If Git is not installed, then it will go about using the correct installer (aka Provider) to obtain and install the package.
Testing this on my Ubuntu workstation, you can see that Git was not installed initially. Then, by using the
chef-apply command, the
git package is installed. Then to confirm that nothing happens if you try to use
chef-apply to install a package that is already installed, I ran the same
chef-apply again to show what happens:
How about creating a "workstation" recipe to set up all the "missing" items from a new ChefDK workstations setup? You create the recipe that defines the state of having your editor installed and integrated with Chef, defines the state of having
git installed, and defines the state of having your custom OS X "tree" script created in
/usr/local/bin. You create this recipe once, and you (and anyone in your organization) can quickly complete their workstation setup by using chef-apply with your workstation recipe.
The link to get the details of an interesting use case for chef-apply posted by Chris Doherty on chef.io is https://www.chef.io/blog/2015/01/16/convert-your-bash-scripts-with-chef-apply/.
The link to the chef.io documentation on chef-apply is https://docs.chef.io/ctl_chef_apply.html.
In this chapter, we've taken a quick stroll down memory lane and reviewed some Chef basics so that those concepts are fresh and easy to mentally access during the rest of the journey through this book. Next up, we took a deeper dive into the Chef Development Kit and learned how to download, install, and configure it for our Chef workstation. We've exposed the small gap in the ChefDK installer when used on Mac OS X and Linux systems, which is the setup of the shell environment and learned the simple way (provided by the new Chef tool) to fill that gap. We also explored the setup and configuration of your very own on-premise Chef server, including creating your admin user and the first organization. After this, we took a look at setting up our Chef repo, giving a BIG WARNING about the starter kit. And then we considered what was still missing from a well-rounded Chef workstation. That will be our favorite editor, a version control system, a virtualization layer to simplify our testing flow, and Vagrant to glue things together. Finally, as an added bonus, we talked about the ChefDK tool chef-apply.
Wow! That was a lot of ground for a single chapter, but I hope that you were able to stick with me till the end. This chapter is a great foundation for the Chef knowledge transfer coming your way in the chapters ahead.
So without any further delay, let's charge ahead into Chapter 2, Knife Tooling and Plugins, and carve things up with some Swiss Army knife skills.