Chapter 13. (Ab)Using Chef

We've explored various aspects of the Chef ecosystem and we've tried to get our hands dirty with the exploration of Chef's wonderful API as well. However, as a common practice, nothing is good enough on its own, and the real benefits of a particular technology or tool can be only realized once we've used it along with other tool sets. This is true for Chef too. Chef on its own is a wonderful piece of software; however, once we start integrating it along with other tools, we realize the true benefits of Chef. In this chapter, we'll look at a few such integrations and also see how we can extend Chef by mashing together various different APIs with Chef's API. This chapter is going to be very code intensive and you'll be introduced to a few APIs outside Chef. These APIs can change at any point in time and if you plan on using the code given in this chapter directly, ensure that you have read through the API documentation. If the API provider has introduced any changes...

The dynamic server list in Capistrano using Chef

Capistrano is a remote server automation tool. It can be used to execute an arbitrary set of tasks on remote servers. It's primarily used for the purpose of remote deployments. As per their official documentation, Capistrano can be used to:

• Reliably deploy to any number of machines simultaneously, in the sequence of a rolling set

• Automate audits of any number of machines

• Script arbitrary workflows over SSH

Although Chef also provides a deploy resource, I personally prefer push-based deployment solutions such as Capistrano, as they provide more control and I can easily hook them up with a release management system to provide visibility.

Capistrano considers a list of servers as a role and it can either deploy to an individual host or an entire fleet. Usually, this list of servers is maintained in configuration files, or supplied as a command line option during execution. This works pretty well for environments where the total number of servers isn...

Capistrano style deployments using Chef's "deploy" resource

Capistrano is a very popular push-based deployment tool used extensively in the world of Ruby on Rails applications. However, once you've moved your infrastructure to the cloud, in addition to the automatic provisioning of machines, you also need to be interested in ensuring that once the machines are up and running, they come up with the right version of your application code. Now, the provisioning of machines is the domain of Chef, while application deployment is classically a task belonging to the realm of Capistrano. However, now with the "deploy" resource of Chef, you can deploy your favorite Ruby on Rails application, just as you would do with Capistrano.

The deploy resource is meant to provide the facility of the deploy and deploy:migration tasks in Capistrano.

The syntax of the deploy resource is as follows:

deploy "name" do
  attribute "value"
  ...
  callback do
     # callback, include release_path or new_resource
  end...

Extending Chef to bootstrap distributed systems

Most configuration management systems such as Chef, Puppet, CFEngine, Ansible, and so on, operate at a node level, and any configuration change that is required to be made is applied only upon the convergence runs that happen at the scheduled intervals and aren't event-based. For example, say you've a setup comprising of a load balancer (say HAProxy) and web servers. Now, you are running a website and you want to ensure that as soon as the traffic spikes up, you should be able to provision a new web server.

You've written the HAProxy cookbook so that it searches for nodes of type webserver, populates its config, and reloads the HAProxy process. You've also written the Chef code to bring up the web server; however, even once the server is up and running, your load balancer has no knowledge of this and either you have to manually trigger the chef-client run, or if you are running a chef-client as a daemon or cron job, you have to wait for the...

Test-driven development with Chef

As we are trying to specify our infrastructure as code, it would be prudent of us to take some good stuff from devs practices and incorporate them into our coding practices. The following figure illustrates a few such ideas:

Ops:

• TDD: Test-Driven Development

• CI: Continuous Integration

• CD: Continuous Delivery/Deployment

Development without TDD

The usual practice followed by operations people can be understood from this flow chart:

Development with TDD

With TDD practices in use, the following is how the development cycle looks:

With TDD, the tests are an integral part of the development phase and either tests are written even before the code is written, or they are written alongside the code. So, whether you are building a new feature or fixing a bug, you'll always be writing test cases and running them continuously to ensure that things are behaving as intended. This is a habit that needs to be cultivated, and the following are the steps you need to take to follow...

Types of tests

The following are the two main categories of tests that are written by developers:

Unit tests:

These tests are meant to test an individual component and such testing is also known as "component testing". Ideally, each test case is independent of the other. It's written to ensure that the code meets its design and behaves as intended.

Unit tests are usually written before the code is written. When the tests pass, the code is considered to be complete. In case the test fails, it's an indicator of a bug in the code or test itself.

The following are the key points related to unit tests:

1. Tests only a single component.
Does not require other components.

2. Should be fast.

Integration tests:

Integration testing is a phase in software testing where individual software modules are combined and tested as a collective group. During the integration test, input modules that have already been unit tested are taken as input, grouped into larger aggregates, and the tests defined in the integration test...

Summary

In this chapter, we saw a few practical uses of Chef. We learnt how we can extend it to manage deployments using a dynamic server list via Capistrano. Next, we went about learning how to deploy applications using the deploy resource provided by Chef. This resource is extremely useful as it allows Chef to act as both a configuration management system and a deployment system. We also saw how we can use Chef to provision clusters and build distributed systems. Finally, we learnt about Test-driven development using Chefspec and minitest-chef-handler.