Puppet Platform Parts and Functions

So far, we have discussed Puppet as a language, but in this chapter and the following chapters, we will start to focus on Puppet as a platform and the infrastructure and components of the platform.

In Figure 10.1, the full architecture of services involved in Puppet Server and the Puppet client, to be discussed in this chapter, is shown. These services focus on how Puppet code is enforced on servers:

Figure 10.1 – Puppet server and client components

We will start by highlighting that we do not run through installation methods in this book. There are several open source projects to base automation on for open source Puppet and Puppet Enterprise; throughout this book, we have used the peadznd pecdm modules as the most automated mechanism for installing Puppet Editor (PE). As components are discussed, it will be noted how the versioning of Puppet packages can differ, and we’ll look at some related install...

Technical requirements

Clone the control repo from https://github.com/puppetlabs/control-repo to your GitHub account in a repo called controlrepo-chapter10.

Build a large cluster with three compilers and three clients by downloading the params.json file from https://github.com/PacktPublishing/Puppet-8-for-DevOps-Engineers/blob/main/ch10/params.json and update it with the location of your control repo and your SSH key for the control repo. Then, run the following command from your pecdm directory:

bolt --verbose plan run pecdm::provision –-params @params.json

Puppet platform installation and versioning

This book makes the choice not to go into the methods of installing Puppet; there is little to add to the installation instructions for open source, documented at https://puppet.com/docs/puppet/latest/server/install_from_packages.html, and any further choice of automation will depend heavily on the use case of your organization and available tooling and product sets you want to integrate with.

For open source Puppet, there are a number of projects automating the deployment, configuration, and integration of Puppet, such as example42’s psick (https://github.com/example42/psick) or the Foreman project (https://github.com/theforeman/foreman-installer), which has a specific module for installing Puppet Server (https://forge.puppet.com/modules/theforeman/puppet) that can be used even outside of Foreman to install Puppet. Dashboards similar to what has been provided by the PE setup can also be found in projects such as Puppetboard (https...

Puppet Server

In historic versions of Puppet, Ruby-based solutions such as WEBrick or Passenger were used for running the Puppet service, but in all modern versions of Puppet, to improve scaling and performance, Puppet Server is run as a Clojure and Ruby application on a Java Virtual Machine (JVM). Puppet Server has a number of related services that share state and route requests between them. These services run inside a single JVM process, using the Trapperkeeper service framework, which is a Clojure framework for hosting long-running applications.

Puppet Server is installed via the puppetserver package in open source Puppet and the pe-puppetserver package in PE. This will create a system service of the same name and configuration files that, by default, will be placed in /etc/puppetlabs/puppetserver/conf.d in Human-Optimized Config Object Notation (HOCON) format.

Note

The Puppet hocon module is the best way to automate the management of HOCON files (https://forge.puppet...

The Puppet agent-to-server lifecycle

This section will look at how the Puppet agent makes requests to the Puppet server components we have run through and how it secures its communications before requesting configuration to enforce from the Puppet servers. It should be noted the Puppet servers themselves also contain Puppet agents.

The installation of Puppet agents is detailed at https://puppet.com/docs/puppet/latest/install_agents.html#install_agents for open source and https://puppet.com/docs/pe/2021.7/installing_agents.html#installing_agents for PE. Integrating this install with your server deployment workflow and ensuring the necessary configuration is placed at /etc/puppetlab/puppet.conf is critical for automation.

Note

The puppet_conf module provides tasks to manage Puppet configuration files (https://forge.puppet.com/modules/puppetlabs/puppet_conf).

Most of the settings will depend on your environment setup, but for most environments, the defaults will be taken with...

PuppetDB and PostgreSQL

PuppetDB allows for the collection of Puppet data and advanced features such as exported resources. In open source Puppet, it is entirely optional, while PE installs PuppetDB by default. The following is kept by PuppetDB:

• The last facts from the nodes
• The last catalog compiled for each node
• 14 days (default) of event reports for each node
• Exported resources

PuppetDB is a Clojure frontend application running on a JVM, using PostgreSQL as a backend database. This common architecture is where the backend database just provides the tables, and the frontend database contains the application objects, giving some key advantages compared to a single database. It eases the updating process of PuppetDB since the actual data can be left in the backend table, and it also allows great scalability—as we will see in the last section of this chapter, Scaling with compilers—where PuppetDB can be scaled horizontally by running PuppetDB...

Scaling with compilers

1. The review of Puppet platform components so far assumes that all components are present on a single primary server. However, as the number of managed nodes increases, it becomes impractical for a single server to handle them. According to Puppet’s documentation, a primary server can manage up to 2,500 clients on default settings. To handle the growing number of nodes, Puppet uses horizontal scaling, which involves using Puppet compile servers. In Figure 10.4, a subset of primary services is shown to be moved onto compile servers. These servers can be configured in a round-robin selection in the client’s configuration file or placed behind a load balancer. This enables multiple nodes to work together to compile catalogs while still allowing certain services to run on the primary server. According to Puppet’s documentation, with the default compiler settings, up to 3,000 clients can be served per compiler:

...

Summary

In this chapter, we learned about the services provided by the Puppet server and how the embedded web server attaches handlers to mount points, which can then be requested via HTTP requests to endpoints.

It was shown that the /puppet endpoint provides services for configuration requests and how indirectors or environments can request specific components such as requesting a catalog from a server. The /puppet-ca endpoint similarly used indirectors to allow for requests to the CA. The /puppet-admin-api endpoint was then shown to allow for clearing the environment cache and JRuby instances as more advanced administrative actions.

It was then shown how Puppet creates a CA server with a root CA and an intermediate CA to sign or can run in legacy mode with a single combined CA. The options for using externally provided certificates were then discussed. The process of signing certificate requests was shown, with the puppetserver certificate command for managing certificates...