Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Learning Puppet Security
Learning Puppet Security

Learning Puppet Security: Secure your IT environments with the powerful security tools of Puppet

eBook
$32.39 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Learning Puppet Security

Chapter 1. Puppet as a Security Tool

Imagine you're sitting at home one day after a long day of work. Suddenly, you get a phone call that a new security vulnerability was found and all 300 of your servers will need to be patched. How would you handle it?

With Puppet, finding which one of your servers was vulnerable would be an easier task than doing so by hand. Furthermore, with a little additional work, you could ensure that every one of your servers is running a newer nonvulnerable version of the Puppet package.

In this chapter, we will touch on the following concepts:

  • What is Puppet?
  • Declarative versus imperative systems
  • The Puppet client-server model
  • Other components of the Puppet ecosystem used for security
  • Installing Puppet
  • How Puppet fits into a security role

Once this is complete, we will build the environment we'll use to run examples in this book and then run our first example.

Much of the information in this chapter is presented as a guide to what we will accomplish later on in this book.

What is Puppet?

The Puppet Labs website describes open source Puppet as follows:

Open source Puppet is a configuration management system that allows you to define the state of your IT infrastructure, then automatically enforces the correct state.

What does this mean, though?

Puppet is a configuration management tool. A configuration management tool is a tool that helps the user specify how to put a computer system in a desired state. Other popular tools that are considered as configuration management tools are Chef and CFEngine. There are also a variety of other options that are gaining a user base, such as Bcfg2 and Salt.

Chef is another configuration management tool. It uses pure Ruby Domain-specific Language (DSL) similar to Puppet. We'll cover what a domain-specific language is shortly. This difference allows you to write the desired state of your systems in Ruby. Doing so allows one to use the features of the Ruby language, such as iteration, to solve some problems that can be more difficult to solve in the stricter domain-specific language of Puppet. However, it also requires you to be familiar with Ruby programming. More information on Chef can be found at http://www.getchef.com.

CFEngine is the oldest of the three main tools mentioned here. It has grown into a very mature platform as it has expanded. Puppet was created out of some frustrations with CFEngine. One example of this is that the CFEngine community was formally quite closed, that is, they didn't accept user input on design decisions. Additionally, there was a focus in CFEngine on the methods used to configure systems. Puppet aimed to be a more open system that was community-focused. It also aimed to make the resource the primary actor, and relied on the engine to make necessary changes instead of relying on scripts in most cases.

Note

Many of these issues were addressed in CFEngine 3, and it retains a very large user base. More information on CFEngine can be found at http://www.cfengine.com.

Bcfg2 and Salt are both tools that are gaining a user base. Both written in Python, they provide another option for a user who may be more familiar with Python than other languages. Information on these tools, as well as a list of others that are available, can be found at https://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software.

Configuration management tools were brought about by a desire to make system administration work repeatable, as well as automate it.

In the early days of system administration, it was very common for an administrator to install the operating system needed as well as install any necessary software packages. When systems were simple and few in number, this was a low effort way of managing them.

As systems grew more complex and greater numbers of them were installed, this became much more difficult. Troubleshooting an application as it began to run on multiple systems also became difficult. The difference in software versions on installed nodes and other configuration differences created inconsistencies in the behavior of multiple systems that were running the same application. Installation manuals, run books, and other forms of documentation were often deployed to try to remedy this, but it was clear that we needed a better way.

As time moved on, system administrators realized that they needed a better way to manage their systems. A variety of methods were born, but many of them were home built. They often used SSH to manage remote hosts. I also built several such systems at various places before coming across Puppet.

Puppet sought to ease the pain and shortcomings of the early days. It was a big change from anything that was present at the time. A large part of this was because of its declarative nature.

Declarative versus imperative approaches

At the core of Puppet is software that allows you to specify the state of the system and let Puppet get the system there. It differs from many of the other products in the configuration management space due to its declarative nature.

In a declarative system, we model the desired state of the resources (things being managed).

Declarative systems have the following properties:

  • Desired state is expressed, not steps used to get there
  • Usually no flow control, such as loops; it may contain conditional statements
  • Actions are normally idempotent
  • Dependency is usually explicitly declared

Tip

The concept of actions being idempotent is a very important one in Puppet. It means that actions can be repeated without causing unnecessary side effects. For example, removing a user is idempotent, because removing it when it doesn't exist causes no side effects. Running a script that increments to the next user ID and creates a user may not be idempotent, because the user ID might change.

Imperative systems, on the other hand, use algorithms and steps to express their desired state. Most traditional programming languages, such as C and Java, are considered imperative. Imperative systems have the following properties:

  • They use algorithms to describe the steps to the solution
  • They use flow control to add conditionals and loops
  • Actions may not be idempotent
  • Dependency is normally executed by ordering

In Puppet, which is declarative, the users can describe how they want the system to look in the end, and leave the implementation details of how to get there up to the types and providers within Puppet. Puppet uses types, which represent resources, such as files or packages. Each type can optionally be implemented by one or more provider.

Types provide the core functionality available in Puppet. The type system is extensible, and additional types can be added using pure Ruby code. Later on in this chapter, we'll use the file and package types in our example.

Providers include the code for the type that actually does the low level implementation of a resource. Many types have several providers that implement their functionality in different ways. An example of this is the package type. It has providers for RPM, Yum, dpkg, Windows using MSI, and several others. While it is not a requirement that all types have multiple providers, it is not uncommon to see them, especially for resources that have different implementation details across operating systems.

This system of types and providers isolates the user from having to have specific knowledge of how a given task is done. This allows them to focus on how the system should be configured, and leave specific implementation details, such as how to put it in that state, to Puppet.

A few tools, such as Chef, actually use more of a hybrid approach. They can be used in a declarative state, but also allow the use of loops and other flow control structures that are imperative. Puppet is slowly starting to gain some support for this in their new future parser, however these are experimental and advanced features at this point.

While the declarative approach may have a larger learning curve, especially around dependency management, many sysadmins find it a much better fit with their way of thinking once they learn how it works.

The Puppet client-server model

Puppet uses a client-server model in the most common configurations. In this mode, one or more systems, called Puppet Masters, contain files called manifests. Manifests are code written in the Puppet DSL. A DSL is a language designed to be used for a specific application. In this case, the language is used to describe the desired state of a system. This differs from more general purpose languages, such as C and Ruby, in that it contains specialized constructs for the problem being solved. In this case, the resources in the language are specific to the configuration management domain.

Manifests contain the classes and resources which Puppet uses to describe the state of the system. They also contain declarations of the dependencies between these resources.

Classes are often bundled up into modules which package up classes into reusable chunks that can be managed separately. As your system becomes more complicated, using modules helps you manage each subsystem independently of the others.

The client systems contain the Puppet agent, which is the component that communicates with the master. At specified run intervals (30 minutes by default), the agent will run and the following actions will take place:

  1. Custom plugins, such as facts, types, and providers, are sent to the client, if configured.
  2. The client collects facts and sends them to the master.
  3. The master compiles a catalog and sends it to the client.
  4. The client processes the catalog sent by the master.
  5. The client sends the reporting data to the master, if configured.

The catalog, sent to the client by the master, contains a compiled state of the system resources of the client. The client then applies this information using types and providers to bring the system into the desired state. The following illustration shows how data flows between the components:

The Puppet client-server model

It is also possible to run Puppet in a masterless mode. In this mode, the Puppet manifests and other needed components, such as custom facts, types, and providers, are distributed to each system using an out of band method, such as scp or rsync. Puppet is then applied on the local node using cron or some other tool.

cron has the advantage of not requiring the server setup with open ports that the master-based setup has. In some organizations, this makes it easier to get past information security teams. However, many of the reporting and other benefits we will explore in this book are less effective when run in this fashion. The book Puppet 3: Beginners Guide, John Arundel, Packt Publishing, has a good amount of information about such a masterless setup.

Other Puppet components

Puppet has a number of other components that form part of the Puppet ecosystem, which are worth exploring due to their use as security tools. The specific components we are going to explore here include PuppetDB and Hiera.

PuppetDB

PuppetDB is an application used to store information on the Puppet infrastructure. Released in 2012, PuppetDB solved performance issues present in the older storeconfigs method that stored information about Puppet runs.

PuppetDB allows you to store facts, catalogs, reports, and resource information (via exported resources). Mining this data, using one of the reporting APIs, is an easy and powerful way to get a view of your infrastructure. More information on PuppetDB will be presented in Chapter 3, Puppet for Compliance, as well as Chapter 4, Security Reporting with Puppet.

Hiera

Hiera was a new feature introduced in Puppet 3. It is a hierarchal data store, which helps to keep information about your environment. This allows you to separate data about the environment from code that acts on the environment. By doing so, you can apply separate security policies to the code that drives the environment and data about the systems.

Before Hiera, it was not uncommon to see large sections of Puppet code dedicated to maintaining sites or installation of specific information on the systems under management. This area was often difficult to maintain if the ability to override parameters using many different factors was needed.

By adding a hierarchy that can depend on any facts, it becomes much easier to store the data needed for the systems under management. A model of most specific to least specific can then be applied, which makes it much easier to override the default data at a site, environment, or system level.

For example, let's say you had a set of development environments where a certain group of development accounts needed to get created, and SSH access to those accounts was granted. However, these accounts and the access granted should only exist in the development machines, and not in production. Without Hiera, there would likely be site-specific information in the modules to manage the SSH configuration, and perhaps in the user creation module to manage the users. Using Hiera, we can add a fact for the type of system (production or development) and store which users get created there, or have access. This moves the list of users with access to the system out of the code itself, and into a data file.

As our examples get more complicated later in this book, we will explore using Hiera to store some system data.

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Installing and configuring Puppet

Puppet can be installed in a variety of ways. Since this book is focused on the security-related aspects of Puppet and is not a beginner's guide, we will cover the most common way it is installed on our target system. There are many good reference books available for more in-depth information on installing Puppet, including Puppet 3: Beginner's Guide, John Arundel, Packt Publishing.

In our examples, we'll be using CentOS 6 as our operating system. If you are using a different operating system and following along on your own, please see the installation instructions for your operating system at http://www.puppetlabs.com, or follow along using Vagrant as outlined later.

Since we will be using Vagrant for our examples, the base box we are using already has the Puppet repository installed on it as well as the Puppet agent. We'll provide instructions for the installation of these elements for those who wish to use CentOS without using Vagrant.

Installing the Puppet Labs Yum repository

The currently recommended way to install Puppet on CentOS machines is to use the Puppet Labs Yum repository. This repository, which can be found at https://yum.puppetlabs.com, contains all the Puppet Labs software as well as the dependencies required to install them, such as several Ruby gems not present in the main CentOS repository. On installation, Ruby and these dependencies will also be installed.

Adding this repository is relatively simple. Execute the following command as a root (or using sudo, as shown here):

sudo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

After running this command, you will see an output similar to this:

Retrieving https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
Preparing...                ########################################### [100%]
   1:puppetlabs-release     ########################################### [100%]

Once this is complete, you're done! The Puppet Labs repository is added and we can use it to install the current version of any of the Puppet Labs products.

Installing the Puppet Master

The next step is to install the Puppet Master. As mentioned earlier, this system acts as the controller that all of your client agents will then use to communicate with to receive catalog information. This package is normally installed on only a few systems that act as servers for configuration management information.

Installing the master with the repository is as easy as executing the following command:

sudo yum -y install puppet-server

This will instruct yum to install the Puppet server without confirmation. The output will be as follows:

Installing the Puppet Master

Installing the Puppet agent

On all the systems that we wish to manage by using Puppet, we'll need to install the Puppet agent. This agent is a piece of software that is responsible for communicating with the master and applying changes.

Installing the Puppet agent is very easy and similar to installing the master in the preceding section. You simply run the following:

sudo yum -y install puppet

After this is complete, you'll see that the the Puppet agent is installed on the local machine and is ready to talk to the master.

Configuring Puppet

Now that we have a perfectly working Puppet Master, we need to configure it. Installation of the packages will include a base level configuration. There are some changes we will want to make to the base Puppet configuration to enable some features that we'll use in the future. As we go through this book, we'll make changes to these files several times.

The main configuration files in use by Puppet are present in the /etc/puppet directory.

In this directory, there are a number of configuration files that control how Puppet behaves. Information on these files can be found at https://docs.puppetlabs.com/puppet/3.7/reference/config_about_settings.html. For now, we only need to concern ourselves with the Puppet configuration file.

Open the /etc/puppet/puppet.conf file with your favorite editor (make sure that you use sudo) and edit it to look similar to the following:

[main]
    # The Puppet log directory.
    # The default value is '$vardir/log'.
    logdir = /var/log/puppet

    # Where Puppet PID files are kept.
    # The default value is '$vardir/run'.
    rundir = /var/run/puppet

    # Where SSL certificates are kept.
    # The default value is '$confdir/ssl'.
    ssldir = $vardir/ssl

[agent]
    # The file in which puppetd stores a list of the classes
    # associated with the retrieved configuratiion.  Can be loaded in
    # the separate ``puppet`` executable using the ``--loadclasses``
    # option.
    # The default value is '$confdir/classes.txt'.
    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An
    # extension indicating the cache format is added automatically.
    # The default value is '$confdir/localconfig'.
    localconfig = $vardir/localconfig
    report = true
    pluginsync = true
[master]
    reports = store

We've made a handful of changes to the file from the default version and will cover them here.

The first change is adding the report = true section to the agent configuration section. This will cause clients to send reports containing information about the Puppet run. We'll use these reports for later analysis in Chapter 4, Security Reporting with Puppet.

The second change is to add pluginsync = true to the agent section. While this has become the default in the more recent versions of Puppet, it does not hurt to add it in. This causes the clients to sync custom facts, providers, and other Puppet libraries from the master. We will see how this is used in later chapters.

The final change we have made is to add the master section and add reports = store. This causes the master to save reports to the local filesystem on the Puppet Master. We'll use this later to do analysis of our Puppet runs for security-related purposes.

Puppet services

Both the Puppet Master and the agent are usually run as services. This allows the agent to check its run frequency and apply any changes. We've not explicitly started the services here, although we'll need to start the master in order to use it from our agent. To do this, we run the following command:

sudo service puppetmaster start

In order for the Puppet Master to start at boot, we'll also issue the following command to enable it to autostart:

sudo chkconfig puppetmaster on

It's pretty common to use Puppet to manage Puppet, and in a later chapter, we'll do this to show how we can use Puppet to secure the Puppet Master.

Note

It's worth noting that Puppet running with a default web server configuration will not scale beyond a few dozen hosts. Scaling Puppet is outside the scope of this book. More information on scaling Puppet can be found at http://docs.puppetlabs.com/guides/scaling.html.

Preparing the environment for examples

As mentioned in the preface, we're going to use Vagrant to run our examples. In case you missed it, Vagrant is a tool that helps you automate the creation of virtual machines for testing. In this case, it's a great tool for us to use to quickly build-out our build and example environments.

We'll be using CentOS 6 in these examples, but most of them should run without much modification on other platforms. You will need to adjust the package names and perhaps configure the filenames for other operating systems. Many community modules, which we will explore in later chapters, support multiple flavors of Linux as well as other Unix-like systems. The powerful descriptive language of Puppet makes this easy to do.

While the use of Vagrant is not required, it will help us to maintain a clean environment for each of the examples we run, and will also ease the creation of virtual machines. If you choose not to use Vagrant for this, you can still run the examples using the manifests and modules provided with the source accompanying this book.

Installing Vagrant and VirtualBox

In order for us to use Vagrant, we must first install it. To do this, we need to install the required dependencies followed by Vagrant itself. We'll be using VirtualBox to host the virtual machines in these examples, since it is the most supported virtual machine provider.

VirtualBox can be downloaded from http://www.virtualbox.org. On this site, you will find packages for installing a variety of operating systems. You simply need to pick the package for your chosen operating system and install it using the instructions found on the site.

Once we have VirtualBox installed, we can approach installing Vagrant. Vagrant has several methods of installation. These methods include OS packages for Linux, as well as installers for OS X and Windows. Older versions of Vagrant supported installation via the Ruby gem utility, but this has been removed in later versions.

Vagrant can be found at http://www.vagrantup.com. Once you're there, you can download the package or installer for your OS. Once downloaded, you can install the package using your operating system's package manager, or by executing the downloaded package. In Windows and OS X, this is sufficient to have a working installation of Vagrant.

More in-depth installation instructions can be found on the Documentation tab on the Vagrant website; however, the package or installer will do most of the work.

It is worth noting that if you are using Windows, you will perform most of the work we're doing in a command shell on the DOS command box. However, if you use a local editor, you should be able to follow along with no issues.

Creating our first Vagrantfile

Now that we have Vagrant installed, we'll create our first Vagrant configuration. Vagrant uses a file called Vagrantfile to control its operation.

First, we start by creating a directory for our project. In this case, we'll call it puppetbook. We'll end up building on this setup in later chapters to automate configuration of our examples. This will allow us to focus on the Puppet tasks, and not so much on getting our test systems into the desired state.

Inside this directory, we'll create a directory called master_manifests. The purpose of this directory is to hold the Puppet manifests that we'll use to provision the base VM.

We'll be using the Puppet provisioner to do our work. This is one of a handful of methods you can use to provision a Vagrant virtual machine. Using this provisioner, we'll write a Puppet manifest that will describe the desired state of our machine. Vagrant will then use this manifest to run Puppet locally and configure the system.

Next, we'll create a Vagrantfile. In your favorite editor, go ahead and open Vagrantfile. Add the following contents. We'll cover what each one does in a moment:

Vagrant.configure(2) do |config|
  config.vm.define :puppetmaster do |master|

    master.vm.box = "centos65-x64-puppet"
    master.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-puppet.box"
    master.vm.hostname = "puppet.book.local"
    master.vm.network "private_network", ip: "10.78.78.30", netmask: "255.255.255.0"

    master.vm.provision "shell", inline: "yum –y update puppet"

    master.vm.provision "puppet" do |puppet|
      puppet.manifests_path = "master_manifests"
      puppet.manifest_file = "init.pp"
    end

  end
end

Note

It's possible that by the time you read this, the Vagrant box referenced in the preceding code will be deprecated. This book was written using the Puppet Labs CentOS 6 machine images. You can go to http://puppet-vagrant-boxes.puppetlabs.com/ and find a replacement. You want a CentOS 6 x86_64 box with Puppet (called plain there) and VirtualBox addons.

Go ahead and save the file. We'll cover what each file does here:

Vagrant.configure(2) do |config|

This line sets up Vagrant using configuration version 2. It uses Ruby blocks to create a Vagrant configuration with the config variable:

config.vm.define :puppetmaster do |master|

This line defines a virtual machine called puppetmaster. Vagrant supports multimachine setups, which is a feature we'll use later on in the book. For now, we'll define a single machine. Much like the preceding code, we use a block called master:

master.vm.box = "centos65-x64-puppet"

This defines the box we'll use for our Puppet Master. It is a symbolic name, but it makes sense to name it according to what it is. If you refer to the same box later, it'll use the same base and not download the box files an additional time:

master.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-puppet.box"

This defines the URL we'll download our box file from. In this case, we're grabbing it from the hosted Puppet Vagrant boxes on Puppet Labs. We could get a box from any number of other places, but the Puppet Labs boxes come with the Puppet agent preinstalled and the Puppet repository is already available and ready for use. If you wish to explore other box options, there is a directory of them available at http://www.vagrantcloud.com:

master.vm.hostname = "puppet.book.local"

This command simply sets the host name of our machine. It is important for the master as it influences the certificate name that gets created at installation:

master.vm.network "private_network", ip: "10.78.78.30", netmask: "255.255.255.0"

This line creates a private network for our virtual machines to use. We assign it the IP address 10.78.78.30/24 (78 is PU on a phone dial pad):

master.vm.provision "shell", inline: "yum –y update puppet

"Wait," you say, "I thought we were using the Puppet provisioner?"

As it turns out, the Puppet Labs base box comes with Puppet 3.4 installed. The current version we wish to use in this book is 3.7.3. We use the yum statement to upgrade Puppet before the provisioner starts. Otherwise, we get issues when the Puppet run updates the agent:

master.vm.provision "puppet" do |puppet|

Here, we tell Vagrant we're going to use the Puppet provisioner, and open a block called puppet to do so:

puppet.manifests_path = "master_manifests"

Here, we give the path to the manifest directory. This is relative to the path that the Vagrantfile is in. As you can recall, we created this directory earlier:

puppet.manifest_file = "init.pp"

We define the Puppet manifest to be called init.pp. This is the default name of a Puppet manifest. Vagrant defaults to default.pp if it's not specified:

end
end
end

These lines undo each of the preceding blocks and close out the file.

If we run Vagrant now, it will throw an error because it cannot find the init.pp file, so let's go ahead and create it inside the master_manifests directory. To save space, we'll call out each block and describe its function rather than giving the entire file and explaining it:

package { 'puppet-server':
  ensure => 'present',
}

The preceding resource declaration will install the Puppet Master. By specifying the ensure value of present, we make sure it's installed; however, we tell Puppet that we do not care about the version and do not wish to upgrade it:

file { '/etc/puppet/puppet.conf':
  ensure  => 'present',
  owner   => 'root',
  group   => 'root',
  mode    => '0644',
  source  => '/vagrant/master_manifests/files/puppet.conf',
  require => Package['puppet-server'],
}

The preceding resource declaration has a good amount more going on. Here, we're going to manage a file called /etc/puppet/puppet.conf. We ensure that it is present, then set the owner, group, and mode to set the values. Using the source parameter, we source the file from the local filesystem. Vagrant, by default, will mount the directory containing the Vagrantfile as /vagrant, so we can take advantage of that mount to get the file without otherwise copying it.

The last line here shows off the explicit dependency management of Puppet. We require that the puppet-server package is installed before we install the configuration file. This will ensure that the directory is created, and the package installation does not overwrite the configuration file:

service { 'puppetmaster':
  ensure  => 'running',
  require => File['/etc/puppet/puppet.conf'],
}

This last resource declaration ensures that the Puppet Master service is running. It depends on the configuration file being there.

In a real-world example, we're likely to use subscribe instead of require here. This would restart the service if the configuration file changed. However, since we're using the local Puppet provisioner and not running this code under a Puppet Master, this code will only be run once, so it is unnecessary to use subscribe.

We need one last file to make the system work. The file resource depends on a file called master_manifests/files/puppet.conf. We've covered the contents of this file in the Puppet installation section, so we will not repeat them here. You simply need to copy the file to the directory for the provisioner to use.

When we're done, the complete directory structure of this setup will look as follows:

.
├── Vagrantfile
└── master_manifests
    ├── files
    │   └── puppet.conf
    └── init.pp

Once we're set up, we're in a good position to run the examples that we'll present in this book. As these examples get more complex, we'll add the necessary data to this structure to add things such as client machines.

Puppet for security and compliance

Puppet is a perfect tool for security and compliance. So much security work involves ensuring that a given version of a service is on every server, or whether a user account exists or not.

Much of this work is also very tedious and repetitive. When work such as this is done across many servers, the likelihood that some of them will be different grows. These snowflakes, or systems that are unique and unlike other systems, can cause security issues or can be hard to troubleshoot.

On top of being able to maintain a system in a fixed state, we can use some Puppet resources, such as PuppetDB, to do some fairly in-depth reporting. Using custom facts, you can collect any information you wish to send to a central place. This can include things such as software versions, hardware configuration, and much more. By using this information, we can start to work toward creating a full configuration management and security platform.

Through Puppet, you will be able to centrally manage the major configuration aspects of all of your systems. Keeping this configuration in version control and treating it as code gives you all the benefits that developers have been able to enjoy for years. You'll quickly be able to see how the state of a system has evolved over time, as well as look where bugs might have been introduced and have caused security issues.

Additionally, there is an increasing movement to use Puppet for compliance and auditing. By demonstrating that Puppet is indeed running on a system and showing the manifests running on it, you can ensure that a system is in a given state. This information can be shown to auditors as documentation on how systems are configured.

Getting to the point of 100-percent coverage in system configuration using Puppet requires commitment and time. Using community modules, as we'll explore later, can lessen that work. However, the results of doing this are very high. Disaster recovery can be made simpler because systems can quickly be rebuilt. Installing the latest tripwire on all systems becomes as simple as updating the manifests and letting the systems check in. These benefits can make the job of a security professional much easier.

As we progress through this book, we will explore many of these abilities in-depth, but for now, let's look at a simple example we can use to learn some of the Puppet concepts and language.

Example – using Puppet to secure openssh

Now that we've got the system set up for our use, we can finally approach the main example for this chapter. In this example, we're going to use what has traditionally been one of the first things used to show off Puppet and install SSH. However, in this case, we're going to use a hardened configuration utilizing some options recommended by the security community.

The example of securing SSH is one that we will return to several times in this book as we expand upon our configuration management toolkit and branch out into things such as firewall management.

Starting the Vagrant virtual machine

Since this is our first time using Vagrant, we'll cover how to start a virtual machine. In the directory with the Vagrantfile, run the following command:

vagrant up

Once this is done, you'll see the output from Vagrant indicating the actions it's taking, as well as output from the commands it runs—this includes the Shell provisioner and the Puppet provisioner. When it's done, you'll end up with something that is similar to the following:

Starting the Vagrant virtual machine

You'll notice some warnings on the screen here. These are options that are changing with the newer version of Puppet. Our manifest could add an allow_virtual setting to get rid of the second warning. The first warning, however, is a result of how Vagrant is calling Puppet.

Connecting to our virtual machine

Once your machine has booted, simply issue the following command to connect:

vagrant ssh

This will connect you to the machine using ssh. Once this is complete, we can start working on our module.

Creating the module

We'll be using a Puppet module to secure SSH. As such, we should go ahead and create the directory to hold our module. You can issue the following commands to create the module skeleton on the guest virtual machine:

sudo mkdir –p /etc/puppet/modules/openssh/manifests
sudo mkdir –p /etc/puppet/modules/openssh/files

These directories will hold the manifests for Puppet to compile as well as our configuration file. For our first simplistic example, we will use a static SSH configuration file. In later chapters, we will build upon it and make it dynamic with the various options that are available.

Tip

It's also possible to make the /etc/puppet/modules/openssh directory a symlink to a directory in /vagrant. If you create the directory in /vagrant, you can use any editor on your host system to edit the files and have it immediately available in the guest. This saves you the trouble of having to configure a good editing environment on the guest machine.

Building the module

Now that we have the framework, we'll build our first module. Much like the preceding code, we'll go through it section by section covering what each resource does. The manifest we're building will be very similar to the one we used to provision the Puppet Master for the use of.

First, we'll edit the /etc/puppet/modules/openssh/manifests/init.pp file to create the module's main manifest. This manifest is the main unit of the Puppet code, which is invoked when we include the module. As we go through each of the sections, we'll go through what they do. A complete manifest file can be found on this book's website, but you should really build it along with us. This will help you with understanding and memorization:

class openssh {

The preceding line defines the class. The class in the init.pp file is always named after the module. It's a new construct we've not seen before that is unique to creating modules:

  package { 'openssh-server':
    ensure => 'latest',
  }

The preceding section is similar to the puppetmaster section. The only difference is that we're using latest instead of present. Being a security-related package, it may make sense to make sure that you keep openssh up to date.

Alternatively, if your environment requires it, you could specify a fixed version to install. This might be useful if you require pretested versions or have validated versions. You must weigh the benefits, ensuring that you run the most recent version of the software, including the risk of almost immediately installing it when it is available, and that you're using the latest tag:

  file { '/etc/ssh/sshd_config':
    ensure => 'present',
    owner  => 'root',
    group  => 'root',
    mode   => '0600',
    source => 'puppet:///modules/openssh/sshd_config',
  }

Tip

As your Puppet code becomes more complex, care must be taken on how you name your files inside your module. It can sometimes be useful to create the full path to the file under the modules directory, so there is no confusion as to the destination of the time. We omit these here only because our modules are simple, and it makes the examples easier to follow.

This is similar to the Puppet Master configuration file, but we introduced a new construct here. We're sourcing the file from puppet master by using the special puppet:// uniform resource identifier (URL). When Puppet runs, it will fetch the file from the master for use on the agent. The source file should be present in the /etc/puppet/modules/openssh/files directory on the master:

  service { 'sshd':
    ensure => 'running',
  }

Here, as before, we ensure that ssh is running when we run Puppet:

  Package['openssh-server'] 
  -> File['/etc/ssh/sshd_config']
  ~> Service['sshd']
}

This is also a new construct called resource chaining. It is an alternative way to specify that we do things in the order listed: first, the package, followed by the file, and then the service. Note the tilde on the service dependency. This shows that we're notifying the service. It means that if the configuration file changes, the service will be restarted.

Tip

In a declarative system, there needs to be a way to ensure that things are run in the correct order. One of the more difficult things for new Puppet users is to grasp the concept that their manifests don't necessarily run in a top-down order. This concept is so hard that in recent versions of Puppet, the default has been changed to a process in the manifest order by default. More information on resource ordering and this change can be found at http://puppetlabs.com/blog/introducing-manifest-ordered-resources.

The openssh configuration file

To build the configuration file we're going to use, we'll start with the openssh configuration file shipped with CentOS and make a few changes. First, we'll copy the existing configuration file with the following command:

sudo cp /etc/ssh/sshd_config /etc/puppet/modules/openssh/files/

Next, we'll edit the file with your favorite editor. Make sure you run it in sudo as you won't have permission to edit the file. We'll uncomment and change the following lines in the file:

PermitRootLogin no
MaxAuthTries 3

We'll start with these changes to demonstrate how the process works. Then, save the file.

Next, we need to make sure the Puppet agent can read it. We'll set the permissions in such a manner that the Puppet user can read it. Execute the following:

sudo chgrp puppet /etc/puppet/modules/openssh/files/sshd_config
sudo chmod 640 /etc/puppet/modules/openssh/files/sshd_config

The site.pp file

Now, we need to bring it all together to tell Puppet to use our module. By default, Puppet runs a file called site.pp on the master to determine what actions to take when a node checks in. We need to add the new module to the file for Puppet to run it.

The file lives in /etc/puppet/manifests on our Vagrant guest. Go ahead and open it in your favorite editor and add the following section:

node default {
  include openssh
}

This adds a default node declaration and includes our openssh module on that node. It will ensure that our new module gets used.

Running our new code

Now that we've got it all built, let's go ahead and see the fruits of our labor. Execute the following command:

sudo puppet agent --test

You should see the output as follows:

Running our new code

Note

If you're running these examples outside Vagrant, you will have a bit more work to do. We're using Vagrant to set our hostname to Puppet, and the master by default has its own certificate signed. If you are running without Vagrant, you will need to add a host file entry or DNS pointing to your master, and you may need to sign the certificate. We'll cover certificate singing in Chapter 5, Securing Puppet.

Victory! You can see that Puppet changed the file to disallow root logins and change the maximum authentication attempts to 3.

As with any new technology, the learning curve can seem somewhat overwhelming at first. We've now gone through a rather lengthy example to effectively make a two-line edit to a configuration file on a single machine. This was a short and simple example to explore some base concepts of Puppet. Using this concept, we could apply this same edit to hundreds or even thousands of machines in our infrastructure with very little additional effort. We'll also be exploring more in-depth examples as we gain a skillset. With some practice, you will find that applying changes across one of many machines becomes second nature with Puppet.

Summary

In this chapter, we built a foundation for things we will do in chapters to come. First, we covered what Puppet is, and how it differs from other tools in its space. We gave a brief introduction to some of the other Puppet components we'll be using in this book as well.

Moving on from this, we covered how to install Puppet on CentOS. We went through a full installation example and covered the basics of configuration files.

Then, we covered the configuration and installation of Vagrant and used it to run our first example. In this example, we configured SSH with a secure configuration file.

Finally, we introduced how Puppet fits into a security ecosystem. While keeping with the basics, we've begun exploring how Puppet can be used to process simple configuration tasks to secure your systems.

This chapter focused on several high-level concepts. As we get further into the book, we'll go more in-depth in examples and they will get much more powerful. As an introductory chapter, the hope was to get you up and running with a working manifest. In future chapters, we will assume a base level of knowledge and link to references you can use if needed.

Additionally, if you wish to get some more information on the base Puppet language before we proceed, there are several books available. Some of them were mentioned earlier in this chapter, and we'll cover more as we proceed through the book. The documentation at http://docs.puppetlabs.com is also very informative, if a little dry at times.

In the next chapter, we'll begin to use our knowledge gained here to explore how Puppet can be used to track changes to resources on our filesystems.

Left arrow icon Right arrow icon

Description

If you are a security professional whose workload is increasing, or a Puppet professional looking to increase your knowledge of security, or even an experienced systems administrator, then this book is for you. This book will take you to the next level of security automation using Puppet. The book requires no prior knowledge of Puppet to get started.

Who is this book for?

If you are a security professional whose workload is increasing, or a Puppet professional looking to increase your knowledge of security, or even an experienced systems administrator, then this book is for you. This book will take you to the next level of security automation using Puppet. The book requires no prior knowledge of Puppet to get started.

What you will learn

  • Use Puppet manifests to show system compliance and track changes to the operating system resources
  • Generate security reports using PuppetDB to show that the systems are up to date
  • Automate CIS compliance using community modules
  • Configure firewalls automatically based on roles
  • Demystify the Puppet SSL stack
  • Set up centralized logging with dashboard search functionality using Elasticsearch, Logstash, and Kibana
  • Configure your systems to be secure automatically using SELinux with Puppet
  • Use Puppet to assist with PCI DSS compliance
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Mar 27, 2015
Length: 236 pages
Edition : 1st
Language : English
ISBN-13 : 9781784397753
Category :
Tools :

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to United States

Economy delivery 10 - 13 business days

Free $6.95

Premium delivery 6 - 9 business days

$21.95
(Includes tracking information)

Product Details

Publication date : Mar 27, 2015
Length: 236 pages
Edition : 1st
Language : English
ISBN-13 : 9781784397753
Category :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 140.97
Mastering Puppet
$47.99
Puppet Cookbook - Third Edition
$48.99
Learning Puppet Security
$43.99
Total $ 140.97 Stars icon

Table of Contents

11 Chapters
1. Puppet as a Security Tool Chevron down icon Chevron up icon
2. Tracking Changes to Objects Chevron down icon Chevron up icon
3. Puppet for Compliance Chevron down icon Chevron up icon
4. Security Reporting with Puppet Chevron down icon Chevron up icon
5. Securing Puppet Chevron down icon Chevron up icon
6. Community Modules for Security Chevron down icon Chevron up icon
7. Network Security and Puppet Chevron down icon Chevron up icon
8. Centralized Logging Chevron down icon Chevron up icon
9. Puppet and OS Security Tools Chevron down icon Chevron up icon
A. Going Forward Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(3 Ratings)
5 star 66.7%
4 star 0%
3 star 0%
2 star 33.3%
1 star 0%
JORGE CARLOS FRANCO Apr 16, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
A very useful resource to learn something about how get your Puppet environment secure and without annoyances.
Amazon Verified review Amazon
puppetfans May 28, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Tt is a good learning puppet information, very good to share,thanks
Amazon Verified review Amazon
Jascha Casadio May 19, 2016
Full star icon Full star icon Empty star icon Empty star icon Empty star icon 2
Despite being such a popular and mature configuration management system, the material covering Puppet is still very limited. Even more, titles covering advanced topics, or version 4 of the language, are as rare as unicorn sighting. Learning Puppet Security belongs to this category and is one of those titles we all Puppetters are eager to get our hands on. It targets intermetidate users who know the basics and are interested in securing the castle.Released early 2015, it covers version 3.x of the language—despite a stable 4 was already released—and spans through some 200 pages. As stated, the book is for intermediate users, which means nothing is said about installing and configuring Puppet.The first thing that comes to the mind reading this book is the interesting choice of topics the author has picked. None of them is usually found in any other Puppet title—apart the certification validation—and all of them play an important role in the toolset of any good DevOps or Systems Engineer. Among them, auditing, logging and network security. Validating certificates, as stated, is a typical topic of the very first chapters of any book of Puppet. Nothing new or different here: The discussion is smooth and well laid out, but is limited to the master-agents scenario. It would have certainly been interesting to have some kind of validation covering the masterlless scenario, which is widely used.Talking about auditing, the author starts with the a discussion about how important it is and why, followed by different options to achieve it and, finally, some code. While enjoyable to read, the topic is covered too superficially. The examples should be more real world and complex. And this is a recurring scenario throughout the whole book: many different interesting concepts are taken into the discussion yes, but while something is said, the reader will often find himself with many doubts and open questions.Overall is an interesting book that covers a very specific advanced topic of Puppet. Given its nature and the levels required to the reader, i would have expected something more complex, complete. The reader gets an idea of what topics to investigate next, but is not given much. For example, the chapters covering the ELK stack, provided that the user know how to set it up correctly, come with very basic hello world like examples, not a complete master/agents example that shows data being gathered and analyzed.I am not fully satisfied with this title and, as such, I don’t feel like recommending it.As usual, you can find more reviews on my personal blog: books.lostinmalloc.com. Feel free to pass by and share your thoughts!
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the digital copy I get with my Print order? Chevron down icon Chevron up icon

When you buy any Print edition of our Books, you can redeem (for free) the eBook edition of the Print Book you’ve purchased. This gives you instant access to your book when you make an order via PDF, EPUB or our online Reader experience.

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
Modal Close icon
Modal Close icon