The vast majority of Salt users see it as a configuration management platform. And in truth, it handles that very well. But it did not start off with that as a design goal. In its early days, Salt was a communication framework that was designed to be useful even to those who did not write code. But for those who were willing, it was also designed to be heavily extensible to those users who had some Python in their toolbelt.
Before we get into writing modules, it will help to have a basic understanding of how the Salt module system works. In this chapter, you'll learn the following:
How the loader system works
How Salt uses Python
As Salt was originally designed as a backbone that other software could use to communicate, its earliest purpose was to collect information from a large cluster of both physical and virtual machines, and return that data either to the user or to a database. Various programs, such as
netstat, were used to collect that information. Because of that, each program was wrapped with a plugin, which contained various functions to call those programs, and parse the return data.
Those plugins were originally called modules. Later, when other types of module were added to Salt, the original modules began to be referred to as execution modules. This is because the execution modules would do the heavy lifting, and other types of module would generally wrap around them and extend their functionality.
Like many data centers, the one that Salt was created in had various servers that used different software packages to perform their work. One server would be running Nginx, while another would be running DNSMasq. It wouldn't make sense to enable the
nginx module on the DHCP server, or a
dnsmasq module on the web server. A number of popular programs solve this by allowing the user to configure which plugins will be loaded before starting the service.
Salt had a different way of handling plugins. In a large infrastructure, individual configuration of servers can be costly in terms of time. And as configuration management was added to Salt, a core belief grew that configuration management platforms should require as little configuration themselves as possible. What is the point of using such a suite to save time if so much time is required to get it going in the first place?
Execution modules are a type of plugin that performs most of the heavy lifting inside of Salt. These were the first to use the loader system, and for a short time there was no other type of module. As the functionality of Salt increased, it quickly became evident that other types of module would be needed. For instance, return output was originally just printed to the console. Then the output was changed to be easier to handle from shell scripts. Then the outputter system was added, so that output could be displayed in JSON, YAML, Python's
pprint, and any other format that might be useful.
In the beginning, there were some types of module that would always be loaded. The first of these was the
test module, which required nothing more than Salt's own dependencies; in particular, it would only require Python.
Other modules were also designed for general use, requiring no more than Salt's own dependencies. The
file module would perform various file-based operations. The
useradd module would wrap the standard Unix
useradd program. This was fine, so long as Salt was only used on Unix-like platforms. When users started running Salt on Windows, where those utilities were not readily available, things changed. This is where virtual modules really started to shine.
Supporting Salt on various platforms, such as both Unix-like and Windows, presents the same problem as whether or not to make the
nginx module available: if that platform is installed and available, make the module available. Otherwise, don't. Salt handles the availability problem by implementing virtual modules.
The idea behind a virtual module is that it will contain a piece of code that will detect whether or not its dependencies are met, and if so, the module will be loaded and made available to Salt on that system. We'll get into the details of actually doing this in Chapter 2, Writing Execution Modules.
In the beginning, if a module was detected as being loadable, then it would be loaded as the Salt service was started. A number of modules may be loaded for a particular system, which the administrator never intends to use. It may be nice to have them, but in some cases it's better to only load them when they're needed.
When the Salt service starts, the lazy loader will detect which modules may be used on a particular system, but it won't immediately load them into memory. Once a particular module is called, Salt will load it on demand, and then keep it in memory. On a system that typically only uses a small handful of modules, this can result in a much smaller footprint than before.
As we said before, the loader system was originally designed for one type of module: what we now call execution modules. Before long, other types of module were added, and that number continues to grow even today.
This book does not include every type of module, but it does cover quite a few. The following list is not comprehensive, but it will tell you much of what is available now, and possibly give you an idea of what other types of module to look at after you finish this book:
Execution modules do much of the heavy lifting inside of Salt. When a program needs to be called, an execution module will be written for it. When other modules need to use that program, they will call out to that module.
Engines allow Salt to provide internal information and services to long-running external processes. In fact, it may be best to think of engines as programs in their own right, with a special connection to Salt.
These modules were generally created as necessity dictated. All of them are written in Python. And while some can be pretty extensive, most are pretty simple to create. In fact, a number of modules that now ship with Salt were actually provided by users who had no previous Python experience.
Python is well suited to building a loader system. Despite being classified as a very high-level language (and not a mid-level language like C), Python has a lot of control over how it manages its own internals. The existence of robust module introspection built into Python was very useful for Salt, as it made the arbitrary loading of virtual modules at runtime a very smooth operation.
Each Salt module can support a function called
__virtual__(). This is the function that detects whether or not a module will be made available to Salt on that system.
salt-minion service loads, it will go through each module, looking for a
__virtual__() function. If none is found, then the module is assumed to have all of its requirements already met, and it can be made available. If that function is found, then it will be used to detect whether the requirements for that module are met.
If a module type uses the lazy loader, then modules that can be loaded will be set aside to be loaded when needed. Modules that do not meet the requirements will be discarded.
On a Minion, the most important things to load are probably the grains. Although grain modules are important (and are discussed in Chapter 3, Extending Salt Configuration), there are in fact a number of core grains that are loaded by Salt itself.
A number of these grains describe the hardware on the system. Others describe the operating system that Salt is running on. Grains such as
os _family are set, and used later to determine which of the core modules will be loaded.
For example, if the
os_family grain is set to
redhat, then the execution module located at
salt/modules/yumpkg.py will be loaded as the
pkg module. If the
os_family grain is set to
salt/modules/aptpkg.py will be loaded as the
Grains aren't the only mechanism used for determining whether a module should be loaded. Salt also ships with a number of utilities that can be used. The
salt.utils library contains a number of functions that are often faster than grains, or have more functionality than a simple
name=value (also known as a key-value pair) configuration can provide.
One example is the
salt.utils.is_windows() function that, as the name implies, reports whether Salt is being run inside of Windows. If Windows is detected, then
salt/modules/win_file.py will be loaded as the
file module. Otherwise,
salt/modules/file.py will be loaded as the
Another very common example is the
salt.utils.which() function, which reports whether a necessary shell command is available. For instance, this is used by
salt/modules/nginx.py to detect whether the
nginx command is available to Salt. If so, then the
nginx module will be made available.
There are a number of other examples that we could get into, but there is not nearly enough room in this book for all of them. As it is, the most common ones are best demonstrated by example. Starting with Chapter 2, Writing Execution Modules, we will begin writing Salt modules that make use of the examples that we've already gone over, plus a wealth of others.
Salt is made possible by the existence of the loader system, which detects which modules are able to load, and then only what is available. Types of module that make use of the lazy loader will only be loaded on demand.
Python is an integral part of Salt, allowing modules to be easily written and maintained. Salt ships with a library of functions that help support the loader system, and the modules that are loaded with it. These files live in various directories under the
salt/ directory in Salt's code base. For example, execution modules live in
This chapter barely brushed the surface of what is possible with Salt, but it got some necessary concepts out of the way. From here on in, the focus will be all about writing and maintaining modules in Python.