When administrating IT devices from a remote location, be it network equipment or servers, SSH has become the standard. With its secure transport and various authentication methods, it's a safe choice that is widely used to this day to administer and configure servers or network devices. It is thus only natural when getting started with programmability and network automation, to find a way of issuing SSH commands not by hand but from a script. With this, you can take a sequence of commands you used to type into the device by hand and execute them programmatically on one or more devices. The last part is crucial. With a script that executes commands for you, you can easily apply the same sequence of commands to another device.

While we could implement the SSH protocol ourselves, this would be cumbersome work. Luckily, the Python community has already developed an SSH client library that is available for our use, called Paramiko.

In this chapter, we are going to learn the basics of programmatically connecting to a network device using SSH. We are going to use Cisco devices for our examples but the workflow is the same regardless of the vendor.

In this chapter, we will work through the following recipes:

• Initiating an SSH session with Paramiko
• Executing a command via SSH
• Reading the output of an executed command
• Executing the same command against multiple devices
• Executing a sequence of commands
• Using public/private keys for authentication
• Loading local SSH configuration

For this section and the remainder of the book, you'll need an installation of Python. Specifically, you'll need a Python interpreter of version 3.6.1 or higher. This book makes use of language constructs of Python 3 and thus is incompatible with Python 2.x. If you haven't done so already in the previous chapter, please go ahead and install the Paramiko package (python3 -m pip install paramiko). At the time of writing, we are using the latest version of Paramiko, version 2.7.1. You may install this exact version by issuing python3 -m pip install paramiko==2.7.1.

You also want a code editor. Popular choices include Microsoft Visual Studio Code or Notepad++. Additionally, you'll need a device (virtual or physical) that you can log into via SSH.

You can view this chapter's code in action here: https://bit.ly/37Ih46N

The basis of connecting to a device via SSH with Python and Paramiko is the SSHClient object of the library. We will use this object to create an initial connection to the SSH server and later we will use the functions of this object to execute commands on the device.

In this recipe, you will see how to programmatically open an SSH connection.

Getting ready

Open your code editor and start by creating a file called initiating.py. Next, navigate your terminal to the same directory that you just created the initiating.py file in.

How to do it...

Let's start by importing the Paramiko library and creating a client object. We will also specify the host, username, and password in variables and then initiate a connection to the specified host:

1. Import the Paramiko library:

   from paramiko.client import SSHClient

2. Specify the host, username, and password. You can name these variables however you like. In the Python community, it has become standard to uppercase these global variables. The three variables SSH_USER, SSH_PASSWORD, and SSH_HOST are variables of type string and we thus use double quotes to mark them. The SSH_PORT variable is an integer and thus does not use double quotes:

   SSH_USER = "<Insert your ssh user here>"
   SSH_PASSWORD = "<Insert your ssh password here>"
   SSH_HOST = "<Insert the IP/host of your device/server here>"
   SSH_PORT = 22 # Change this if your SSH port is different

3. Create an SSHClient object, which we just imported from Paramiko:

   client = SSHClient()

4. While we have created our client object, we have not yet connected to the device. We will use the connect method of the client object to do so. Before actually connecting, we will need to make sure that our client knows the host keys:

   client.load_system_host_keys()
   try:
       client.connect(SSH_HOST, port=SSH_PORT,
                                username=SSH_USER,
                                password=SSH_PASSWORD,
                                look_for_keys=False)
       print("Connected successfully!")
   except Exception:
       print("Failed to establish connection.")

5. Finally, we have created our connection. It is a good habit to close connections after we are done using them. To do so, we can use the close() function of our client:

   finally:
       client.close()

6. To run this script, go to your terminal and execute it with the following:

   python3 initiating.py

How it works...

In this example, we first imported the Paramiko library's SSHClient class. Next, we set up our connection details. While you could also provide these details directly when calling the connect method on the client object, it is good practice to put them into variables. This means that if you are creating multiple client objects at different points of your script, you don't have to change the username/password and host in each of these calls but just once in the variables. Be careful when submitting these scripts to your colleagues or uploading them to code hosting services though, as they do contain your login details. You can have a look at the There's more section of this recipe to see how you can either prompt for this information interactively or get it from your environment.

Before connecting to the device, using the connect method, we load the host keys. SSH uses host keys and the fingerprints of an SSH server to make sure that the IP you are connecting to is the server you connected to before. When connecting to a brand-new device, you'll have to accept this new host key. While we can keep a separate set of host keys for our Paramiko client, it is usually best to just use the host keys that the user executing the script has. By doing this, you can connect to every device you have previously connected to from your command line, also from your Paramiko scripts.

To do this loading, we are using the load_system_host_keys() function. This function searches the default known hosts file used by OpenSSH and copies them over into Paramiko. See the following section for an example of how to make Paramiko accept new keys by default.

With our host keys configured, we can now actually connect to the device that we have specified. To do so, we are using a try-catch block. This is a Python construct that allows us to catch exceptions, errors that can be raised by any part of the code we are using, and handle them. Python will attempt to execute the instructions in the try block. If any part of that code, in our example the connect() method, errors out and raises an exception, Python will jump into the except block and execute the instructions within that block. In our example, we are only printing out a message that our connection was unsuccessful. The third block, our finally block, will be executed both when the connection has been successful (our except block was not executed) as well as after a failed connection (our except block was executed). This allows us to clean up after both a successful and unsuccessful connection and avoids dangling SSH connections.

There's more...

In this example, we relied on the user having already logged into the device from their command line in order for the host to be known. If we use the preceding code to connect to a device that was not previously known, the code will fail with an exception.

The way Paramiko handles unknown host keys can be specified using a policy. One of these policies, AutoAddPolicy, allows us to just add unknown host keys to our scripts set of host keys:

from paramiko.client import SSHClient, AutoAddPolicy
SSH_USER = "<Insert your ssh user here>"
SSH_PASSWORD = "<Insert your ssh password here>"
SSH_HOST = "<Insert the IP/host of your device/server here>"
SSH_PORT = 22 # Change this if your SSH port is different
client = SSHClient()
client.set_missing_host_key_policy(AutoAddPolicy())
client.connect(SSH_HOST, port=SSH_PORT,
                         username=SSH_USER,
                         password=SSH_PASSWORD)

The preceding code will automatically add these host keys. Be aware that this might be a potential security risk since you are not verifying that the host you are connecting to is the one you connected to last time.

In this example, we passed connection details such as username, hostname, and password directly as a variable in the script. While this is great for testing, you might want to have your script prompt you for a variable upon execution. For non-secret variables such as the username, host, and port, we can use the built-in input() function, but for passwords, it's better to use a dedicated password prompt that hides what you have typed so that someone looking over your console history can't retrieve your password. For this purpose, Python has the built-in getpass module.

Follow these steps to retrieve the configuration variables necessary, not as static information in the script but rather interactively from the user using a combination of input and the getpass module:

import getpass
SSH_PASSWORD = getpass.getpass(prompt='Password: ', stream=None)
SSH_USER = input("Username: ")
SSH_HOST = input("Host: ")
SSH_PORT = int(input("Port: "))

With our connection now open, we can go ahead and execute a command on our remote device. Similar to the way we deal with issuing commands on a remote device by hand, we have three different streams that come back to us: the standard out (or stdout), which is the normal output, the standard error (or stderr), which is the default stream for the system to return errors on, and the standard in (or stdin), which is the stream used to send text back into the executed command. This can be useful if, in your workflow, you would normally interact with the command line.

In this recipe, you will see how to programmatically open an SSH connection and then send a command of your choice to the device.

Getting ready

Open your code editor and start by creating a file called command.py. Next, navigate your terminal to the same directory that you just created the command.py file in.

How to do it...

Let's start by importing the Paramiko library and create a client object as seen in the last recipe. We'll then execute a single command of your choice on this device:

1. Import the Paramiko library:

   from paramiko.client import SSHClient

2. Specify the host, username, and password. You can name these variables however you like. In the Python community, it has become a standard to uppercase these global variables:

   SSH_USER = "<Insert your ssh user here>"
   SSH_PASSWORD = "<Insert your ssh password here>"
   SSH_HOST = "<Insert the IP/host of your device/server here>"
   SSH_PORT = 22 # Change this if your SSH port is different

3. Create an SSHClient object, which we just imported from Paramiko:

   client = SSHClient()

4. While we have created our client object, we have not yet connected to the device. We will use the connect method of the client object to do so. Before actually connecting, we will need to make sure that our client knows the host keys:

   client.load_system_host_keys()
   client.connect(SSH_HOST, port=SSH_PORT,
                            username=SSH_USER,
                            password=SSH_PASSWORD)

5. Finally, we can use the client to execute a command. Executing a command will return three different file-like objects to us representing stdin, stdout, and stderr:

   CMD = "show ip interface brief" # You can issue any command you want
   stdin, stdout, stderr = client.exec_command(CMD)
   client.close()

6. To run this script, go to your terminal and execute it with this:

   python3 command.py

How it works...

In this example, we first created a new client as seen in the previous example. We then used the exec_command() method to execute a command of our choice.

The function returns three different file-like objects for the three different streams: stdin, stdout, and stderr. In the next recipe, Reading the output of an executed command, we will use this to read back the output that was provided when executing a command.

In the previous recipe, we saw how to first connect to a device and then execute a command. So far, we have ignored the output though.

In this recipe, you will see how to programmatically open an SSH connection, send a command, and then write the output of that command back to a file. We will use this to back up a running configuration.

Getting ready

Open your code editor and start by creating a file called read_out.py. Next, navigate your terminal to the same directory that you just created the read_out.py file in.

How to do it...

Let's start by importing the Paramiko library and create a client object as seen in the last recipe. Then execute a single command of your choice on this device and save the output:

1. Import the Paramiko library:

   from paramiko.client import SSHClient

2. Specify the host, username, and password. You can name these variables however you like. In the Python community, it has become a standard to uppercase these global variables:

   SSH_USER = "<Insert your ssh user here>"
   SSH_PASSWORD = "<Insert your ssh password here>"
   SSH_HOST = "<Insert the IP/host of your device/server here>"
   SSH_PORT = 22 # Change this if your SSH port is different

3. Create an SSHClient object, which we just imported from Paramiko:

   client = SSHClient()

4. While we have created our client object, we have not yet connected to the device. We will use the connect method of the client object to do so. Before actually connecting, we will need to make sure that our client knows the host keys:

   client.load_system_host_keys()
   client.connect(SSH_HOST, port=SSH_PORT,
                            username=SSH_USER,
                            password=SSH_PASSWORD)

5. Finally, we can use the client to execute a command. Executing a command will return three different file-like objects to us representing stdin, stdout, and stderr:

   CMD = "show running-config" 
   stdin, stdout, stderr = client.exec_command(CMD)

6. We will use the stdout object to retrieve what the command has returned:

   output = stdout.readlines()

7. Next, we write the output back to a file:

   with open("backup.txt", "w") as out_file
       for line in output:
           out_file.write(line)

8. To run this script, go to your terminal and execute it with this:

   python3 read_out.py

How it works...

In this example, we first created a new client as seen in the previous example. We then used the exec_command() method to execute a command of our choice. We then used the returned file-like object for the standard out to read everything that our remote device printed into the standard out and write it into a file.

A file-like object in the context of Python is an object that offers the same functions as a file object. When using the open() function to create a new file object to read/write from locally, we are offered some functions such as write() or readlines(). These functions allow us to read or write from or to a file. A file-like object offers the same functions and can thus be used as if it was a remote file. We use the readlines() function on the stdout object to read everything returned – in this example, the running configuration of our device – and write it line by line to a local file.

There's more...

If you prefer to just see the output of your command instead of writing it to a file, you can also use the following construct, instead of the code provided in step 7 of the recipe, to print out your entire configuration:

for line in output:
    print(line.strip())  

The strip() method used in the preceding example will delete any unnecessary leading or trailing whitespaces.

In the previous recipes, we have always only dealt with a single device. Quite often we have a fleet of similar devices that we want to configure in unison.

In this recipe, you will see how to programmatically open an SSH connection to multiple devices, issue the same command to all of them, and then save the output. We will again use this example to back up the running configuration of multiple devices.

Getting ready

Open your code editor and start by creating a file called exec_multiple.py. Next, navigate your terminal to the same directory that you just created the exec_multiple.py file in. Additionally, create a file called credentials.json. We will use this file to retrieve credentials such as the username and password of our devices.

How to do it...

Let's start by creating our credentials file. We will then read that file from our Python script, create clients for each of these devices, and finally execute a command while also saving the output back to our file:

1. Import the required libraries, Paramiko and json:

   import json
   from paramiko.client import SSHClient

2. Open up the credentials.json file and provide the credentials to your device(s) in the format shown in the following code. You can specify as many devices as you want:

   [
    {
      "name": "<insert a unique name of your device>",
      "host": "<insert the host of your device>",
      "username": "<insert the username>",
      "password": "<insert the password",
      "port": 22
    },
    {
      "name": "<insert a unique name of your device>",
      "host": "<insert the host of your device>",
      "username": "<insert the username>",
      "password": "<insert the password",
      "port": 22
    }
   ]

3. Go back to your exec_multiple.py file. We will now open the JSON file in our Python script:

   credentials = {}
   with open("credentials.json") as fh:
        json.load(fh)

4. Create a variable holding the command you want to execute. We will then loop over all the devices specified in our credentials.json file and create an SSH client object. Additionally, we will create an individual output file for each of our devices based on the name we specified in the JSON file:

   CMD = "show running-config"
   for cred in credentials:
        out_file_name = str(cred['name']) + ".txt"
        client = SSHClient()
        client.load_system_host_keys()
        client.connect(SSH_HOST, port=cred['port'],
                                 username=cred['username'],
                                 password=cred['password'])
        stdin, stdout, stderr = client.exec_command(CMD)
        out_file = open(out_file_name, "w")
        output = stdout.readlines()
        for line in output:
             out_file.write(line)
        out_file.close()
        client.close()
        print("Executed command on " + cred['name'])

5. To run this script, go to your terminal and execute it with this:

   python3 exec_multiple.py

How it works...

In this example, we first create a JSON file containing the credentials and connection details for all of our devices. We can view this file as a type of inventory. In general, it is good practice to keep this information separate from the Python code that is acting upon it.

We then use the built-in json module to read the credentials file into a list of dictionaries that we can loop over.

Based on the credentials specified in the credentials file, we then open up a new Paramiko connection, execute the command, and write the output to a new log file, that is, by including the name we set for our device in the JSON file, which is unique for each of the devices.

With this, we can back up the running configuration of an entire fleet of devices from one simple script.

In the previous recipes, we have always only dealt with a single command that we wanted to execute. Maybe you have tried adding another call to the exec_command() function to the client already and have run into an error telling you that the session is closed. It is indeed correct that, once your command is done executing, the connection will close.

But quite often we don't want to execute only one but a sequence of commands one after another. We could reconnect for each of the commands, but this is a workaround that would create a lot of unneeded disconnecting and reconnecting. What we can do instead is, if the target device's configuration allows it, open up a shell session. This shell session is a single command, and we can then use stdin, stderr, and stdout to send multiple commands in the same session.

In this recipe, you will see how to programmatically open an SSH connection to a device, open a shell, and then send a list of commands to the device before closing the connection.

Getting ready

Open your code editor and start by creating a file called exec_multiple_commands.py. Next, navigate your terminal to the same directory that you just created the exec_multiple_commands.py file in.

How to do it...

Let's start by creating our credentials file. We will then read that file from our Python script, create clients for each of these devices, and finally execute a command while also saving the output back to our file:

1. Import the Paramiko library. We will also need the built-in time library:

   from paramiko.client import SSHClient
   import time

2. Specify the host, username, and password. You can name these variables however you like. In the Python community, it has become a standard to uppercase these global variables:

   SSH_USER = "<Insert your ssh user here>"
   SSH_PASSWORD = "<Insert your ssh password here>"
   SSH_HOST = "<Insert the IP/host of your device/server here>"
   SSH_PORT = 22 # Change this if your SSH port is different

3. Create an SSHClient object, which we just imported from Paramiko:

   client = SSHClient()

4. While we have created our client object, we have not yet connected to the device. We will use the connect method of the client object to do so. Before actually connecting, we will need to make sure that our client knows the host keys:

   client.load_system_host_keys()
   client.connect(SSH_HOST, port=SSH_PORT,
                            username=SSH_USER,
                            password=SSH_PASSWORD)

5. Open up an interactive shell session and a channel that we can use to retrieve the output:

   channel = client.get_transport().open_session()
   shell = channel.invoke_shell()

6. Next, specify the list of commands we want to execute on the device:

   commands = [
        "configure terminal",
        "hostname test"
   ]

7. Iterate over each of the commands, execute them, and then wait for 2 seconds:

   for cmd in commands:
        shell.send(cmd + "\n")
         out = shell.recv(1024)
         print(out)
        time.sleep(1)

8. Finally, we need to close the connection:

   client.close()

9. To run this script, go to your terminal and execute it with this:

   python3 exec_multiple_commands.py

How it works...

In this example, we use Paramiko's concept of an interactive shell to send multiple commands to the remote device one after another.

When invoking a shell with Paramiko, we will retrieve a channel. This channel was used by Paramiko in the background all along to execute our commands but has been hidden from us so far. The channel takes care of low-level aspects such as sending and receiving the data to and from the raw network connection to our device. The channel function we use in this example is the send() function, which sends a string to the remote device. Mind the carriage return we added to the command. The same way you, when connecting to a device via SSH, have to execute a command by typing enter, the interactive session has to indicate that via a linebreak, which is the same symbol sent by your interactive session when hitting the Enter key.

We are using the sleep function here to wait for the commands you have passed to have finished executing. For short-running commands, you can get away with not using this sleep function.

So far, we have always used a username/password combination to connect to our device. This is not the most secure way, however, and many security policies advocate using public-private key pairs instead of a static password.

In this recipe, you will see how to programmatically open an SSH connection using a password-protected private key.

Getting ready

Open your code editor and start by creating a file called key_file.py. Next, navigate your terminal to the same directory that you just created the key_file.py file in.

You'll also need a password-protected private key for the device/server you are trying to connect to and have the device/server configured to allow or require key-based logins.

How to do it...

Let's start by importing the required libraries, define our new connection details, and finally open up a connection using key-based authentication:

1. Import the Paramiko library:

   from paramiko.client import SSHClient

2. Specify the host and username. You can name these variables however you like. In the Python community, it has become a standard to uppercase these global variables. Instead of the device password, we will now need two new variables – the path to the private key file that we want to use to authenticate and the password for that private key file:

   SSH_USER = "<Insert your ssh user here>"
   SSH_HOST = "<Insert the IP/host of your device/server here>"
   SSH_PORT = 22 # Change this if your SSH port is different
   SSH_KEY = "<Insert the name of your private key here>"
   SSH_KEY_PASSWORD = "<Insert the password here>"

3. Create an SSHClient object, which we just imported from Paramiko:

   client = SSHClient()

4. While we have created our client object, we have not yet connected to the device. We will use the connect method of the client object to do so. Before actually connecting, we will still need to make sure that our client knows the host keys:

   client.load_system_host_keys()
   client.connect(SSH_HOST, port=SSH_PORT,
                            username=SSH_USER,
                            look_for_keys=True,
                            key_filename=SSH_KEY,
                            passphrase=SSH_KEY_PASSWORD)

5. As seen before, we can now execute a command once the connection is established:

   stdin, stdout, stderr = client.exec_command('<your command>')

6. Finally, we need to close the connection:

   client.close()

7. To run this script, go to your terminal and execute it with this:

   python3 key_file.py

How it works...

In this example, we use Paramiko's ability to load RSA keys for authentication to avoid using a username/password combination for authentication.

We need to import the same packages we have used before. The difference lies in the parameters that we pass to the connect function. Instead of specifying a password, we specify the name of our ssh key. The library will then, as indicated by setting the look_for_keys flag to true, search common places such as ~/.ssh for keys and match them with the name provided. The passphrase argument is used to provide the passphrase used to decode the private key. If your private key does not have a passphrase, you can omit this argument.

Once the connection is established, we can use the client in the same way as we did before, when dealing with username-password authentication.

There's more...

In the preceding example, we relied on the key file being present in one of the known paths. Sometimes you might want to explicitly specify the path you are loading a key file from. You can do so by, instead of just specifying the filename in the key_filename attribute, specifying the entire path where Paramiko can find your private key.

For example, if your private key is in /home/user/my_keys/id_rsa, you could modify the preceding example like so:

SSH_KEY = "/home/user/my_keys/id_rsa"

If you want to connect to different devices and have multiple keys, one for each device, you can also pass a list of key names or paths to ssh keys to the key_filename attribute:

SSH_KEY = [
           "/home/user/my_keys/device_1",
           "/home/user/my_keys/device_2",
           "/home/user/my_keys/device_3"
]

Paramiko will then try out all the keys in the provided list for each device you are connecting to.

When dealing with multiple different devices and connecting to them via SSH, it can be convenient to specify information such as the hostname, port, username, or identity file to use in a specific configuration file. The OpenSSH implementation stores this file in a file called config in the .ssh directory in your home directory (~/.ssh/config on macOS and Linux).

While we could copy and paste this information into our Python scripts or try to write a parsing function for the format ourselves, it is easier and more convenient to use Paramiko's SSH configuration parser.

In this recipe, you will see how to programmatically parse your SSHConfig file, extract the relevant information based on a host, and store it in a dictionary.

Getting ready

Open your code editor and start by creating a file called parse_config.py. Next, navigate your terminal to the same directory that you just created the parse_config.py file in.

You'll also need an SSH config file for the device you are trying to connect to. In this example, we will be using a config file that has the following content:

Host example
     Host <insert your host address here>
     User <insert your user here>
     Port <insert the port here>
     IdentityFile <insert the path to your private key here>

How to do it...

Let's start by importing the required libraries and defining the path to our SSH configuration:

1. Import the Paramiko library:

   from paramiko.client import SSHClient
   from paramiko import SSHConfig

2. Specify the path to your SSH config file and the name of your host as it appears in your SSH configuration (example in this snippet). We will populate all the other variables from the configuration we are reading:

   SSH_CONFIG = "<insert path to ssh config here>"
   SSH_HOST = "example"

3. Create an SSHConfig object, which we just imported from Paramiko, and create a local file object with the path to our SSH configuration:

   config = SSHConfig()
   config_file = open(SSH_CONFIG)

4. Next, we need to tell the SSHConfig object to load and parse the configuration file:

   config.parse(config_file)

5. With the config parsed, we can now do a lookup on this configuration object to extract all information stored in the configuration itself. The lookup function will return a dictionary:

   dev_config = config.lookup(SSH_HOST)

6. With our device configuration extracted from the SSH config we can go ahead and fill our connection details with what we have extracted from the SSH configuration file:

   client.load_system_host_keys()
   HOST = dev_config['hostname'],
   client.connect(HOST, port=int(dev_config['port']),
                        username=dev_config['user'],
                        key_filename=dev_config['identityfile'])

7. With the connection established, we can do all the different things we discovered in previous recipes before finally closing the connection:

   client.close()

8. To run this script, go to your terminal and execute it with this:

   python3 parse_config.py

How it works...

In this example, we use Paramiko's ability to parse an SSH configuration file to not define this information in multiple different locations.

We start by importing the SSHConfig class, in addition to the already established SSHClient class. Instead of manually specifying the host, username, and key file information, we now create a local file object that points to our SSH configuration.

With that file opened, we can now have Paramiko parse this configuration. The SSHConfig object now contains all the different information for each of the hosts. We can then do a lookup on our host – in this recipe, the host is called example – and extract all configuration variables that are known in the configuration file.

From that, we proceed to providing that information to the SSHClient. Instead of statically specifying it, we just access it from the dictionary that was returned by Paramiko when doing the lookup on the host.