Hands-On Enterprise Automation with Python

4 (2 reviews total)
By Basim Aly
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Setting Up Our Python Environment

About this book

Hands-On Enterprise Automation with Python starts by covering the set up of a Python environment to perform automation tasks, as well as the modules, libraries, and tools you will be using.

We’ll explore examples of network automation tasks using simple Python programs and Ansible. Next, we will walk you through automating administration tasks with Python Fabric, where you will learn to perform server configuration and administration, along with system administration tasks such as user management, database management, and process management. As you progress through this book, you’ll automate several testing services with Python scripts and perform automation tasks on virtual machines and cloud infrastructure with Python. In the concluding chapters, you will cover Python-based offensive security tools and learn how to automate your security tasks.

By the end of this book, you will have mastered the skills of automating several system administration tasks with Python.

Publication date:
June 2018
Publisher
Packt
Pages
398
ISBN
9781788998512

 

Chapter 1. Setting Up Our Python Environment

In this chapter, we will provide a brief introduction to the Python programming language and the differences between the current versions. Python ships in two active versions, and making a decision on which one to use during development is important. In this chapter, we will download and install Python binaries into the operating system.

At the end of the chapter, we will install one of the most advanced Integrated Development Editors (IDEs) used by professional developers around the world: PyCharm. PyCharm provides smart code completion, code inspections, on-the-fly error highlighting and quick fixes, automated code refactoring, and rich navigation capabilities, which we will go over throughout this book, as we write and develop Python code.

The following topics will be covered in this chapter:

  • An introduction to Python
  • Installing the PyCharm IDE
  • Exploring some nifty PyCharm features
 

An introduction to Python


Python is a high-level programming language that provides a friendly syntax; it is easy to learn and use, for both beginner and expert programmers.

Python was originally developed by Guido van Rossum in 1991; it depends on a mix of C, C++, and other Unix shell tools. Python is known as a language for general purpose programming, and today it's used in many fields, such as software development, web development, network automation, system administration, and scientific fields. Thanks to its large number of modules available for download, covering many fields, Python can cut development time down to a minimum.

The Python syntax was designed to be readable; it has some similarities to the English language, while the code construction itself is beautiful. Python core developers provide 20 informational rules, called the Zen of Python, that influenced the design of the Python language; most of them involve building clean, organized, and readable code. The following are some of the rules:

Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated.

You can read more about the Zen of Python at https://www.python.org/dev/peps/pep-0020/.

Python versions

Python comes with two major versions: Python 2.x and Python 3.x. There are subtle differences between the two versions; the most obvious is the way their print functions treat multiple strings. Also, all new features will only be added to 3.x, while 2.x will receive security updates before full retirement. This won't be an easy migration, as many applications are built on Python 2.x.

Why are there two active versions?

I will quote the reason from the official Python website:

"Guido van Rossum (the original creator of the Python language) decided to clean up Python 2.x properly, with less regard for backwards compatibility than is the case for new releases in the 2.x range. The most drastic improvement is the better Unicode support (with all text strings being Unicode by default) as well as saner bytes/Unicode separation.

"Besides, several aspects of the core language (such as print and exec being statements, integers using floor division) have been adjusted to be easier for newcomers to learn and to be more consistent with the rest of the language, and old cruft has been removed (for example, all classes are now new-style, "range()" returns a memory efficient iterable, not a list as in 2.x)."

You can read more about this topic at https://wiki.python.org/moin/Python2orPython3.

Should you only learn Python 3?

It depends. Learning Python 3 will  future-proof your code, and you will use up-to-date features from the developers. However, note that some third-party modules and frameworks lack support for Python 3 and will continue to do so for the near future, until they completely port their libraries to Python 3.

Also, note that some network vendors, such as Cisco, provide limited support for Python 3.x, as most of the required features are already covered in Python 2.x releases. For example, the following are the supported Python versions for Cisco devices; you will see that all devices support 2.x, not 3.x:

Source: https://developer.cisco.com/site/python/

Does this mean I can't write code that runs on both Python 2 and Python 3?

No, you can, of course, write your code in Python 2.x and make it compatible with both versions, but you will need to import a few libraries first, such as the __future__ module, to make it backward compatible. This module contains some functions that tweak the Python 2.x behavior and make it exactly like Python 3.x. Take a look at the following examples to understand the differences between the two versions:

#python 2 only
print "Welcome to Enterprise Automation"

The following code is for Python 2 and 3:

# python 2 and 3
print("Welcome to Enterprise Automation")

Now, if you need to print multiple strings, the Python 2 syntax will be as follows:

# python 2, multiple strings
print "welcome", "to", "Enterprise", "Automation"

# python 3, multiple strings
print ("welcome", "to", "Enterprise", "Automation")

If you try to use parentheses to print multiple strings in Python 2, it will interpret it as a tuple, which is wrong. For that reason, we will import the __future__ module at the beginning of our code, to prevent that behavior and instruct Python to print multiple strings.

The output will be as follows:

Python installation

Whether you choose to go with a popular Python version (2.x) or build future-proof code with Python 3.x, you will need to download the Python binaries from the official website and install them in your operating system. Python provides support for different platforms (Windows, Mac, Linux, Raspberry PI, and so on):

  1. Go to https://www.python.org/downloads/ and choose the latest version of either 2.x or 3.x:

  1. Choose your platform from the Download page, and either the x86 or x64 version:
  1. Install the package as usual. It's important to select the Add python to the path option during the installation, in order to access Python from the command line (in the case of Windows). Otherwise, Windows won't recognize the Python commands and will throw an error:
  1. Verify that the installation is complete by opening the command line or terminal in your operating system and typing python. This should access the Python console and provide a verification that Python has successfully installed on your system:
 

Installing the PyCharm IDE


PyCharm is a fully fledged IDE, used by many developers around the world to write and develop Python code. The IDE is developed by the Jetbrains company and provides rich code analysis and completion, syntax highlighting, unit testing, code coverage, error discovery, and other Python linting operations.

Also, PyCharm Professional Edition supports Python web frameworks, such as Django, web2py, and Flask, beside integrations with Docker and vagrant for running a code over them. It provides amazing integration with multiple version control systems, such as Git (and GitHub), CVS, and subversion.

In the next few steps, we will install PyCharm Community Edition:

  1. Go to the PyCharm download page (https://www.jetbrains.com/pycharm/download/) and choose your platform. Also, choose to download either the Community Edition (free forever) or the Professional Edition (the Community version is completely fine for running the codes in this book):
  1. Install the software as usual, but make sure that you select the following options:
    • 32- or 64-bit launcher (depending on your operating system).
    • Create Associations (this will make PyCharm the default application for Python files).
    • Download and install JRE x86 by JetBrains:
  1. Wait until PyCharm downloads the additional packages from the internet, and installs it, then choose Run PyCharm Community Edition:

  1. Since this is a new and fresh installation, we won't import any settings from
  1. Select the desired UI theme (either the default or darcula, for dark mode). You can install some additional plugins, such as Markdown and BashSupport, which will make PyCharm recognize and support those languages. When you finish, click on Start Using PyCharm:

 

Setting up a Python project inside PyCharm

Inside PyCharm, a Python project is a collection of Python files that you have developed and Python modules that are either built in or were installed from a third party. You will need to create a new project and save it to a specific location inside your machine before starting to develop your code. Also, you will need to choose the default interpreter for this project. By default, PyCharm will scan the default location on the system and search for the Python interpreter. The other option is to create a completely isolated environment, using Python virtualenv. The basic problem with the virtualenv address is its package dependencies. Let's assume that you're working on multiple different Python projects, and one of them needs a specific version of x package. On the other hand, one of the other projects needs a completely different version from the same package. Notice that all installed Python packages go to /usr/lib/python2.7/site-packages, and you can't store different versions of the same package. The virtualenv will solve this problem by creating an environment that has its own installation directories and its own package; each time you work on either of the two projects, PyCharm (with the help of virtualenv) will activate the corresponding environment to avoid any conflict between packages.

Follow these steps to set up the project:

  1. Choose Create New Project:

  1. Choose the project settings:
    1. Select the type of project; in our case, it will be Pure Python.
    2. Choose the project's location on the local hard drive.
    3. Choose the Project Interpreter. Either use the existing Python installation in the default directory, or create a new virtual environment tied specifically to that project.
    4. Click on Create.
  1. Create a new Python File inside the project:

    1. Right-click on the project name and select New.
    2. Choose Python File from the menu, then choose a filename.

A new, blank file is opened, and you can write a Python code directly into it. Try to import the __future__ module, for example, and PyCharm will automatically open a pop-up window with all possible completions available as shown in the following screenshot:

  1. Run your code:
    1. Enter the code that you wish to run.
    2.  Choose Edit Configuration to configure the runtime settings for the Python file.
  1. Configure new Python settings for running your file:
    1. Click on the + sign to add a new configuration, and choose Python.
    2. Choose the configuration name.
    3. Choose the script path inside your project.
    4. Click on OK.
  1. Run the code:

    1. Click on the play button beside the configuration name.
    2. PyCharm will execute the code inside the file specified in the configuration, and will return the output to the terminal.
 

Exploring some nifty PyCharm features


In this section, we will explore some of PyCharm's features. PyCharm has a huge collection of tools out of the box, including an integrated debugger and test runner, Python profiler, a built-in Terminal, integration with major VCS and built-in database tools, remote development capabilities with remote interpreters, an integrated SSH Terminal, and integration with Docker and Vagrant. For a list of other features, please check the official site (https://www.jetbrains.com/pycharm/features/).

Code debugging

Code debugging is a process that can help you to understand the cause of an error, by providing an input to the code and walking through each line of the code and seeing how it evaluates at the end. The Python language contains some debugging tools to get insights from the code, starting with a simple print function, assert command till a complete unit testing for the code. PyCharm provides an easy way to debug the code and see the evaluated values.

To debug code in PyCharm (say, a nested for loop with if clauses), you need to set a breakpoint on the line at which you want PyCharm to stop the program execution. When PyCharm hits this line, it will pause the program and dump the memory to see the contents of each variable:

Notice that the value of each variable is printed besides it, on the first iteration:

Also, you can right-click on the breakpoint and add a specific condition for any variable. If the variable is evaluated to a specific value, then a log message will be printed:

Code refactoring

Refactoring the code is the process of changing the structure of a specific variable name inside your code. For example, you may choose a name for your variable and use it for a project that consists of multiple source files, then later decide to rename the variable to something more descriptive. PyCharm provides many refactoring techniques, to make sure that the code can be updated without breaking the operation.

PyCharm does the following:

  • The refactoring itself
  • Scans every file inside the project and makes sure that the references to the variables are updated
  • If something can't be updated automatically, it will give you a warning and open a menu, so you can decide what to do
  • Saves the code before refactoring it, so you can revert it later

Let's look at an example. Assume that we have three Python files in our project, called refactor_1.py, refactor_2.py, and refactor_3.py. The first file contains important_funtion(x), which is also used in both refactor_2.py and refactor_3.py.

 

Copy the following code in a refactor_1.py file:

def important_function(x):
print(x)

Copy the following code in a refactor_2.py file:

from refactor_1 import important_function
important_function(2)

Copy the following code in a refactor_3.py file:

from refactor_1 import important_function
important_function(10)

To perform the refactoring, you need to right-click on the method itself, select Refactor | Rename, and enter the new name for the method:

Notice that a window opens at the bottom of the IDE, listing all references of this function, the current value for each one, and which file will be affected after the refactoring:

If you choose Do Refactor, all of the references will be updated with the new name, and your code will not be broken.

Installing packages from the GUI

PyCharm can be used to install packages for existing interpreters (or the virtualenv) using the GUI. Also, you can see a list of all installed packages, and whether upgrades are available for them.

First, you need to go to File | Settings | Project | Project Interpreter:

As shown in the preceding screenshot, PyCharm provides a list of installed packages and their current versions. You can click on the + sign to add a new package to the project interpreter, then enter the package initials into the search box:

You should see a list of available packages, containing a name and description for each one. Also, you can specify a specific version to be installed on your interpreter. Once you have clicked on Install Package, PyCharm will execute a pip command on your system (and may ask you for a permission); then, it will download the package onto the installation directory and execute the setup.py file.

 

Summary


In this chapter, you learned the differences between Python 2 and Python 3, and how to decide which one to use, based on your needs. Also, you learned how to install a Python interpreter and how to use PyCharm as an advanced editor to write and manage your code's life cycle.

In the next chapter, we will discuss the Python package structure and the common Python packages used in automation.

 

About the Author

  • Basim Aly

    Bassem Aly is an experienced SDN/NFV solution consultant at Juniper Networks and has been working in the Telco industry for last 9 years. He focused on designing and implementing next generation by leveraging different automation and devops frameworks.Also he has extensive experience in architecting and deploying telco applications over the openstack. Bassem also conducts corporate training on network automation & network programmability using python and ansible.

    Browse publications by this author

Latest Reviews

(2 reviews total)
Ta książka jest dla mnie przydatna
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Recommended For You

Book Title
Unlock this full book FREE 10 day trial
Start Free Trial