Python Automation Cookbook - Second Edition

By Jaime Buelta
    What do you get with a Packt Subscription?

  • Instant access to this title and 7,500+ eBooks & Videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Free Chapter
    Automating Tasks Made Easy
About this book
In this updated and extended version of Python Automation Cookbook, each chapter now comprises the newest recipes and is revised to align with Python 3.8 and higher. The book includes three new chapters that focus on using Python for test automation, machine learning projects, and for working with messy data. This edition will enable you to develop a sharp understanding of the fundamentals required to automate business processes through real-world tasks, such as developing your first web scraping application, analyzing information to generate spreadsheet reports with graphs, and communicating with automatically generated emails. Once you grasp the basics, you will acquire the practical knowledge to create stunning graphs and charts using Matplotlib, generate rich graphics with relevant information, automate marketing campaigns, build machine learning projects, and execute debugging techniques. By the end of this book, you will be proficient in identifying monotonous tasks and resolving process inefficiencies to produce superior and reliable systems.
Publication date:
May 2020


Automating Tasks Made Easy

To properly automate tasks, we need a way to make them execute automatically at the proper times. A task that needs to be started manually is not really fully automated.

However, in order to be able to leave them running in the background while worrying about more pressing issues, the task will need to be adequate to run in fire-and-forget mode. We should be able to monitor that it executes correctly, be sure that we capture relevant information (such as receiving notifications if something interesting arises), and know whether there have been any errors while running it.

Ensuring that a piece of software runs consistently with high reliability is actually a very big deal. It is one area that, in order to be done properly, requires specialized knowledge and staff, who typically go by the names of sysadmin, operations, or SRE (Site Reliability Engineering). Big operations, such as Amazon and Google, require huge investment in ensuring...


Preparing a task

It all starts with defining precisely the work that needs to be executed, and designing it in a way that doesn't require human intervention to run.

Some ideal characteristic points are as follows:

  1. Single, clear entry point: No confusion on how to start the task.
  2. Clear parameters: If there are any parameters, they should be as explicit as possible.
  3. No interactivity: Stopping the execution to request information from the user is not possible.
  4. The result should be stored: In order to be checked at a different time than when it runs.
  5. Clear result: When we oversee the execution of a program ourselves, we can accept more verbose results, such as unlabeled data or extra debugging information. However, for an automated task, the final result should be as concise and to the point as possible.
  6. Errors should be logged: To analyze what went wrong.

A command-line program has a lot of those characteristics already...


Setting up a cron job

Cron is an old-fashioned but reliable way of executing commands. It has been around since the 1970s in Unix, and it's an old favorite in system administration to perform maintenance tasks such as freeing up disk space, rotating log files, making backups, and other common, repetitive operations.

This recipe is Unix and Unix-like operating systems specific, so it will work in Linux and macOS. While it's possible to schedule a task in Windows, it's very different and uses Task Scheduler, which won't be described here. If you have access to a Linux server, this can be a good way of scheduling periodic tasks.

The main advantages are as follows:

  • It's present in virtually all Unix or Linux systems and configured to run automatically.
  • It's easy to use, although a little deceptive at first.
  • It's well known. Almost anyone involved with admin tasks will have a general idea of how to use it.
  • It allows...

Capturing errors and problems

An automated task's main characteristic is its fire-and-forget quality. We are not actively looking at the result, but making it run in the background.

Most of the recipes in this book deal with external information, such as web pages or other reports, so the likelihood of finding an unexpected problem when running it is high. This recipe will present an automated task that will safely store unexpected behaviors in a log file that can be checked afterward.

Getting ready

As a starting point, we'll use a task that will divide two numbers, as described in the command line.

This task is very similar to the one presented in step 5 of How it works for the Preparing a task recipe, earlier this chapter. However, instead of multiplying two numbers, we'll divide them.

How to do it...

  1. Create the file, as follows:
    import argparse
    import sys
    def main(number, other_number...

Sending email notifications

Email has become an inescapable tool for everyday use. It's arguably the best place to send a notification if an automated task has detected something. On the other hand, email inboxes are already too full up with spam messages, so be careful.

Spam filters are also a reality. Be careful with whom to send emails to and the number of emails to be sent. An email server or address can be labeled as a spam source, and all emails may be quietly dropped by the internet.

This recipe will show you how to send a single email using an existing email account.

This approach is viable for spare emails sent to a couple of people, as a result of an automated task, but no more than that. Refer to Chapter 9, Dealing with Communication Channels, for more ideas on how to send emails, including groups.

Getting ready

For this recipe, we require a valid email account set up, which includes the following:

  • A valid email server using SMTP...
About the Author
  • Jaime Buelta

    Jaime Buelta is a Software Architect who has been a professional programmer since 2002 and a Python enthusiast since 2010. He has developed software for a variety of fields, focusing, in the last 10 years, on developing web services in Python in the gaming, finance and education industries. He is a strong proponent of automating everything to make computers do most of the heavy lifting, so humans can focus on the important stuff. He published his first book, "Python Automation Cookbook", in 2018 (recently with an extended second edition), followed a year later by "Hands-On Docker for Microservices with Python" describing how to migrate to a microservice architecture. He is currently living in Dublin, Ireland, and is a regular speaker at PyCon Ireland.

    Browse publications by this author
Python Automation Cookbook - Second Edition
Unlock this book and the full library FREE for 7 days
Start now