Automating Tasks Made Easy
To properly automate tasks, we need a way to make them execute automatically at the proper times. A task that needs to be started manually is not really fully automated.
However, in order to be able to leave them running in the background while worrying about more pressing issues, the task will need to be adequate to run in fire-and-forget mode. We should be able to monitor that it executes correctly, be sure that we capture relevant information (such as receiving notifications if something interesting arises), and know whether there have been any errors while running it.
Ensuring that a piece of software runs consistently with high reliability is actually a very big deal. It is one area that, in order to be done properly, requires specialized knowledge and staff, who typically go by the names of sysadmin, operations, or SRE (Site Reliability Engineering). Big operations, such as Amazon and Google, require huge investment in ensuring...
Preparing a task
It all starts with defining precisely the work that needs to be executed, and designing it in a way that doesn't require human intervention to run.
Some ideal characteristic points are as follows:
- Single, clear entry point: No confusion on how to start the task.
- Clear parameters: If there are any parameters, they should be as explicit as possible.
- No interactivity: Stopping the execution to request information from the user is not possible.
- The result should be stored: In order to be checked at a different time than when it runs.
- Clear result: When we oversee the execution of a program ourselves, we can accept more verbose results, such as unlabeled data or extra debugging information. However, for an automated task, the final result should be as concise and to the point as possible.
- Errors should be logged: To analyze what went wrong.
A command-line program has a lot of those characteristics already...
Setting up a cron job
Cron is an old-fashioned but reliable way of executing commands. It has been around since the 1970s in Unix, and it's an old favorite in system administration to perform maintenance tasks such as freeing up disk space, rotating log files, making backups, and other common, repetitive operations.
This recipe is Unix and Unix-like operating systems specific, so it will work in Linux and macOS. While it's possible to schedule a task in Windows, it's very different and uses Task Scheduler, which won't be described here. If you have access to a Linux server, this can be a good way of scheduling periodic tasks.
The main advantages are as follows:
- It's present in virtually all Unix or Linux systems and configured to run automatically.
- It's easy to use, although a little deceptive at first.
- It's well known. Almost anyone involved with admin tasks will have a general idea of how to use it.
- It allows...
Capturing errors and problems
An automated task's main characteristic is its fire-and-forget quality. We are not actively looking at the result, but making it run in the background.
Most of the recipes in this book deal with external information, such as web pages or other reports, so the likelihood of finding an unexpected problem when running it is high. This recipe will present an automated task that will safely store unexpected behaviors in a log file that can be checked afterward.
As a starting point, we'll use a task that will divide two numbers, as described in the command line.
This task is very similar to the one presented in step 5 of How it works for the Preparing a task recipe, earlier this chapter. However, instead of multiplying two numbers, we'll divide them.
How to do it...
- Create the
task_with_error_handling_step1.pyfile, as follows:
import argparse import sys def main(number, other_number...
Sending email notifications
Email has become an inescapable tool for everyday use. It's arguably the best place to send a notification if an automated task has detected something. On the other hand, email inboxes are already too full up with spam messages, so be careful.
Spam filters are also a reality. Be careful with whom to send emails to and the number of emails to be sent. An email server or address can be labeled as a
spam source, and all emails may be quietly dropped by the internet.
This recipe will show you how to send a single email using an existing email account.
This approach is viable for spare emails sent to a couple of people, as a result of an automated task, but no more than that. Refer to Chapter 9, Dealing with Communication Channels, for more ideas on how to send emails, including groups.
For this recipe, we require a valid email account set up, which includes the following:
- A valid email server using SMTP...