Over the course of this book, you will be introduced to multiple concepts that will enable you to build a complete modern web application. You will progress from a "Hello world" web page to a complete web application that uses databases, caches, asynchronous task processing, authentication, role-based access, a REST API, and internationalization. You will learn a comprehensive way of structuring your application so that it can grow effortlessly. To choose between SQL and NoSQL technologies, you will learn how to use the most common Flask extensions to help you leverage multiple technologies, from sending emails to authentication using social media accounts. Toward the end, you will learn how to write tests, build a modern continuous integration/delivery pipeline with Docker and Jenkins, deploy your application to multiple cloud services, and know how to deal with high availability and scaling. We will tackle all of these topics with a simple and practical approach.

Flask is the Python web framework that we are going to use. It has a very well-designed API, is very easy to learn, and makes no assumptions whatsoever as to what technology stack you are going to use, so it won't get in your way. Flask has a micro footprint, but leverages an extension system that contains hundreds of packages from a very active and vibrant community.

In this first chapter, you will learn how to set up your development environment and build your first Flask application. We will be covering the following topics:

• Setting up and learning how to use Git, a powerful version control system
• Learning pip, the Python management system, and how to create virtual environments with different setups
• Setting up and learning the basic facts about Docker
• Building a first simple Flask application

Using Python or any other language requires you to use a version control system. A version control system is a tool that records changes in files over time. This allows a programmer to revert to an earlier version of the file and identify bugs more easily. You can test new ideas without fear of breaking your current code, and your team can work using a predefined workflow without stepping on each others' toes. Git was developed by Linus Torvalds, the father of Linux. It's decentralized, light, and has great features that get the job done the right way.

Installing Git is very simple. Simply go to http://www.git-scm.com/downloads and click on the operating system (OS) that is being run. A program will begin to download will walk you through the basic installation process.

Git was originally solely developed for Unix OSes (for example, Linux and macOS X). Consequently, using Git on Windows is not seamless. During the installation, the installer will ask whether you want to install Git alongside the normal Windows Command Prompt. Do not pick this option. Choose the default option that will install a new type of command processor on your system named Bash (Bourne-again shell), which is the same command processor that the Unix systems use. Bash is much more powerful than the default Windows command line, and this is what we will be using for all the examples in this book.

Git is a very complex tool; only the basics that are needed for this book will be covered in this section.

Git does not track your changes automatically. In order for Git to run properly, we have to give it the following information:

• Which folders to track
• When to save the state of the code
• What to track and what not to track

Before we can do anything, we have to tell Git to initialize a new git repository in our directory. Run the following code on your Terminal:

$ git init

Git will now start to track changes in our project. As git tracks our files, we can see the status of our tracked files and any files that are not tracked by typing the following command:

$ git status

Now we can save our first commit, which is a snapshot of our code at the time that we run the commit command:

# In Bash, comments are marked with a #, just like Python
# Add any files that have changes and you wish to save in this      
# commit
$ git add main.py
# Commit the changes, add in your commit message with -m
$ git commit -m "Our first commit"

Now, at any point in the future, we can return to this point in our project. Adding files that are to be committed is called staging files in Git. Remember that you should only add stage files if you are ready to commit them. Once the files are staged, any further changes will not be staged. For an example of more advanced Git usage, add any text to your main.py file with your text editor and then run the following:

    # To see the changes from the last commit
    $ git diff
    # To see the history of your changes
    $ git log
    # As an example, we will stage main.py
    # and then remove any added files from the stage
    $ git add main.py
    $ git status
    $ git reset HEAD main.py
    # After any complicated changes, be sure to run status
    # to make sure everything went well
    $ git status
    # lets delete the changes to main.py, reverting to its state at the   
    # last commit # This can only be run on files that aren't staged
    $ git checkout -- main.py

Your terminal should look something like the following:

Note that in the preceding example I have modified the main.py file by adding the comment # Changed to show the git diff command.

One important step to include in every Git repository is a .gitignore file. This file tells Git what files to ignore. This way you can safely commit and add all your files. The following are some common files that you can ignore:

• Python's byte code files (*.pyc)
• Databases (specially for our examples using SQLLite database files) (*.db)
• Secrets (never push secrets (password, keys, and so on) to your repositories)
• IDE metadata files (.idea)
• The Virtualenv directory (env or venv)

Here's a simple example of a gitignore file:

*.pyc
*.pem
*.pub
*.tar.gz
*.zip
*.sql
*.db
secrets.txt
./tmp
./build/*
.idea/*
.idea
env
venv

Now we can safely add all the files to git and commit them:

  $ git add --all
  $ git status
  $ git commit -a -m "Added gitignore and all the projects missing 
    files"

The Git system's checkout command is rather advanced for this simple introduction, but it is used to change the current status of the Git system's HEAD pointer, which refers to the current location of our code in the history of our project. This will be shown in the next example.

Now, if we wish to see the code in a previous commit, we should first run the following command:

$ git log
commit cd88be37f12fb596be743ccba7e8283dd567ac05 (HEAD -> master)
Author: Daniel Gaspar
Date: Sun May 6 16:59:46 2018 +0100

Added gitignore and all the projects missing files
commit beb471198369e64a8ee8f6e602acc97250dce3cd
Author: Daniel Gaspar
Date: Fri May 4 19:06:57 2018 +0100

Our first commit

The string of characters next to our commit message, beb4711, is called the hash of our commit. It is the unique identifier of the commit that we can use to return to the saved state. Now, to take the project back to the previous state, run the following command:

$ git checkout beb4711

Your Git project is now in a special state where any changes or commits will neither be saved nor affect any commits that were made after the one you checked out. This state is just for viewing old code. To return to the normal mode of Git, run the following command:

$ git checkout master

Source control branches are an important feature that works great in team projects. A developer can create a new line of code from a specific point in time, revision, or tag. In this way, developing new features, creating releases, and making bugfixes or hotfixes can be done safely and subjected to team revision, and/or automatic integration tools (such as tests, code coverage, lint tools). A branch can be merged with other branches until it finally reaches the main line of code, called the master branch.

But let's get our hands on a practical exercise. Let's say that we want to develop a new feature. Our first chapter example displays the traditional "Hello World" message, but we want it to say "good morning" to the users. First, we create a branch from a special branch called the feature/good-morning that for now is a copy of the master branch, as shown in the following code:

# Display our branches
$ git branch
* master
# Create a branch called feature/good-morning from master
$ git branch feature/good-morning
# Display our branches again
$ git branch
  feature/good-morning
* master
# Check out the new feature/good-morning branch
$ git checkout feature/good-morning

This could be resumed to the following:

$ git checkout -b feature/good-morning master

Now let's change our code to display good morning to the visitors of a certain URL, along with their names. To do this, we change main.py, which looks like the following code:

@app.route('/')
def home():
    return '<h1>Hello world</h1>'

We change main.py to the following:

@app.route('/username')
def home():
    return '<h1>Good Morning %s</h1>' % username

Let's look at what we have done:

$ git diff
diff --git a/main.py b/main.py
index 3e0aacc..1a930d9 100755
--- a/main.py
+++ b/main.py
@@ -5,9 +5,9 @@ app = Flask(__name__)
 app.config.from_object(DevConfig)

 # Changed to show the git diff command
-@app.route('/')
-def home():
- return '<h1>Hello World!</h1>'
+@app.route('/<username>')
+def home(username):
+ return '<h1>Good Morning %s</h1>' % username

 if __name__ == '__main__':
     app.run()

Looks good. Let's commit, as shown in the following code:

$ git commit -m "Display good morning because its nice"
[feature/good-morning d4f7fb8] Display good morning because its nice
 1 file changed, 3 insertions(+), 3 deletions(-)

Now, if we were working as part of a team, or if our work was open source (or if we just wanted to back up our work), we should upload (push) our code to a centralized remote origin. One way of doing this is to push our code to a version control system, such as Bitbucket or GitHub, and then open a pull request to the master branch. This pull request will show our changes. As such, it may need approval from other team members, and many other features that these systems can provide.

For our example, let's just merge to the master, as shown in the following code:

# Get back to the master branch
$ git checkout master
Switched to branch 'master'
bash-3.2$ git log
commit 139d121d6ecc7508e1017f364e6eb2e4c5f57d83 (HEAD -> master)
Author: Daniel Gaspar
Date: Fri May 4 23:32:42 2018 +0100

    Our first commit
# Merge our feature into the master branch
$ git merge feature/good-morning
Updating 139d121..5d44a43
Fast-forward
 main.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
bash-3.2$ git log
commit 5d44a4380200f374c879ec1f7bda055f31243263 (HEAD -> master, feature/good-morning)
Author: Daniel Gaspar
Date: Fri May 4 23:34:06 2018 +0100

Display good morning because its nice

commit 139d121d6ecc7508e1017f364e6eb2e4c5f57d83
Author: Daniel Gaspar <daniel.gaspar@miniclip.com>
Date: Fri May 4 23:32:42 2018 +0100

Our first commit

As you can see from the output, Git uses the fast-forward strategy by default. If we wanted to keep an extra commit log message that mentions the merge itself, then we could have used the --no-ff flag on the git merge command. This flag will disable the fast-forward merging strategy.

Now imagine that we regret our change and want to revert the feature that we have just created back to an earlier version. To do this, we can use the following code:

$ git revert

With Git, you can actually delete your commits, but this is considered a really bad practice. Note that the revert command did not delete our merge, but created a new commit with the reverted changes. It's considered a good practice not to rewrite the past.

What was shown is a feature branch simple workflow. With big teams or projects, the use of more complex workflows is normally adopted to better isolate features, fixes, and releases, and to keep a stable line of code. This is what is proposed when using the git-flow process.

Now that we have a version control system, we are ready to cover Python's package management system.

In Python, programmers can download libraries from other programmers that extend the functionality of the standard Python library. As you already know from using Flask, a lot of Python's power comes from its large number of community-created libraries.

However, installing third-party libraries can be a huge pain to do correctly. Say that you want to install package X. Simple enough: download the ZIP file and run setup.py, right? Not quite. Package X relies on package Y, which in turn relies on Z and Q. None of this information was listed on package X's website, but these packages need to be installed for X to work at all. You then have to find all of the packages one by one and install them, and then hope that the packages you are installing don't require any extra packages themselves.

In order to automate this process, we use pip, the Python package manager.

If you are using Windows, and your previously installed version of Python is the current version, then you already have pip! If your Python installation is not the most recent version, then the easiest thing to do is to simply reinstall it. Download the Python Windows installer at https://www.python.org/downloads/.

In Windows, the variable that controls which programs are accessible from the command line is the path. To modify our path to include Python and pip, we have to add C:\Python27 and C:\Python27\Tools. Edit the Windows path by opening the Windows menu, right-clicking on Computer, and clicking on Properties. Under Advanced system settings, click Environment Variables.... Scroll down until you find Path, double-click on it, and add ;C:\Python27;C:\Python27\Tools to the end.

To make sure that you have modified your path correctly, close and reopen your Terminal and type the following into the command line:

pip --help

Pip should have printed its usage message, as shown in the following screenshot:

Some Python installations of Linux do not come with pip, and Mac OS X's installations doesn't come with pip by default. If you are using Python 2.7, then you may need to install pip, but pip is already included in Python 3.4, and in later versions. You can check this using the following:

$ python3 -m pip list

If you need to install it, download the get-pip.py file from https://bootstrap.pypa.io/get-pip.py.

Once you have downloaded it, run it with elevated privileges using the following code:

# Download and install pip
$ wget https://bootstrap.pypa.io/get-pip.py    
$ sudo python get-pip.py

Once this has been entered, pip will be installed automatically.

We are now going to learn the basic commands for using Python package manager. To install a package with pip, enter the following code:

$ pip install [package-name]

On Mac and Linux, because you are installing programs outside of the user-owned folders, you might have to prepend sudo to the install commands. To install Flask, simply run the following:

$ pip install flask

Once you have done this, all of the requirements that you need for using Flask will be installed for you.

If you want to remove a package that you are no longer using, run the following:

$ pip uninstall [package-name]

If you wish to explore or find a package, but don't know its exact name, you can use the search command:

$ pip search [search-term]

Now that we have a couple of packages installed, it is common courtesy in the Python community to create a list of packages that are required to run the project so that others can quickly install every necessary package. This also has the added benefit that any new member of your project will be able to run your code quickly.

This list can be created with pip by running the following command:

$ pip freeze > requirements.txt

What exactly did this command do? The pip freeze command automatically prints out a list of the installed packages and their versions. For our example, it prints the following:

click==6.7
Flask==0.12.4
itsdangerous==0.24
Jinja2==2.10
MarkupSafe==1.0
Werkzeug==0.14.1

The > operator tells Bash to take everything printed by the last command and write it to this file. If you look in your project directory, you can see a new file named requirements.txt that contains the output of pip freeze.

To install all the packages from this file, a new project maintainer would have to run this, as shown in the following code. Normally, this will also be used to deploy the production environment of your project:

$ pip install -r requirements.txt

The preceding code tells pip to read all the packages listed in requirements.txt and install them.

So you have installed all the packages that you want for your new project. Great! But what happens when we develop a second project some time later that will use newer versions of the same packages? And what happens when a library that you wish to use depends on a library that you installed for the first project, but which uses an older version of these packages? When newer versions of packages contain breaking changes, upgrading them would require extra development work on an older project that you may not be able to afford. So in our system, we could have clashing Python packages between projects.

We should also consider automated build environments, such as Jenkins, where we want to run tests. These builds may run on the same system on which other projects are being built, so it's essential that during the build jobs we create a contained Python package environment that is not shared between jobs. This environment is created from the information in the requirements.txt file that we created earlier. This way, multiple Python applications can be built and tested on the same system without clashing with each other.

Thankfully, there is virtualenv, a tool that sandboxes your Python projects. The secret to virtualenv is in tricking your computer to look for and install packages in the project directory rather than in the main Python directory, which allows you to keep them completely separate.

If you're using Python 3—and I recommend that you do, because Python 2 support will end in 2020—then you don't have to install virtualenv; you can use it just by running it like a package, as shown in the following code:

# Create a python 3 virtualenv
$ python3 -m venv env

Now that we have pip, if we need to install virtualenv, then we can just run the following command:

$ pip install virtualenv

Let's initialize virtualenv for our project, as follows:

$ virtualenv env

The extra env tells virtualenv to store all the packages in a folder named env. Virtualenv requires you to start it before it will sandbox your project. You can do this using the following code:

$ source env/bin/activate
# Your prompt should now look like
(env) $

The source command tells Bash to run the env/bin/activate script in the context of the current directory. Let's reinstall Flask in our new sandbox, as follows:

# you won't need sudo anymore
(env) $ pip install flask
# To return to the global Python
(env) $ deactivate

Your development projects normally need more then a web server application layer; you will most definitely need some kind of database system. You might be using a cache, redis, workers with Celery, a messaging queuing system, or something else. Normally, all of the systems that are needed for your application to work are collectively referred to as stack. One simple way to easily define and quickly spawn all these components is to use Docker containers. With Docker, you define all of your application components and how to install and configure them, and you can then share your stack with your team, and send it to production with the exact same specification.

You can download and install Docker from https://docs.docker.com/install/.

First, let's create a very simple Dockerfile. This file defines how to set up your application. Each line will serve as a container layer for very fast rebuilds. A very simple Dockerfile will look like the following:

FROM python:3.6.5
# Set the working directory to /app
WORKDIR /app
# Copy local contents into the container
ADD . /app
# Install all required dependencies
RUN pip install -r requirements.txt
EXPOSE 5000
CMD ["python", "main.py"]

Next, let's build out first container image. We will tag it as chapter_1 for further ease of use, as shown in the following code:

$ docker build -t chapter_1 .

Then we will run it, as shown in the following code:

$ docker run -p 5000:5000 chapter_1
# List all the running containers
$ docker container list

Docker is easy, but it's a complex tool with lots of options for configuring and deploying containers. We will look at Docker in more detail in Chapter 13, Deploying Flask Apps.

Finally, we can get to our first Flask project. In order to build a complex project at the end of this book, we will need a simple Flask project to start us off.

Flask is very powerful, but will most definitely not get in your way. You can use it to create a simple web application using a single file. Our aim is to create a project that is structured in a way that it can scale and be easy to understand. For now, we will create a config file first. In the file named config.py, add the following:

class Config(object): 
    pass 
 
class ProdConfig(Config): 
    pass 
 
class DevConfig(Config): 
    DEBUG = True 

Now, in another file named main.py, add the following:

from flask import Flask 
from config import DevConfig 
 
app = Flask(__name__) 
app.config.from_object(DevConfig) 
 
@app.route('/') 
def home(): 
    return '<h1>Hello World!</h1>' 
 
if __name__ == '__main__': 
    app.run() 

For anyone who is familiar with the base Flask API, this program is very basic. It will simply show Hello World! on the browser if we navigate to http://127.0.0.1:5000. One point that may be unfamiliar to Flask users is the use of the phrase config.from_object rather than app.config['DEBUG']. We use from_object because in future, multiple configurations will be used, and manually changing every variable when we need to switch between configurations is time consuming.

We have created a very simple project structure, but can it serve as the base skeleton for any Python project. In Chapter 5, Advanced Application Structure, we will get our hands on a more scalable structure, but for now, let's go back to our environment, as shown in the following code:

Dockerfile # Instructions to configure and run our application on a container
requirements.txt # All the dependencies needed to run our application
/venv # We will not add this folder to our Git repo, our virtualenv
.gitignore # Instruction for Git to ignore files
main.py # Our main Flask application
config.py # Our configuration file

Remember to commit these changes in Git, as shown in the following code:

# The --all flag will tell git to stage all changes you have made
# including deletions and new files
$ git add --all
$ git commit -m" ""created the base application"

In order to make the next chapters easier for the reader, we will look at how to use the Flask CLI (using version 0.11 onward). The CLI allows programmers to create commands that act within the application context of Flask—that is, the state in Flask that allows the modification of the Flask object. The Flask CLI comes with some default commands to run the server and a Python shell in the application context.

Let's take a look at the Flask CLI and how to initialize it. First, we must tell it how to discover our application using the following code:

$ export FLASK_APP=main.py

Then, we will use the Flask CLI to run our application using the following code:

$ flask run

Now, let's enter the shell on the application context and see how to get all the defined URL routes, using the following code:

$ flask shell
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 03:03:55)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
App: main [debug]
Instance: /chapter_1/instance
>>> app.url_map
Map([<Rule '/' (OPTIONS, GET, HEAD) -> home>,
 <Rule '/static/<filename>' (OPTIONS, GET, HEAD) -> static>])

As you can see, we already have two routes defined: the / where we display the "Hello World" sentence and the static default route created by Flask. Some other useful information shows where Flask thinks our templates and static folders are, as shown in the following code:

>>> app.static_folder
/chapter_1/static'
>>> app.template_folder
'templates'

Flask CLI, uses the click library from the creator of Flask itself. It was designed to be easily extensible so that the Flask extensions can extend it and implement new commands that are available when you use them. We should indeed extend it—it makes it more useful to extend it ourselves. This is the right way to create management commands for our applications. Think about commands that you can use to migrate database schemas, create users, prune data, and so on.

Now that we have set up our development environment, we can move on to implementing advanced application features in Flask. Before we can do anything visual, we need content to display. This content will be kept on a database. In the next chapter, you will be introduced to working with databases in Flask, and you will learn how master them.