Django 3 Web Development Cookbook - Fourth Edition

2 (1 reviews total)
By Aidas Bendoraitis , Jake Kronika
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Getting Started with Django 3.0

About this book

Django is a web framework for perfectionists with deadlines, designed to help you build manageable medium and large web projects in a short time span. This fourth edition of the Django Web Development Cookbook is updated with Django 3's latest features to guide you effectively through the development process.

This Django book starts by helping you create a virtual environment and project structure for building Python web apps. You'll learn how to build models, views, forms, and templates for your web apps and then integrate JavaScript in your Django apps to add more features. As you advance, you'll create responsive multilingual websites, ready to be shared on social networks. The book will take you through uploading and processing images, rendering data in HTML5, PDF, and Excel, using and creating APIs, and navigating different data types in Django. You'll become well-versed in security best practices and caching techniques to enhance your website's security and speed. This edition not only helps you work with the PostgreSQL database but also the MySQL database. You'll also discover advanced recipes for using Django with Docker and Ansible in development, staging, and production environments.

By the end of this book, you will have become proficient in using Django's powerful features and will be equipped to create robust websites.

Publication date:
March 2020
Publisher
Packt
Pages
608
ISBN
9781838987428

 

Getting Started with Django 3.0

In this chapter, we will cover the following topics:

  • Working with a virtual environment
  • Creating a project file structure
  • Handling project dependencies with pip
  • Configuring settings for development, testing, staging, and production environments
  • Defining relative paths in the settings
  • Handling sensitive settings
  • Including external dependencies in your project
  • Setting up STATIC_URL dynamically
  • Setting UTF-8 as the default encoding for the MySQL configuration
  • Creating the Git ignore file
  • Deleting Python-compiled files
  • Respecting the import order in Python files
  • Creating an app configuration
  • Defining overwritable app settings
  • Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL
 

Introduction

In this chapter, we will see a few valuable practices to follow when starting a new project with Django 3.0 using Python 3. We have picked the most useful ways to deal with scalable project layout, settings, and configurations, whether using virtualenv or Docker to manage your project.

We are assuming that you are already familiar with the basics of Django, Git version control, MySQL as well as PostgreSQL databases, and command-line usage. We also assume that you are using a Unix-based operating system, such as macOS or Linux. It makes more sense to develop with Django on Unix-based platforms as the Django websites will most likely be published on a Linux server, meaning that you can establish routines that work in the same way, whether you're developing or deploying. If you are locally working with Django on Windows, the routines are similar; however, they are not always the same.

Using Docker for your development environment, regardless of your local platform, can improve the portability of your applications through deployment since the environment within the Docker container can be matched precisely to that of your deployment server. We should also mention that for the recipes in this chapter, we are assuming that you have the appropriate version control system and database server already installed on your local machine, whether you are developing with Docker or not.

 

 

Technical requirements

To work with the code of this book, you will need the latest stable version of Python, which can be downloaded from https://www.python.org/downloads/. At the time of writing, the latest version is 3.8.X. You will also need a MySQL or PostgreSQL database. You can download the MySQL database server from https://dev.mysql.com/downloads/. The PostgreSQL database server can be downloaded from https://www.postgresql.org/download/. Other requirements will be requested in specific recipes.

You can find all the code for this chapter at the ch01 directory of the GitHub repository at https://github.com/PacktPublishing/Django-3-Web-Development-Cookbook-Fourth-Edition.

 

Working with a virtual environment

It is very likely that you will develop multiple Django projects on your computer. Some modules, such as virtualenv, setuptools, wheel, or Ansible, can be installed once and then shared for all projects. Other modules, such as Django, third-party Python libraries, and Django apps, will need to be kept isolated from each other. The virtualenv tool is a utility that separates all of the Python projects and keeps them in their own realms. In this recipe, we will see how to use it.

Getting ready

To manage Python packages, you will need pip. If you are using Python 3.4+, then it will be included in your Python installation. If you are using another version of Python, you can install pip by executing the installation instructions at http:/​/​pip.​readthedocs.​org/​en/​stable/installing/​. Let's upgrade the shared Python modules, pip, setuptools, and wheel:

$ sudo pip3 install --upgrade pip setuptools wheel

The virtual environment has been built into Python since version 3.3.

How to do it...

Once you have your prerequisites installed, create a directory where all your Django projects will be stored—for example, projects under your home directory. Go through the following steps after creating the directory:

  1. Go to the newly created directory and create a virtual environment that uses the shared system site packages:
$ cd ~/projects
$ mkdir myproject_website
$ cd myproject_website
$ python3 -m venv env
  1. To use your newly created virtual environment, you need to execute the activation script in your current shell. This can be done with the following command:
$ source env/bin/activate
  1. Depending on the shell you are using, the source command may not be available. Another way to source a file is with the following command, which has the same result (note the space between the dot and env):
$ . env/bin/activate
  1. You will see that the prompt of the command-line tool gets a prefix of the project name, as follows:
(env)$
  1. To get out of the virtual environment, type the following command:
(env)$ deactivate

How it works...

When you create a virtual environment, a few specific directories (bin, include, and lib) are created in order to store a copy of the Python installation, and some shared Python paths are defined. When the virtual environment is activated, whatever you install with pip or easy_install will be put in and used by the site packages of the virtual environment, and not the global site packages of your Python installation.

To install the latest Django 3.0.x in your virtual environment, type the following command:

(env)$ pip install "Django~=3.0.0"

 

See also

  • The Creating a project file structure recipe
  • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
  • The Deploying on Apache with mod_wsgi for the staging environment recipe in Chapter 12, Deployment
  • The Deploying on Apache with mod_wsgi for the production environment recipe in Chapter 12, Deployment
  • The Deploying on Nginx and Gunicorn for the staging environment recipe in Chapter 12, Deployment
  • The Deploying on Nginx and Gunicorn for the production environment recipe in Chapter 12, Deployment
 

Creating a project file structure

A consistent file structure for your projects makes you well organized and more productive. When you have the basic workflow defined, you can get stuck into the business logic more quickly and create awesome projects.

Getting ready

If you haven't done it yet, create a ~/projects directory, where you will keep all your Django projects (you can read about this in the Working with a virtual environment recipe).

Then, create a directory for your specific project—for example, myproject_website. Start the virtual environment in an env directory there. Activate it and install Django there, as described in the previous recipe. We would suggest adding a commands directory for local shell scripts that are related to the project, a db_backups directory for database dumps, a mockups directory for website design files, and, most importantly, an src directory for your Django project.

How to do it...

Follow these steps to create a file structure for your project:

  1. With the virtual environment activated, go to the src directory and start a new Django project, as follows:
(env)$ django-admin.py startproject myproject

The executed command will create a directory called myproject, with project files inside. This directory will contain a Python module, also called myproject. For clarity and convenience, we will rename the top-level directory as django-myproject. It is the directory that you will put under version control, and so it will have a .git or similarly named subdirectory.

  1. In the django-myproject directory, create a README.md file to describe your project to the new developdjango-admin.py startproject myprojecters.
  2. The django-myproject directory will also contain the following:
  • Your project's Python package, named myproject.
  • Your project's pip requirements with the Django framework and other external dependencies (read about this in the Handling project dependencies with pip recipe).
  • The project license in a LICENSE file. If your project is open source, you can choose one of the most popular licenses from https://choosealicense.com.
    1. In your project's root, django-myproject, create the following:
    • A media directory for project uploads
    • A static directory for collected static files
    • A locale directory for project translations
    • An externals directory for external dependencies that are included in this project when you can't use the pip requirements
    1. The myproject directory should contain these directories and files:
    • The apps directory where you will put all your in-house Django apps for the project. It is recommended that you have one app called core or utils for the projects' shared functionality.
    • The settings directory for your project settings (read about this in the Configuring settings for development, testing, staging, and production environments recipe).
    • The site_static directory for project-specific static files.
    • The templates directory for the project's HTML templates.
    • The urls.py file for the project's URL configuration.
    • The wsgi.py file for the project's web server configuration.
    1. In your site_static directory, create the site directory as a namespace for site-specific static files. Then, we will divide the static files between the categorized subdirectories within it. For instance, see the following:
    • scss for Sass files (optional)
    • css for the generated minified Cascading Style Sheets (CSS)
    • img for styling images, favicons, and logos
    • js for the project's JavaScriptdjango-admin.py startproject myproject
    • vendor for any third-party module combining all types of files, such as the TinyMCE rich-text editor
    1. Besides the site directory, the site_static directory might also contain overwritten static directories of third-party apps—for example, it might contain cms, which overwrites the static files from Django CMS. To generate the CSS files from Sass and minify the JavaScript files, you can use the CodeKit (https://codekitapp.com/) or Prepros (https://prepros.io/) applications with a graphical user interface.
    1. Put your templates that are separated by the apps in your templates directory. If a template file represents a page (for example, change_item.html or item_list.html), then put it directly in the app's template directory. If the template is included in another template (for example, similar_items.html), put it in the includes subdirectory. Also, your templates directory can contain a directory called utils for globally reusable snippets, such as pagination and the language chooser.

    How it works...

    The whole file structure for a complete project will look similar to the following:

    myproject_website/
    ├── commands/
    ├── db_backups/
    ├── mockups/
    ├── src/
    │ └── django-myproject/
    │ ├── externals/
    │ │ ├── apps/
    │ │ │ └── README.md
    │ │ └── libs/
    │ │ └── README.md
    │ ├── locale/
    │ ├── media/
    │ ├── myproject/
    │ │ ├── apps/
    │ │ │ ├── core/
    │ │ │ │ ├── __init__.py
    │ │ │ │ └── versioning.py
    │ │ │ └── __init__.py
    │ │ ├── settings/
    │ │ │ ├── __init__.py
    │ │ │ ├── _base.py
    │ │ │ ├── dev.py
    │ │ │ ├── production.py
    │ │ │ ├── sample_secrets.json
    │ │ │ ├── secrets.json
    │ │ │ ├── staging.py
    │ │ │ └── test.py
    │ │ ├── site_static/
    │ │ │ └── site/
    │ │ │ django-admin.py startproject myproject ├── css/
    │ │ │ │ └── style.css
    │ │ │ ├── img/
    │ │ │ │ ├── favicon-16x16.png
    │ │ │ │ ├── favicon-32x32.png
    │ │ │ │ └── favicon.ico
    │ │ │ ├── js/
    │ │ │ │ └── main.js
    │ │ │ └── scss/
    │ │ │ └── style.scss
    │ │ ├── templates/
    │ │ │ ├── base.html
    │ │ │ └── index.html
    │ │ ├── __init__.py
    │ │ ├── urls.py
    │ │ └── wsgi.py
    │ ├── requirements/
    │ │ ├── _base.txt
    │ │ ├── dev.txt
    │ │ ├── production.txt
    │ │ ├── staging.txt
    │ │ └── test.txt
    │ ├── static/
    │ ├── LICENSE
    │ └── manage.py
    └── env/

    There's more...

    To speed up the creation of a project in the way we just described, you can use the project's boilerplate from https://github.com/archatas/django-myproject. After downloading the code, perform a global search and replace myproject with a meaningful name for your project, and you should be good to go.

    See also

    • The Handling project dependencies with pip recipe
    • The Including external dependencies in your project recipe
    • The Configuring settings for development, testing, staging, and production environments recipe
    • The Deploying on Apache with mod_wsgi for the staging environment recipe in Chapter 12, Deployment
    • The Deploying on Apache with mod_wsgi for the production environment recipe in Chapter 12, Deployment
    • The Deploying on Nginx and Gunicorn for the staging environment recipe in Chapter 12, Deployment
    • The Deploying on Nginx and Gunicorn for the production environment recipe in Chapter 12, Deployment
     

    Handling project dependencies with pip

    The most convenient tool to install and manage Python packages is pip. Rather than installing the packages one by one, it is possible to define a list of packages that you want to install as the contents of a text file. We can pass the text file into the pip tool, which will then handle the installation of all packages in the list automatically. An added benefit to this approach is that the package list can be stored in version control.

    Generally speaking, it is ideal and often sufficient to have a single requirements file that directly matches your production environment. You can change versions or add and remove dependencies on a development machine and then manage them through version control. This way, going from one set of dependencies (and associated code changes) to another can be as simple as switching branches.

    In some cases, environments differ enough that you will need to have at least two different instances of your project:

    • The development environment, where you create new features
    • The public website environment, which is usually called the production environment in a hosted server

    There might be development environments for other developers, or special tools that are needed during development but that are unnecessary in production. You might also have a testing and staging environment in order to test the project locally and in a public website-like setup.

    For good maintainability, you should be able to install the required Python modules for development, testing, staging, and production environments. Some of the modules will be shared and some of them will be specific to a subset of the environments. In this recipe, we will learn how to organize the project dependencies for multiple environments and manage them with pip.

    Getting ready

    Before using this recipe, you need to have a Django project ready with pip installed and a virtual environment activated. For more information on how to do this, read the Working with a virtual environment recipe.

    How to do it...

    Execute the following steps one by one to prepare pip requirements for your virtual environment Django project:

    1. Let's go to the Django project that you have under version control and create a requirements directory with the following text files:
    • _base.txt for shared modules
    • dev.txt for the development environment
    • test.txt for the testing environment
    • staging.txt for the staging environment
    • production.txt for production
    1. Edit _base.txt and add the Python modules that are shared in all environments, line by line:
    # requirements/_base.txt
    Django~=3.0.4
    djangorestframework
    -e git://github.com/omab/python-social-[email protected]#egg=python-social-auth
    1. If the requirements of a specific environment are the same as in _base.txt, add the line including _base.txt in the requirements file of that environment, as shown in the following example:

    # requirements/production.txt
    -r _base.txt
    1. If there are specific requirements for an environment, add them after the _base.txt inclusion, as shown in the following code:
    # requirements/dev.txt
    -r _base.txt
    coverage
    django-debug-toolbar
    selenium
    1. You can run the following command in a virtual environment in order to install all of the required dependencies for the development environment (or an analogous command for other environments), as follows:

    (env)$ pip install -r requirements/dev.txt

    How it works...

    The preceding pip install command, whether it is executed explicitly in a virtual environment or at the global level, downloads and installs all of your project dependencies from requirements/_base.txt and requirements/dev.txt. As you can see, you can specify a version of the module that you need for the Django framework and even directly install it from a specific commit at the Git repository, as is done for python-social-auth in our example.

    When you have many dependencies in your project, it is good practice to stick to a narrow range of release versions for Python module release versions. Then you can have greater confidence that the project integrity will not be broken because of updates in your dependencies, which might contain conflicts or backward incompatibility. This is particularly important when deploying your project or handing it off to a new developer.

    If you have already manually installed the project requirements with pip one by one, you can generate the requirements/_base.txt file using the following command within your virtual environment:

    (env)$ pip freeze > requirements/_base.txt

    There's more...

    If you want to keep things simple and are sure that, for all environments, you will be using the same dependencies, you can use just one file for your requirements named requirements.txt, generated by definition, as shown in the following:

    (env)$ pip freeze > requirements.txt

    To install the modules in a new virtual environment, simply use the following command:

    (env)$ pip install -r requirements.txt

    If you need to install a Python library from another version control system, or on a local path, then you can learn more about pip from the official documentation at https://pip.pypa.io/en/stable/user_guide/.

    Another approach to managing Python dependencies that is getting more and more popular is Pipenv. You can get it and learn about it at https://github.com/pypa/pipenv

    See also

    • The Working with a virtual environment recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
    • The Including external dependencies in your project recipe
    • The Configuring settings for development, testing, staging, and production environments recipe
     

    Configuring settings for development, testing, staging, and production environments

    As noted earlier, you will be creating new features in the development environment, testing them in the testing environment, and then putting the website onto a staging server to let other people try the new features. Then, the website will be deployed to the production server for public access. Each of these environments can have specific settings, and you will learn how to organize them in this recipe.

    Getting ready

    In a Django project, we'll create settings for each environment: development, testing, staging, and production.

    How to do it...

    Follow these steps to configure the project settings:

    1. In the myproject directory, create a settings Python module with the following files:
    • __init__.py makes the settings directory a Python module.
    • _base.py for shared settings
    • dev.py for development settings
    • test.py for testing settings
    • staging.py for staging settings
    • production.py for production settings
    1. Copy the contents of settings.py, which was automatically created when you started a new Django project, to settings/_base.py. Then, delete settings.py.
    2. Change the BASE_DIR in the settings/_base.py to point one level up. It should first look as follows:
    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    After changing it, it should look like the following:

    BASE_DIR = os.path.dirname(
    os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    )
    1. If the settings of an environment are the same as the shared settings, then just
      import everything from _base.py there, as follows:
    # myproject/settings/production.py
    from ._base import *
    1. Apply the settings that you want to attach or overwrite for your specific environment in the other files—for example, the development environment settings should go to dev.py, as shown in the following code snippet:
    # myproject/settings/dev.py
    from ._base import *
    EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"

    1. Modify the manage.py and myproject/wsgi.py files to use one of the environment settings by default by changing the following line:
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
    1.  You should change this line to the following:
    os.environ.setdefault('DJANGO_SETTINGS_MODULE',  'myproject.settings.production')

    How it works...

    By default, the Django management commands use the settings from myproject/settings.py. Using the method that is defined in this recipe, we can keep all of the required nonsensitive settings for all environments under version control in the config directory. On the other hand, the settings.py file itself would be ignored by version control and will only contain the settings that are necessary for the current development, testing, staging, or production environments.

    For each environment, it is recommended that you set the DJANGO_SETTINGS_MODULE environment variable individually, either in PyCharm settings, the env/bin/activate script, or in .bash_profile.

    See also

    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
    • The Handling sensitive settings recipe
    • The Defining relative paths in the settings recipe
    • The Creating a Git ignore file recipe
     

    Defining relative paths in the settings

    Django requires you to define different file paths in the settings, such as the root of your media, the root of your static files, the path to templates, and the path to translation files. For each developer of your project, the paths may differ as the virtual environment can be set up anywhere and the user might be working on macOS, Linux, or Windows. Even when your project is wrapped in a Docker container, it reduces the maintainability and portability to define absolute paths. In any case, there is a way to define these paths dynamically so that they are relative to your Django project directory.

    Getting ready

    Have a Django project started and open settings/_base.py.

    How to do it...

    Modify your path-related settings accordingly, instead of hardcoding the paths to your local directories, as follows:

    # settings/_base.py
    import os
    BASE_DIR = os.path.dirname(
    os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    )
    # ...
    TEMPLATES = [{
    # ...
    DIRS: [
    os.path.join(BASE_DIR, 'myproject', 'templates'),
    ],
    # ...
    }]
    # ...
    LOCALE_PATHS = [
    os.path.join(BASE_DIR, 'locale'),
    ]
    # ...
    STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'myproject', 'site_static'),
    ]
    STATIC_ROOT = os.path.join(BASE_DIR, 'static')
    MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

    How it works...

    By default, Django settings include a BASE_DIR value, which is an absolute path to the directory containing manage.py (usually one level higher than the settings.py file or two levels higher than settings/_base.py). Then, we set all of the paths relative to BASE_DIR using the os.path.join() function.

    Based on the directory layout we set down in the Creating a project file structure recipe, we would insert 'myproject' as an intermediary path segment for some of the previous examples since the associated folders were created within this.

    See also

    • The Creating a project file structure recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
    • The Including external dependencies in your project recipe
     

    Handling sensitive settings

    When working when configuring a Django project, you will surely deal with some sensitive information, such as passwords and API keys. It is not recommended that you put that information under version control. There are two main ways to store that information: in environment variables and in separate untracked files. In this recipe, we will explore both cases.

    Getting ready

    Most of the settings for a project will be shared across all environments and saved in version control. These can be defined directly within the settings files; however, there will be some settings that are specific to the environment of the project instance or that are sensitive and require additional security, such as database or email settings. We will expose these using environment variables.

    How to do it...

    To read sensitive settings from the environment variables, perform these steps:

    1. At the beginning of settings/_base.py, define the get_secret() function as follows:
    # settings/_base.py
    import os
    from django.core.exceptions import ImproperlyConfigured

    def get_secret(setting):
    """Get the secret variable or return explicit exception."""
    try:
    return os.environ[setting]
    except KeyError:
    error_msg = f'Set the {setting} environment variable'
    raise ImproperlyConfigured(error_msg)
    1. Then, whenever you need to define a sensitive value, use the get_secret() function, as shown in the following example:
    SECRET_KEY = get_secret('DJANGO_SECRET_KEY')

    DATABASES = {
    'default': {
    'ENGINE': 'django.db.backends.postgresql_psycopg2',
    'NAME': get_secret('DATABASE_NAME'),
    'USER': get_secret('DATABASE_USER'),
    'PASSWORD': get_secret('DATABASE_PASSWORD'),
    'HOST': 'db',
    'PORT': '5432',
    }
    }

    How it works...

    If you run a Django management command without the environment variable set, you will see an error raised with a message, such as Set the DJANGO_SECRET_KEY environment variable.

    You can set the environment variables in the PyCharm configuration, remote server configuration consoles, in the env/bin/activate script, .bash_profile, or directly in the Terminal like this:

    $ export DJANGO_SECRET_KEY="change-this-to-50-characters-long-random-
    string"

    $ export DATABASE_NAME="myproject"
    $ export DATABASE_USER="myproject"
    $ export DATABASE_PASSWORD="change-this-to-database-password"

    Note that you should use the get_secret() function for all passwords, API keys, and any other sensitive information that you need in your Django project configuration.

    There's more...

    Instead of environment variables, you can also use text files with sensitive information that won't be tracked under version control. They can be YAML, INI, CSV, or JSON files, placed somewhere on the hard disk. For example, for a JSON file, you would have the get_secret() function, like this:

    # settings/_base.py
    import os
    import json


    with open(os.path.join(os.path.dirname(__file__), 'secrets.json'), 'r')
    as f:
    secrets = json.loads(f.read())


    def get_secret(setting):
    """Get the secret variable or return explicit exception."""
    try:
    return secrets[setting]
    except KeyError:
    error_msg = f'Set the {setting} secret variable'
    raise ImproperlyConfigured(error_msg)

    This reads a secrets.json file from the settings directory and expects it to have at least the following structure:

    {
    "DATABASE_NAME": "myproject",
    "DATABASE_USER": "myproject",
    "DATABASE_PASSWORD": "change-this-to-database-password",
    "DJANGO_SECRET_KEY": "change-this-to-50-characters-long-random-string"
    }

    Make sure that the secrets.json file is ignored from the version control, but for convenience, you can create sample_secrets.json with empty values and put it under version control:

    {
    "DATABASE_NAME": "",
    "DATABASE_USER": "",
    "DATABASE_PASSWORD": "",
    "DJANGO_SECRET_KEY": "change-this-to-50-characters-long-random-string"
    }

    See also

    • The Creating a project file structure recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
     

    Including external dependencies in your project

    Sometimes, you can't install an external dependency with pip and have to include it directly within your project, such as in the following cases:

    • When you have a patched third-party app where you yourself fixed a bug or added a feature that did not get accepted by project owners
    • When you need to use private apps that are not accessible at the Python Package Index (PyPI) or public version control repositories
    • When you need to use legacy versions of dependencies that are not available at PyPI anymore

    Including external dependencies in your project ensures that whenever a developer upgrades the dependent modules, all of the other developers will receive the upgraded version in the next update from the version control system.

    Getting ready

    You should start with a Django project under a virtual environment.

    How to do it...

    Execute the following steps one by one for a virtual environment project:

    1. If you haven't done so already, create an externals directory under your Django project directory, django-myproject.
    2. Then, create the libs and apps directories under it. The libs directory is for the Python modules that are required by your project—for example, Boto, Requests, Twython, and Whoosh. The apps directory is for third-party Django apps—for example, Django CMS, Django Haystack, and django-storages.
      We highly recommend that you create README.md files in the libs and apps directories, where you mention what each module is for, what the used version or revision is, and where it is taken from.
    3. The directory structure should look something similar to the following:
    externals/
    ├── apps/
    │ ├── cms/
    │ ├── haystack/
    │ ├── storages/
    │ └── README.md
    └── libs/
    ├── boto/
    ├── requests/
    ├── twython/
    └── README.md
    1. The next step is to put the external libraries and apps under the Python path so that they are recognized as if they were installed. This can be done by adding the following code in the settings:
    # settings/_base.py
    import os
    import sys
    BASE_DIR = os.path.dirname(
    os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
    )
    EXTERNAL_BASE = os.path.join(BASE_DIR, "externals")
    EXTERNAL_LIBS_PATH = os.path.join(EXTERNAL_BASE, "libs")
    EXTERNAL_APPS_PATH = os.path.join(EXTERNAL_BASE, "apps")
    sys.path = ["", EXTERNAL_LIBS_PATH, EXTERNAL_APPS_PATH] + sys.path

    How it works...

    A module is meant to be under the Python path if you can run Python and import that module. One of the ways to put a module under the Python path is to modify the sys.path variable before importing a module that is in an unusual location. The value of sys.path, as specified by the settings file, is a list of directories starting with an empty string for the current directory, followed by the directories in the project, and finally the globally shared directories of the Python installation. You can see the value of sys.path in the Python shell, as follows:

    (env)$ python manage.py shell
    >>> import sys
    >>> sys.path

    When trying to import a module, Python searches for the module in this list and returns the first result that is found.

    Therefore, we first define the BASE_DIR variable, which is the absolute path of django-myproject or three levels higher than myproject/settings/_base.py. Then, we define the EXTERNAL_LIBS_PATH and EXTERNAL_APPS_PATH variables, which are relative to BASE_DIR. Lastly, we modify the sys.path property, adding new paths to the beginning of the list. Note that we also add an empty string as the first path to search, which means that the current directory of any module should always be checked first before checking other Python paths.

    This way of including external libraries doesn't work cross-platform with the Python packages that have C language bindings—for example, lxml. For such dependencies, we would recommend using the pip requirements that were introduced in the Handling project dependencies with pip recipe.

    See also

    • The Creating a project file structure recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
    • The Handling project dependencies with pip recipe
    • The Defining relative paths in the settings recipe
    • The Using the Django shell recipe in Chapter 10, Bells and Whistles
     

    Setting up STATIC_URL dynamically

    If you set STATIC_URL to a static value, then each time you update a CSS file, a JavaScript file, or an image, you and your website visitors will need to clear the browser cache in order to see the changes. There is a trick to work around clearing the browser's cache. It is to have the timestamp of the latest changes shown in STATIC_URL. Whenever the code is updated, the visitor's browser will force the loading of all new static files.

    In this recipe, we will see how to put a timestamp in STATIC_URL for Git users.

    Getting ready

    Make sure that your project is under Git version control and that you have BASE_DIR defined in your settings, as shown in the Defining relative paths in the settings recipe.

    How to do it...

    The procedure to put the Git timestamp in the STATIC_URL setting consists of the following two steps:

    1. If you haven't done so yet, create the myproject.apps.core app in your Django project. You should also create a versioning.py file there:
    # versioning.py
    import
    subprocess
    from datetime import datetime


    def get_git_changeset_timestamp(absolute_path):
    repo_dir = absolute_path
    git_log = subprocess.Popen(
    "git log --pretty=format:%ct --quiet -1 HEAD",
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    shell=True,
    cwd=repo_dir,
    universal_newlines=True,
    )

    timestamp = git_log.communicate()[0]
    try:
    timestamp = datetime.utcfromtimestamp(int(timestamp))
    except ValueError:
    # Fallback to current timestamp
    return datetime.now().strftime('%Y%m%d%H%M%S')
    changeset_timestamp = timestamp.strftime('%Y%m%d%H%M%S')
    return changeset_timestamp
    1. Import the newly created get_git_changeset_timestamp() function in the settings and use it for the STATIC_URL path, as follows:
    # settings/_base.py
    from myproject.apps.core.versioning import get_git_changeset_timestamp
    # ...
    timestamp = get_git_changeset_timestamp(BASE_DIR)
    STATIC_URL = f'/static/{timestamp}/'

    How it works...

    The get_git_changeset_timestamp() function takes the absolute_path directory as a parameter and calls the git log shell command with the parameters to show the Unix timestamp of the HEAD revision in the directory. We pass BASE_DIR to the function, as we are sure that it is under version control. The timestamp is parsed, converted to a string consisting of the year, month, day, hour, minutes, and seconds returned, and is then included in the definition of the STATIC_URL.

    There's more...

    This method works only if each of your environments contains the full Git repository of the project—in some cases, for example, when you use Heroku or Docker for deployments—you don't have access to a Git repository and the git log command in the remote servers. In order to have the STATIC_URL with a dynamic fragment, you have to read the timestamp from a text file—for example, myproject/settings/last-modified.txt—that should be updated with each commit.

    In this case, your settings would contain the following lines:

    # settings/_base.py
    with
    open(os.path.join(BASE_DIR, 'myproject', 'settings', 'last-update.txt'), 'r') as f:
    timestamp = f.readline().strip()

    STATIC_URL = f'/static/{timestamp}/'

    You can make your Git repository update last-modified.txt with a pre-commit hook. This is an executable bash script that should be called pre-commit and placed under django-myproject/.git/hooks/:

    # django-myproject/.git/hooks/pre-commit
    #!/usr/bin/env python
    from subprocess import check_output, CalledProcessError
    import os
    from datetime import datetime

    def root():
    ''' returns the absolute path of the repository root '''
    try:
    base = check_output(['git', 'rev-parse', '--show-toplevel'])
    except CalledProcessError:
    raise IOError('Current working directory is not a git repository')
    return base.decode('utf-8').strip()

    def abspath(relpath):
    ''' returns the absolute path for a path given relative to the root of
    the git repository
    '''
    return os.path.join(root(), relpath)

    def add_to_git(file_path):
    ''' adds a file to git '''
    try:
    base = check_output(['git', 'add', file_path])
    except CalledProcessError:
    raise IOError('Current working directory is not a git repository')
    return base.decode('utf-8').strip()


    def main():
    file_path = abspath("myproject/settings/last-update.txt")

    with open(file_path, 'w') as f:
    f.write(datetime.now().strftime("%Y%m%d%H%M%S"))

    add_to_git(file_path)

    if __name__ == '__main__':
    main()

    This script will update last-modified.txt whenever you commit to the Git repository and will add that file to the Git index.

    See also

    • The Creating the Git ignore file recipe
     

    Setting UTF-8 as the default encoding for the MySQL configuration

    MySQL describes itself as the most popular open source database. In this recipe, we will tell you how to set UTF-8 as the default encoding for it. Note that if you don't set this encoding in the database configuration, you might get into a situation where LATIN1 is used by default with your UTF-8-encoded data. This will lead to database errors whenever symbols such as € are used. This recipe will also save you from the difficulties of converting the database data from LATIN1 to UTF-8, especially when you have some tables encoded in LATIN1 and others in UTF-8.

    Getting ready

    Make sure that the MySQL database management system and the mysqlclient Python module are installed and that you are using the MySQL engine in your project's settings.

    How to do it...

    Open the /etc/mysql/my.cnf MySQL configuration file in your favorite editor and ensure that the following settings are set in the [client], [mysql], and [mysqld] sections, as follows:

    # /etc/mysql/my.cnf
    [client]
    default-character-set = utf8

    [mysql]
    default-character-set = utf8

    [mysqld]
    collation-server = utf8_unicode_ci
    init-connect = 'SET NAMES utf8'
    character-set-server = utf8

    If any of the sections don't exist, create them in the file. If the sections already exist, add these settings to the existing configurations, and then restart MySQL in your command-line tool, as follows:

    $ /etc/init.d/mysql restart

    How it works...

    Now, whenever you create a new MySQL database, the databases and all of their tables will be set in UTF-8 encoding by default. Don't forget to set this up on all computers on which your project is developed or published.

    There's more...

    In PostgreSQL, the default server encoding is already UTF-8, but if you want to explicitly create a PostgreSQL database with UTF-8 encoding, then you can do that with the following command:

    $ createdb --encoding=UTF8 --locale=en_US.UTF-8 --template=template0 myproject

    See also

    • The Creating a project file structure recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
     

    Creating the Git ignore file

    Git is the most popular distributed version control system, and you are probably already using it for your Django project. Although you are tracking file changes for most of your files, it's recommended that you keep some specific files and folders out of version control. Usually, caches, compiled code, log files, and hidden system files should not be tracked in the Git repository.

    Getting ready

    Make sure that your Django project is under Git version control.

    How to do it...

    Using your favorite text editor, create a .gitignore file at the root of your Django project and put the following files and directories there:

    # .gitignore
    ### Python template
    # Byte-compiled / optimized / DLL files
    __pycache__/
    *.py[cod]
    *$py.class

    # Installer logs
    pip-log.txt
    pip-delete-this-directory.txt

    # Unit test / coverage reports
    htmlcov/
    .tox/
    .nox/
    .coverage
    .coverage.*
    .cache
    nosetests.xml
    coverage.xml
    *.cover
    .hypothesis/
    .pytest_cache/

    # Translations
    *.mo
    *.pot

    # Django stuff:
    *.log
    db.sqlite3

    # Sphinx documentation
    docs/_build/

    # IPython
    profile_default/
    ipython_config.py

    # Environments
    env/

    # Media and Static directories
    /media/
    !/media/.gitkeep

    /static/
    !/static/.gitkeep

    # Secrets
    secrets.json

    How it works...

    The .gitignore file specifies patterns that should intentionally be untracked by the 
Git version control system. The .gitignore file that we created in this recipe will ignore the Python-compiled files, local settings, collected static files, 
and media directory with the uploaded files.

    Note that we have exceptional syntax with exclamation marks for media and static files:

    /media/
    !/media/.gitkeep

    This tells Git to ignore the /media/ directory but keep the /media/.gitkeep file tracked under version control. As Git version control tracks files, but not directories, we use .gitkeep to make sure that the media directory will be created in each environment, but not tracked.

    See also

    • The Creating a project file structure recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
     

    Deleting Python-compiled files

    When you run your project for the first time, Python compiles all of your *.py code in bytecode-compiled files, *.pyc, which are used later for execution. Normally, when you change the *.py files, *.pyc is recompiled; however, sometimes when you switch branches or move the directories, you need to clean up the compiled files manually.

    Getting ready

    Use your favorite editor and edit or create a .bash_profile file in your home directory.

    How to do it...

    1. Add this alias at the end of .bash_profile, as follows:
    # ~/.bash_profile
    alias delpyc='
    find . -name "*.py[co]" -delete
    find . -type d -name "__pycache__" -delete'
    1. Now, to clean the Python-compiled files, go to your project directory and type the following command on the command line:
    (env)$ delpyc

    How it works...

    At first, we create a Unix alias that searches for the *.pyc and *.pyo files and __pycache__ directories and deletes them in the current directory, as well as its children. The .bash_profile file is executed when you start a new session in the command-line tool.

    There's more...

    If you want to avoid creating Python-compiled files altogether, you can set an environment variable, PYTHONDONTWRITEBYTECODE=1, in your .bash_profile, env/bin/activate script, or PyCharm configuration.

    See also

    • The Creating the Git ignore file recipe
     

    Respecting the import order in Python files

    When you create the Python modules, it is good practice to stay consistent with the structure in the files. This makes it easier for both you and other developers to read the code. This recipe will show you how to structure your imports.

    Getting ready

    Create a virtual environment and create a Django project in it.

    How to do it...

    Use the following structure for each Python file that you are creating. Categorize the imports into sections, as follows:

    # System libraries
    import os
    import re
    from datetime import datetime

    # Third-party libraries
    import boto
    from PIL import Image

    # Django modules
    from django.db import models
    from django.conf import settings

    # Django apps
    from cms.models import Page

    # Current-app modules
    from .models import NewsArticle
    from . import app_settings

    How it works...

    We have five main categories for the imports, as follows:

    • System libraries for packages in the default installation of Python
    • Third-party libraries for the additional installed Python packages
    • Django modules for different modules from the Django framework
    • Django apps for third-party and local apps
    • Current-app modules for relative imports from the current app

    There's more...

    See also

    • The Handling project dependencies with pip recipe
    • The Including external dependencies in your project recipe
     

    Creating an app configuration

    Django projects consist of multiple Python modules called applications (or, more commonly, apps) that combine different modular functionalities. Each app can have models, views, forms, URL configurations, management commands, migrations, signals, tests, context processors, middlewares, and so on. The Django framework has an application registry, where all apps and models are collected and later used for configuration and introspection. Since Django 1.7, metainformation about apps can be saved in the AppConfig instance for each app. Let's create a sample magazine app to take a look at how to use the app configuration there.

    Getting ready

    You can create a Django app either by calling the startapp management command or by creating the app module manually:

    (env)$ cd myproject/apps/
    (env)$ django-admin.py startapp magazine

    With your magazine app created, add a NewsArticle model to models.py, create administration for the model in admin.py, and put "myproject.apps.magazine" in INSTALLED_APPS in the settings. If you are not yet familiar with these tasks, study the official Django tutorial at https://docs.djangoproject.com/en/3.0/intro/tutorial01/.

    How to do it...

    Follow these steps to create and use the app configuration:

    1. Modify the apps.py file and insert the following content into it, as follows:
    # myproject/apps/magazine/apps.py
    from
    django.apps import AppConfig
    from django.utils.translation import gettext_lazy as _


    class MagazineAppConfig(AppConfig):
    name = "myproject.apps.magazine"
    verbose_name = _("Magazine")

    def ready(self):
    from . import signals
    1. Edit the __init__.py file in the magazine module to contain the following content:
    # myproject/apps/magazine/__init__.py
    default_app_config = "myproject.apps.magazine.apps.MagazineAppConfig"
    1. Let's create a signals.py file and add some signal handlers there:
    # myproject/apps/magazine/signals.py
    from
    django.db.models.signals import post_save, post_delete
    from django.dispatch import receiver
    from django.conf import settings

    from .models import NewsArticle

    @receiver(post_save, sender=NewsArticle)
    def news_save_handler(sender, **kwargs):
    if settings.DEBUG:
    print(f"{kwargs['instance']} saved.")


    @receiver(post_delete, sender=NewsArticle)
    def news_delete_handler(sender, **kwargs):
    if settings.DEBUG:
    print(f"{kwargs['instance']} deleted.")

    How it works...

    When you run an HTTP server or invoke a management command, django.setup() is called. It loads the settings, sets up logging, and prepares the app registry. This registry is initialized in three steps. Django first imports the configurations for each item from INSTALLED_APPS in the settings. These items can point to app names or configurations directly—for example, "myproject.apps.magazine" or "myproject.apps.magazine.apps.MagazineAppConfig".

    Django then tries to import models.py from each app in INSTALLED_APPS and collect all of the models.

    Finally, Django runs the ready() method for each app configuration. This method presents a good point in the development process to register signal handlers, if you have any. The ready() method is optional.

    In our example, the MagazineAppConfig class sets the configuration for the magazine app. The name parameter defines the module of the current app. The verbose_name parameter defines a human name that is used in the Django model administration, where models are presented and grouped by apps. The ready() method imports and activates the signal handlers that, when in DEBUG mode, print in the terminal that a NewsArticle object was saved or deleted.

    There's more...

    After calling django.setup(), you can load the app configurations and models from the registry as follows:

    >>> from django.apps import apps as django_apps
    >>> magazine_app_config = django_apps.get_app_config("magazine")
    >>> magazine_app_config
    <MagazineAppConfig: magazine>
    >>> magazine_app_config.models_module
    <module 'magazine.models' from '/path/to/myproject/apps/magazine/models.py'>
    >>> NewsArticle = django_apps.get_model("magazine", "NewsArticle")
    >>> NewsArticle
    <class 'magazine.models.NewsArticle'>

    You can read more about app configuration in the official Django documentation at
    https://docs.djangoproject.com/en/2.2/ref/applications/​.

    See also

    • The Working with a virtual environment recipe
    • The Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL recipe
    • The Defining overwritable app settings recipe
    • Chapter 6, Model Administration
     

    Defining overwritable app settings

    This recipe will show you how to define settings for your app that can then be overwritten in your project's settings file. This is especially useful for reusable apps that you can customize by adding a configuration.

    Getting ready

    Follow the steps in the Getting ready in the Creating app configuration recipe to create your Django app.

    How to do it...

    1. Define your app settings using the getattr() pattern in models.py if you just have one or two settings, or in the app_settings.py file if the settings are extensive and you want to organize them better:
    # myproject/apps/magazine/app_settings.py
    from django.conf import settings
    from django.utils.translation import gettext_lazy as _

    # Example:
    SETTING_1 = getattr(settings, "MAGAZINE_SETTING_1", "default value")

    MEANING_OF_LIFE = getattr(settings, "MAGAZINE_MEANING_OF_LIFE", 42)

    ARTICLE_THEME_CHOICES = getattr(
    settings,
    "MAGAZINE_ARTICLE_THEME_CHOICES",
    [
    ('futurism', _("Futurism")),
    ('nostalgia', _("Nostalgia")),
    ('sustainability', _("Sustainability")),
    ('wonder', _("Wonder")),
    ]
    )
    1. models.py will contain the NewsArticle model, like this:
    # myproject/apps/magazine/models.py
    from
    django.db import models
    from django.utils.translation import gettext_lazy as _


    class NewsArticle(models.Model):
    created_at = models.DateTimeField(_("Created at"),
    auto_now_add=True)
    title = models.CharField(_("Title"), max_length=255)
    body = models.TextField(_("Body"))
    theme = models.CharField(_("Theme"), max_length=20)

    class Meta:
    verbose_name = _("News Article")
    verbose_name_plural = _("News Articles")

    def __str__(self):
    return self.title
    1. Next, in admin.py, we will import and use the settings from app_settings.py, as follows:
    # myproject/apps/magazine/admin.py
    from
    django import forms
    from django.contrib import admin

    from .models import NewsArticle

    from .app_settings import ARTICLE_THEME_CHOICES


    class NewsArticleModelForm(forms.ModelForm):
    theme = forms.ChoiceField(
    label=NewsArticle._meta.get_field("theme").verbose_name,
    choices=ARTICLE_THEME_CHOICES,
    required=not NewsArticle._meta.get_field("theme").blank,
    )
    class Meta:
    fields = "__all__"


    @admin.register(NewsArticle)
    class NewsArticleAdmin(admin.ModelAdmin):
    form = NewsArticleModelForm
    1. If you want to overwrite the ARTICLE_THEME_CHOICES settings for a given project, you should add MAGAZINE_ARTICLE_THEME_CHOICES in the project settings:
    # myproject/settings/_base.py
    from django.utils.translation import gettext_lazy as _
    # ...
    MAGAZINE_ARTICLE_THEME_CHOICES = [
    ('futurism', _("Futurism")),
    ('nostalgia', _("Nostalgia")),
    ('sustainability', _("Sustainability")),
    ('wonder', _("Wonder")),
    ('positivity', _("Positivity")),
    ('solutions', _("Solutions")),
    ('science', _("Science")),
    ]

    How it works...

    The getattr(object, attribute_name[, default_value]) Python function tries to get the attribute_name attribute from object and returns default_value if it is not found. We try to read different settings from the Django project settings module or, if they don't exist there, the default values are used.

    Note that we could have defined the choices for the theme field in models.py, but instead we created a custom ModelForm in administration and set the choices there. This was done to avoid the creation of new database migrations whenever the ARTICLE_THEME_CHOICES is changed.

    See also

    • The Creating app configuration recipe
    • Chapter 6, Model Administration
     

    Working with Docker containers for Django, Gunicorn, Nginx, and PostgreSQL

    Django projects depend not only on Python requirements, but also on many system requirements, such as a web server, database, server cache, and mail server. When developing a Django project, you need to ensure that all environments and all developers will have all the same requirements installed. One way to keep those dependencies in sync is to use Docker. With Docker, you can have different versions of the database, web, or other servers required individually for each project.

    Docker is a system for creating configured, customized virtual machines called containers. It allows us to duplicate the setup of any production environment precisely. Docker containers are created from so-called Docker images. Images consist of layers (or instructions) on how to build the container. There can be an image for PostgreSQL, an image for Redis, an image for Memcached, and a custom image for your Django project, and all those images can be combined into accompanying containers with Docker Compose.

    In this recipe, we will use a project boilerplate to set up a Django project with a PostgreSQL database, served by Nginx and Gunicorn, and manage all of them with Docker Compose.

    Getting ready

    First, you will need to install the Docker Engine, following the instructions at https://www.docker.com/get-started. This usually includes the Compose tool, which makes it possible to manage systems that require multiple containers, ideal for a fully isolated Django project. If it is needed separately, installation details for Compose are available at https://docs.docker.com/compose/install/​.

    How to do it...

    Let's explore the Django and Docker boilerplate:

    1. Download the code from https://github.com/archatas/django_docker to your computer to the ~/projects/django_docker directory, for example.
    If you choose another directory, for example, myproject_docker, then you will have to do a global search and replace django_docker with myproject_docker.
    1. Open the docker-compose.yml file. There are three containers that need to be created: nginx, gunicorn, and db. Don't worry if it looks complicated; we'll describe it in detail later:
    # docker-compose.yml
    version: "3.7"

    services:
    nginx:
    image: nginx:latest
    ports:
    - "80:80"
    volumes:
    - ./config/nginx/conf.d:/etc/nginx/conf.d
    - static_volume:/home/myproject/static
    - media_volume:/home/myproject/media
    depends_on:
    - gunicorn

    gunicorn:
    build:
    context: .
    args:
    PIP_REQUIREMENTS: "${PIP_REQUIREMENTS}"
    command: bash -c "/home/myproject/env/bin/gunicorn --workers 3
    --bind 0.0.0.0:8000 myproject.wsgi:application"
    depends_on:
    - db
    volumes:
    - static_volume:/home/myproject/static
    - media_volume:/home/myproject/media
    expose:
    - "8000"
    environment:
    DJANGO_SETTINGS_MODULE: "${DJANGO_SETTINGS_MODULE}"
    DJANGO_SECRET_KEY: "${DJANGO_SECRET_KEY}"
    DATABASE_NAME: "${DATABASE_NAME}"
    DATABASE_USER: "${DATABASE_USER}"
    DATABASE_PASSWORD: "${DATABASE_PASSWORD}"
    EMAIL_HOST: "${EMAIL_HOST}"
    EMAIL_PORT: "${EMAIL_PORT}"
    EMAIL_HOST_USER: "${EMAIL_HOST_USER}"
    EMAIL_HOST_PASSWORD: "${EMAIL_HOST_PASSWORD}"

    db:
    image: postgres:latest
    restart: always
    environment:
    POSTGRES_DB: "${DATABASE_NAME}"
    POSTGRES_USER: "${DATABASE_USER}"
    POSTGRES_PASSWORD: "${DATABASE_PASSWORD}"
    ports:
    - 5432
    volumes:
    - postgres_data:/var/lib/postgresql/data/

    volumes:
    postgres_data:
    static_volume:
    media_volume:

    1. Open and read through the Dockerfile file. These are the layers (or instructions) that are needed to create the gunicorn container:
    # Dockerfile
    # pull official base image
    FROM python:3.8

    # accept arguments
    ARG PIP_REQUIREMENTS=production.txt

    # set environment variables
    ENV PYTHONDONTWRITEBYTECODE 1
    ENV PYTHONUNBUFFERED 1

    # install dependencies
    RUN pip install --upgrade pip setuptools

    # create user for the Django project
    RUN useradd -ms /bin/bash myproject

    # set current user
    USER myproject

    # set work directory
    WORKDIR /home/myproject

    # create and activate virtual environment
    RUN python3 -m venv env

    # copy and install pip requirements
    COPY --chown=myproject ./src/myproject/requirements /home/myproject/requirements/
    RUN ./env/bin/pip3 install -r /home/myproject/requirements/${PIP_REQUIREMENTS}

    # copy Django project files
    COPY --chown=myproject ./src/myproject /home/myproject/
    1. Copy the build_dev_example.sh script to build_dev.sh and edit its content. These are environment variables to pass to the docker-compose script:
    # build_dev.sh
    #!/usr/bin/env bash
    DJANGO_SETTINGS_MODULE=myproject.settings.dev \
    DJANGO_SECRET_KEY="change-this-to-50-characters-long-
    random-string"
    \
    DATABASE_NAME=myproject \
    DATABASE_USER=myproject \
    DATABASE_PASSWORD="change-this-too" \
    PIP_REQUIREMENTS=dev.txt \
    docker-compose up --detach --build
    1. In a command-line tool, add execution permissions to build_dev.sh and run it to build the containers:
    $ chmod +x build_dev.sh
    $ ./build_dev.sh
    1. If you now go to http://0.0.0.0/en/, you should see a Hello, World! page there.
      When navigating to http://0.0.0.0/en/admin/, you should see the following:
    OperationalError at /en/admin/
    FATAL: role "myproject" does not exist

    This means that you have to create the database user and the database in the Docker container.

    1. Let's SSH to the db container and create the database user, password, and the database itself in the Docker container:
    $ docker exec -it django_docker_db_1 bash
    /# su - postgres
    /$ createuser --createdb --password myproject
    /$ createdb --username myproject myproject

    When asked, enter the same password for the database as in the build_dev.sh script.

    Press [Ctrl + D] twice to log out of the PostgreSQL user and Docker container.

    If you now go to http://0.0.0.0/en/admin/, you should see the following:

    ProgrammingError at /en/admin/ relation "django_session" does not exist LINE 1: ...ession_data", "django_session"."expire_date" FROM "django_se...

    This means that you have to run migrations to create the database schema.

    1. SSH into the gunicorn container and run the necessary Django management commands:
    $ docker exec -it django_docker_gunicorn_1 bash
    $ source env/bin/activate
    (env)$ python manage.py migrate
    (env)$ python manage.py collectstatic
    (env)$ python manage.py createsuperuser

    Answer all the questions that are asked by the management commands.

    Press [Ctrl + D] twice to log out of the Docker container.

    If you now navigate to http://0.0.0.0/en/admin/, you should see the Django administration, where you can log in with the super user's credentials that you have just created.

    1. Create analogous scripts, build_test.sh, build_staging.sh, and build_production.sh, where only the environment variables differ.

    How it works...

    The structure of the code in the boilerplate is similar to the one in a virtual environment. The project source files are in the src directory. We have the git-hooks directory for the pre-commit hook that is used to track the last modification date and the config directory for the configurations of the services used in the containers:

    django_docker
    ├── config/
    │ └── nginx/
    │ └── conf.d/
    │ └── myproject.conf
    ├── git-hooks/
    │ ├── install_hooks.sh
    │ └── pre-commit
    ├── src/
    │ └── myproject/
    │ ├── locale/
    │ ├── media/
    │ ├── myproject/
    │ │ ├── apps/
    │ │ │ └── __init__.py
    │ │ ├── settings/
    │ │ │ ├── __init__.py
    │ │ │ ├── _base.py
    │ │ │ ├── dev.py
    │ │ │ ├── last-update.txt
    │ │ │ ├── production.py
    │ │ │ ├── staging.py
    │ │ │ └── test.py
    │ │ ├── site_static/
    │ │ │ └── site/
    │ │ │ ├── css/
    │ │ │ ├── img/
    │ │ │ ├── js/
    │ │ │ └── scss/
    │ │ ├── templates/
    │ │ │ ├── base.html
    │ │ │ └── index.html
    │ │ ├── __init__.py
    │ │ ├── urls.py
    │ │ └── wsgi.py
    │ ├── requirements/
    │ │ ├── _base.txt
    │ │ ├── dev.txt
    │ │ ├── production.txt
    │ │ ├── staging.txt
    │ │ └── test.txt
    │ ├── static/
    │ └── manage.py
    ├── Dockerfile
    ├── LICENSE
    ├── README.md
    ├── build_dev.sh
    ├── build_dev_example.sh
    └── docker-compose.yml

    The main Docker-related configurations are at docker-compose.yml and Dockerfile. Docker Compose is a wrapper around Docker's command-line API. The build_dev.sh script builds and runs the Django project under the Gunicorn WSGI HTTP server at port 8000, Nginx at port 80 (serving static and media files and proxying other requests to Gunicorn), and the PostgreSQL database at port 5432.

    In the docker-compose.yml file, the creation of three Docker containers is requested:

    • nginx for the Nginx web server
    • gunicorn for the Django project with the Gunicorn web server
    • db for the PostgreSQL database

    The nginx and db containers will be created from the official images located at https://hub.docker.com. They have specific configuration parameters, such as the ports they are running on, environment variables, dependencies on other containers, and volumes.

    Docker volumes are specific directories that stay untouched when you rebuild the Docker containers. Volumes need to be defined for the database data files, media, static, and the like.

    The gunicorn container will be built from the instructions at the Dockerfile, defined by the build context in the docker-compose.yml file. Let's examine each layer (or instruction) there:

    • The gunicorn container will be based on the python:3.7 image
    • It will take PIP_REQUIREMENTS as an argument from the docker-compose.yml file
    • It will set environment variables for the container
    • It will install and upgrade pip, setuptools, and virtualenv
    • It will create a system user named myproject for the Django project
    • It will set myproject as the current user 
    • It will set the home directory of the myproject user as the current working directory
    • It will create a virtual environment there
    • It will copy pip requirements from the base computer to the Docker container
    • It will install the pip requirements for the current environment defined by the PIP_REQUIREMENTS variable
    • It will copy the source of the entire Django project

    The content of config/nginx/conf.d/myproject.conf will be saved under /etc/nginx/conf.d/ in the nginx container. This is the configuration of the Nginx web server telling it to listen to port 80 (the default HTTP port) and forward requests to the Gunicorn server on port 8000, except for requests asking for static or media content:

    #/etc/nginx/conf.d/myproject.conf
    upstream myproject {
    server django_docker_gunicorn_1:8000;
    }

    server {
    listen 80;

    location / {
    proxy_pass http://myproject;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
    proxy_redirect off;
    }

    rewrite "/static/\d+/(.*)" /static/$1 last;

    location /static/ {
    alias /home/myproject/static/;
    }

    location /media/ {
    alias /home/myproject/media/;
    }
    }

    You can learn more about Nginx and Gunicorn configurations in the Deploying on Nginx and Gunicorn for the staging environment and Deploying on Nginx and Gunicorn for the production environment recipes in Chapter 12, Deployment.

    There's more...

    You can destroy Docker containers with the docker-compose down command and rebuild them with your build script:

    $ docker-compose down
    $ ./build_dev.sh

    If something is not working as expected, you can inspect the logs with the docker-compose logs command:

    $ docker-compose logs nginx
    $ docker-compose logs gunicorn
    $ docker-compose logs db

    To connect to any of the containers via SSH, you should use one of the following:

    $ docker exec -it django_docker_gunicorn_1 bash
    $ docker exec -it django_docker_nginx_1 bash
    $ docker exec -it django_docker_db_1 bash

    You can copy files and directories to and from volumes on Docker containers using the docker cp command:

    $ docker cp ~/avatar.png django_docker_gunicorn_1:/home/myproject/media/
    $ docker cp django_docker_gunicorn_1:/home/myproject/media ~/Desktop/

    If you want to get better a understanding of Docker and Docker Compose, check out the official documentation at https://docs.docker.com/, and specifically https://docs.docker.com/compose/.

    See also

    • The Creating a project file structure recipe
    • The Deploying on Apache with mod_wsgi for the staging environment recipe in Chapter 12, Deployment
    • The Deploying on Apache with mod_wsgi for the production environment recipe in Chapter 12, Deployment
    • The Deploying on Nginx and Gunicorn for the staging environment recipe in Chapter 12, Deployment
    • The Deploying on Nginx and Gunicorn for the production environment recipe in Chapter 12, Deployment

    About the Authors

    • Aidas Bendoraitis

      Aidas Bendoraitis has been professionally building websites for the past 18 years. For the last 14 years, he has been working at a design company, studio 38 pure communication, in Berlin. Together with a small dedicated team, he has mostly used Django in the backend and jQuery in the frontend to create cultural and touristic web platforms.Among different side projects, he is bootstrapping a SaaS business with strategic prioritizer 1st things 1st. Aidas Bendoraitis is active on Twitter and other social media under the username DjangoTricks.

      Browse publications by this author
    • Jake Kronika

      Jake Kronika, a software engineer with 25 years of experience. He has been working with Python since 2005 and Django since 2007. Evolving in lockstep with the web development industry, his skill set encompasses HTML, CSS, full-stack JavaScript, Python, Django, React, Node.js, Ruby on Rails, and several other technologies.Currently a software architect and development team lead, Jake collaborates with designers, business stakeholders, and engineers around the globe to build robust applications. In his spare time, he provides full-spectrum web services as a freelancer. In addition to authoring this book, Jake has reviewed several other Packt titles – most recently, Django 3 By Example, Third Edition by Antonio Melé

      Browse publications by this author

    Latest Reviews

    (1 reviews total)
    Still waiting for it to publish

    Recommended For You

    The Python Workshop

    Cut through the noise and get real results with a step-by-step approach to learning Python 3.X programming

    By Andrew Bird and 4 more
    Mastering Python Networking - Third Edition

    New edition of the bestselling guide to mastering Python Networking, updated to Python 3 and including the latest on network data analysis, Cloud Networking, Ansible 2.8, and new libraries

    By Eric Chou
    The Complete Python Course [Video]

    Go from beginner to expert in Python by building projects. The best investment for your Python journey!

    By Company Eco Web Hosting Ltd and 1 more