Caching, Logging, and Throttling

In Chapter 5, we learned how we can authenticate every request and perform authorization to make sure the client has appropriate permission to access resources. These operations make additional DB calls for each request. For most use cases, these additional DB calls are insignificant. But as we build the system, we might come across use cases where we want faster response times for DB read operations; caching plays an important role when we try to achieve this. In this chapter, we will learn how to use caching in Django. Throttling is the process of limiting the number of requests made by the client in a certain period. Logging and throttling are advanced concepts that we use extensively in the industry.

In this chapter, we will learn:

• Caching with Django
• Logging with Django
• Throttling with Django

Technical requirements

We are expecting the readers to have a basic idea of what caching means and how it is beneficial to end users. We are expecting our readers to be familiar with the concepts related to and the use cases for implementing throttling.

Both caching and throttling need an additional in-memory database: Redis. Though we shall use abstracted Redis APIs for our implementation, it is recommended that the reader knows what Redis is and its basic concepts. Apart from this, we are going to learn about how one can perform logging operations in Django. The Django logging framework uses and extends Python’s built-in logging module, so we are expecting our readers to know how the Python logging module works on a high level.

Learn more

To get a basic understanding of Redis, one can follow the official documentation of Redis or take one of the Redis courses available at https://university.redis.com/.

Python official documentation has a fairly simple guide to...

Caching with Django

Caching is considered one of the necessary evils of computer science. It helps to attain performance gain, but it also makes the system complex. One of the most important complexities in computer science is invalidating the cache. In this section, we will learn how caching is implemented in Django and also how we can improve the overall caching experience for developers using Django and DRF.

Django provides different types of caching strategies out of the box:

• Local memory caching: The data is cached in the application memory itself.
• Database caching: Django would store the cached results in the database itself.
• Filesystem caching: Cached entries are saved as files on the file system.
• Memory database-based caching: Django supports caching databases such as Memcache and Redis to be used in the project. We shall learn how to integrate Redis into our Django project in this chapter.
• Custom caching: Developers can also write a custom backend...

Throttling with Django

Throttling is the process of limiting the number of API requests that can be made by a user in a given time frame. Django does not provide any throttling feature out of the box, but DRF does. In this section, we will learn about how we can incorporate throttling into using DRF.

Different systems have different use cases for throttling. For a SaaS product, throttling is needed to limit the number of API calls per customer per minute depending upon their paid plan. An example is Shopify backend APIs. A standard product company doesn’t want its APIs to be abused by any outside customer, so it would throttle any request beyond a particular threshold.

Important note

We should understand that in this section we are implementing throttling in the application layer. This is generally not a recommended approach to protecting your service from brute force attacks since application-level throttling might still increase the request queue for your application...

Logging with Django

Logging can be considered a record of every data in programming. Django uses Python’s built-in logging module (https://docs.python.org/3/library/logging.html) to capture logs. In Django, logging is configured as part of the general Django django.setup() function, so it’s always available unless explicitly disabled. Python’s logging module provides extensive options to set up and configure logging; hence, all those configurations are also available when we use logging in Django. In this section, we shall learn how we can set up logging in Django and the best practices to use logging in Django projects.

Important note

Logging in Python is an extensive topic, and the configuration of logging can be tricky at times. In this section, we will not go into too much detail about how one can use all the different options provided by Python. Rather, we shall check the standard configuration, which I have used in multiple projects and have found to...

Summary

In this chapter, we learned how to set up a Redis server using RedisLabs. Redis is an important in-memory database used for different applications such as caching and throttling. We learned how to integrate Redis into native Django and use caching with Redis. The built-in support for caching in Django is not sufficient to add caching to APIs, but django-cacheops, a third-party package, can ease the caching implementation. We also learned how to set up django-cacheops with Redis and the best practices for using it in production.

Apart from caching, Redis is used for throttling, and we have learned how to set up throttling in Django and how to create custom application-level throttling. Logging in Django uses Python’s built-in logging module. It is a daunting task for developers to set up logging in Django, so we learned about the basic boilerplate logging setup that you can use in production to log messages. We will revisit logging setup again in Chapter 14, where...