Cloud-native Data Processing with MongoDB

Data and data processing are at the heart of any application, including microservices applications like the one we’ll develop. And in this, databases play a crucial role as they store and process application data. We have a range of database choices, but we’ll apply MongoDB because it’s cloud-native like our microservices will be.

In this chapter, you’ll learn about MongoDB and address questions like how you set up MongoDB and create a database. Furthermore, you’ll learn about NoSQL databases like MongoDB and how they differ from relational databases. You’ll also take an advance on developing RESTful APIs and learn how the Create, Read, Update, Delete (CRUD) operations map to HTTP methods like the GET method for querying APIs.

By the end of this chapter, you know how to lay the data processing foundation for cloud-native Django microservices with MongoDB. Including applying CRUD operation to maintain...

Technical requirements

Depending on your local setup, you may need to add your workstation’s IP address to your network’s whitelist to work with MongoDB from VS Code.

You can find the code samples for this chapter on GitHub at https://github.com/PacktPublishing/Hands-on-Microservices-with-Django/tree/main/Chapter04.

Introducing MongoDB and cloud-native databases

We already peeked into the cloud version of MongoDB in the Chapter 3, Setting Up the Development and Runtime Environment. There is also a community server version, which we can run on-premise (even in a Docker container). Still, we focus on the cloud version because it coheres nicely with our cloud-native microservices.

In the following sub-sections, we’ll look at cloud-native and NoSQL databases and why we would use them.

What are cloud-native databases?

Cloud-native databases are just what their naming implies: databases designed to run in cloud environments. So, instead of running a database on a local server, we connect to a remote cloud server through the internet.

Why use a cloud-native database?

This raises the question of why we would use a cloud-native database, and the answer to that lies in these benefits of cloud-native databases:

• Scalability: Cloud-native databases can easily scale to handle different...

Setting up MongoDB

At the highest level, MongoDB organizes databases in projects, and within a project, you have one or more clusters with one or more databases. During the signing up for MongoDB in the Chapter 3, Setting Up the Development and Runtime Environment, we’ve already created a free cluster for our microservices application.

To set up MongoDB, we’ll start with the option to create a paid cluster. In the consecutive sections we’ll create a database user and set up MongoDB for Django.

Optional: creating a paid cluster for production databases

A free cluster is fine for a learning project like ours, but you’ll need a paid serverless or dedicated cluster for a production database. If you want a production cluster, take these steps to create one:

1. Click Database from the left menu.
2. Click + Create.

   * The Create New Cluster page opens.

   From here, you have two choices:

   • Serverless: pay per operation with a fixed amount per 1M reads.
   • Dedicated...

Creating a Database

Databases are the heart of any database system, and MongoDB is no exception. MongoDB’s setup with projects, clusters, and databases is meant to organize our data layer, and databases are central to that. Creating a database takes the following steps from the MongoDB web interface:

1. Click Overview from the left menu.
2. Click Add Data Options.

   * The Add Data Options page opens.

3. Click START inside the Create Database on Atlas card.
4. In the Database name field, enter TemporaryDB.
5. In the Collection name field, enter Products. Collections are MongoDB’s equal to tables.
6. Click Create Database.

   * MongoDB creates the database and the collection and shows the find/query page for the TemporaryDB.Products collection.

You’ve created the database and can start storing data in collections and documents.

Creating documents inside a collection

Usually, Django applications will add data to MongoDB, but let’s add a product...

Mapping CRUD operations to HTTP methods

Basically, data processing is about creating, reading, updating, and deleting data as it converges in the CRUD acronym. The address-matching microservice from our sample microservices application will add addresses to the Subscription database. We have two options to set up the microservice for this:

1. Let the microservice update the database directly through the pymongo Python package or Django ORM.
2. Engage a Web API, which updates the database, and let the microservice call that Web API.

Since Django ORM is easier to apply than a pymongo – MongoDB connection, we prefer Django ORM within option 1. We still choose option 2, the Web API, because a Web API better aligns with the microservices architecture where the components should be self-contained, single-tasked, and independent.

Choosing the Web API option means we’ll build a RESTful API with DRF to handle the data processing. And this requires us to map CRUD...

Summary

This chapter started by explaining what cloud-native databases are and that we selected MongoDB as the database for our application because the cloud version of MongoDB coheres well with the principles of the microservices architecture.

Next, we learned about NoSQL databases and how they differ from relational databases. This also led us to the vocabulary for MongoDB, where collections are the counterpart of tables.

Then, we prepared MongoDB for our application and created our first database and collection. Finally, we mapped the CRUD operations and their counterpart HTTP methods in anticipation of the RESTful API we’ll build for processing the data in our application.

With this, we have completed the data layer of our application, and we’re set for the next chapter, where we’ll learn about developing RESTful APIs with DRF.