Chapter 9: Data Management and Database Strategies

In this chapter, you will learn how to decide which database to use according to the use case. You will also learn best practices about data management in the cloud, such as how to design your primary key, data backup, data retention, and other related topics.

The following topics are covered in this chapter:

• How to choose which database fits your needs
• Data management strategies for your databases in Google Cloud
• Database and application management best practices

Technical requirements

There are no specific technical requirements for this chapter.

How to decide which database fits your needs

Within Google Cloud, we have multiple storage options depending on the workloads we face. For example, we have options for relational databases such as Cloud SQL, semi-structured data storage with Firestore, high-frequency data ingestion capacity with Bigtable, and global information replication with Cloud Spanner.

In the following sections, we will review these four databases in a general way and, in more detail, review different use cases in which we can make the most of these different database options.

Cloud SQL

Cloud SQL is a fully managed database service for relational databases such as MySQL, PostgreSQL, and SQL Server that allows the operational costs associated with ensuring availability and generation of backups to be reduced by delegating them to Google Cloud.

Cloud Spanner

Cloud Spanner is a fully managed relational database with unlimited scalability, strong consistency, and up to 99.999% availability. It optimizes...

Data management strategies for your databases in Google Cloud

In the previous section, we learned that the choice we make from the different database options offered by Google Cloud will depend on the workloads we face when considering factors such as data structure, data volume, latency, and throughput. In this section, we will focus on factors that allow us to get the best performance from our database, to ensure that applications developed on these databases function optimally.

What is database sharding?

Database sharding is a horizontal partition of data held on a separate server to spread the load.

Some of the database solutions that Google Cloud offers, such as Firestore, Cloud Spanner, and Bigtable, do database sharding in order to give the benefits of horizontal scaling.

One of the problems of database sharding is hot-spotting, a performance problem that we are going to review in the next section.

What is hot-spotting, and why should you avoid it?

Hot-spotting...

Database and application management best practices

In this section, we will review a compilation of good practices for designing and developing in the cloud on our databases.

Keep your Cloud SQL small

It is recommended when possible to keep Cloud SQL instances as small as possible. Instead of having a large instance of Cloud SQL, with lots of resources, it is better and simpler to manage smaller instances, with resources distributed among those instances:

Figure 9.3 – Keeping your instance small

In addition, within these instances, it is not recommended to have more than 10,000 tables, since this can affect both the response times of the instance and the SLA coverage delivered by the cloud provider. The latter is because if the instance does not respond, it is not possible to carry out maintenance operations that directly affect the availability of the service.

If it is necessary to perform a large transaction on the instance, it is recommended...

Summary

In this chapter, we have reviewed how to decide which database to use, depending on the kind of workload that we need to implement. We have looked at what hot-spotting is and how to avoid this problem when we design our primary keys. Finally, we have learned the best practices around performance, error handling, and the reduction of costs and latency. In the next chapter, we will review how to optimize our application's performance using caching strategies.