Chapter 8. Scaling

Imagine you are part of a development and support team that is responsible for developing the company's flagship product: TaxCloud. TaxCloud helps taxpayers file their own taxes and charges them a small fee upon the successful filing of taxes. Consider you had developed this application using microservices. Now say the product gets popular and gains traction, and suddenly, on the last day of tax filing, you get a rush of consumers wanting to use your product and file their taxes. However, the payments service of your system is slow, which has almost brought the system down, and all the new customers are moving to your competitor's product. This is a lost opportunity for your business.

Even though this is a fictitious scenario, it can very well happen to any business. In e-commerce, we have always experienced these kinds of things in real life, especially on special occasions such as Christmas and Black Fridays. All in all, they point toward one major significant characteristic...

Scalability overview

Design decisions go a long way in impacting the scalability of a single microservice. As with other application capabilities, decisions that are made during the design and early coding phases largely influence the scalability of services.

Microservice scalability requires a balanced approach between services and their supporting infrastructures. Services and their infrastructures also need to scale in harmony.

Scalability is one of the important non-functional characteristics of a system by which it can handle more payload. It is often felt that scalability is usually a concern for large-scale distributed systems. Performance and scalability are two different characteristics of a system. Performance deals with the throughput of the system, whereas scalability deals with serving the desired throughput for a larger number of users or a larger number of transactions.

Scaling infrastructure

Microservices are modern applications and usually take advantage of the cloud. Therefore, when it comes to scalability, the cloud provides certain advantages. However, it is also about automation and managing costs. So even in the cloud, we need to understand how to provision infrastructure, such as virtual machines or containers, to successfully serve our microservice-based application even in the case of sudden traffic spikes.

Now we will visit each component of our infrastructure and see how we can scale it. The initial scaling up and scaling out methods are applied more toward hardware scaling. With the Auto Scaling feature, you will understand Azure virtual manager scale sets. Finally, you will learn about scaling with containers with Docker Swarm mode. 

Vertical scaling (scaling up)

Scaling up is a term used for achieving scalability by adding more resources to the same machine. It includes the addition of more memory or processors with higher speed or simply the...

Microservices scalability

In this section, we will view the scaling strategies available for microservices. We will visit the Scale Cube model of scalability, see how to scale the infrastructure layer for microservices, and embed scalability into the microservice design.

Scale Cube model of scalability

One way to look into scalability is by understanding Scale Cube. In the book The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise, Martin L. Abbott and Michael T. Fisher defines Scale Cube to view and understand system scalability. Scale Cube applies to microservice architectures as well: 

In this three-dimensional model of scalability, the origin (0,0,0) represents the least scalable system. It assumes that the system is a monolith deployed on a single server instance. As shown, a system can be scaled by putting the right amount of effort in three dimensions. To move a system toward the right scalable direction, we need to do the right trade...

Scaling the infrastructure

In this section, we will visit all the layers of the microservice infrastructure and see them in relation to each other, that is, how each individual infrastructure layer can be scaled. For our microservice implementation, there are two major components. One is virtual machines and the other is the container hosted on virtual or physical machines. The following diagram shows a logical view of the microservice infrastructure:

Scaling virtual machines using scale sets

Scaling virtual machines is quite simple and easy in Azure Cloud. This is where microservices shine through. With scale sets, you can raise the instances of the same virtual machine images in a short amount of time, and that automatically too, based on the ruleset. Scale sets are integrated with Azure autoscale.

Azure virtual machines can be created in a way that as a group, they would always serve the requests even if the volume of the requests increases. In specific situations, they can also be deleted...

Scaling service design

In this section, we will visit the components/concerns that need to be taken care of while designing or implementing a microservice. With infrastructure scaling taking care of service design, we can truly unleash the power of the microservice architecture and get a lot of business value around making microservices a true success story. So what are those components in service design; let's have a look.

Data persistence model design

In traditional applications, we have always relied on relational databases to persist user data. Relational databases are not new to us. They emerged in the seventies as the way to store persistent information in a structured way that would allow you to make queries and perform data maintenance.

In today's world of microservices, modern applications need to be scaled at the hyperscale stage. We are not recommending here that you abandon the usage of relational databases in any sense. They still have their valid use cases. However, when we...

Summary

Scalability is one of the critical advantages of pursuing the microservice architectural style. We saw the characteristics of microservice scalability. We discussed the Scale Cube model of scalability and how microservices can scale on the y-axis by functional decomposition of the system. Then we approached the scaling problem with the scaling infrastructure. In the infrastructure segment, we saw strong capability of Azure Cloud to scale, utilizing the Azure scale sets and container orchestration solutions, such as Docker swarm, DC/OS, and kubernates.

In later stages of the chapter, we focused on scaling with a service design and discussed how our data model should be designed; we also discussed considerations, such as having a split CQRS style model, while designing the data model for high scalability. We also briefly touched on caching, especially distributed caching, and how it improves the throughput of the system. In the last section, to make our microservices highly scalable...