Download code from GitHub
The advent of cloud computing and the ubiquity of mobile devices have led to the rise of consumer-facing companies (such as Amazon, Netflix, Uber, Google, and Airbnb) that have redefined the entire customer experience. These companies have built their applications (both web and mobile interfaces) on the cloud, using features or services that allow them to scale up or down based on demand, be available at all times, and be ready to handle failures at all levels.
Traditional enterprises are looking at these consumer-facing companies and want to adopt some of their best practices. They do this to help scale up their rapidly evolving enterprise applications, allowing them to take advantage of the elasticity and scalability of the cloud.
Before we dive deep into cloud-native, let's see what this chapter holds. We will cover the following topics in this chapter:
- Why go cloud-native?
- What is cloud-native?
- Intro to the 12-factor app
- Why move from monolithic applications to distributed microservice-based applications?
- The advantages of building a distributed microservice-based application
Let's have a look at the following points to understand why we need to go cloud-native:
- The first wave of cloud adoption was about cost savings and business agility (especially around infrastructure provisioning and cheap storage). With increasing cloud adoption, enterprises started discovering Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) services and their utilization in building applications that leverage the elasticity and scalability of the cloud, all the while embracing the inherent failures of the cloud platform.
- A lot of enterprises are adopting greenfield design and development of microservices in the area of digital initiatives. When dealing with the Internet of Things (IoT), mobile devices, SaaS integration, and online business models, enterprises are working with niche players in the market. These new age business models are being designed and developed as a system of innovation on the enterprise end. The models are iterated rapidly to identify and bubble up the customer's needs, their preferences, what works, and what does not work.
- Enterprises are also developing digital services based on their product lines. The products are enhanced with IoT to enable them to emit data about the products' performance. The data is collated and analyzed for patterns such as predictive maintenance, usage models, and external factors. The data from customers is collated and aggregated to build newer models for product enhancements and new features. A lot of these new digital services use the cloud-native model.
- These modern digital solutions use APIs from various providers, such as Google Maps for location, Facebook/Google for authentication, and Facebook/Twitter for social collaborations. Mashing all these APIs with the features and functionality of enterprise business allows them to build a unique proposition for the customer. All of this integration is happening at the API level. The mobile application is not meant for tens of hundreds of users, but tens of millions of users. This means that, as the load increases, the underlying application functionality should be able to scale up to provide a seamless experience to the customer.
- One way to scale up the resources for the enterprise is to do the heavy lifting in terms of service/environment provisioning as the load goes up or in case of failures. Another way is to offload the heavy lifting of the underlying services to the cloud platform provider. This is the sweet spot where building cloud-native applications that make use of the cloud provider's platform services allows the enterprise to offload the key aspects of scalability and focus on value generation parts.
When applications are designed and architected to take advantage of the underlying IaaS and PaaS services supported by the cloud computing platform, they are called cloud-native applications.
This means building reliable system applications, such as five nines (99.999%), that run on a three nines (99.9%) infrastructure and application components. We need to design our application components to deal with failures. To handle such failures, we need a structured approach for scalability and availability. To support the entire scale of applications, all the pieces need to be automated.
Cloud adoption typically happens in a series of steps, where the enterprise starts exploring the services before they start building cloud-native applications. The adoption starts with the movement of Dev/Test environments to the cloud, where rapid provisioning is the key ask from the business and developer community. Once the enterprise is past the environment provisioning stage, the next step/models in which the enterprise applications are migrated to the cloud-native model will be discussed in the following sections.
Traditionally, enterprises started on their cloud computing journey with IaaS services. They did a lift and shift of the business application workloads from on-premises data centers and moved to the equivalent rented capacity on the cloud computing platform. This is the first wave of adoption of cloud computing platforms, where enterprises are shifted from a capital expenditure model to an operating expenditure model.
IaaS, as the names suggests, is focused on infrastructure—compute nodes, network, and storage. In this model, enterprises can take advantage of the elasticity of the cloud, where compute nodes can be added or removed based on the incoming demand or load. The virtual machine (VM) abstracts out the underlying hardware and provides the ability to scale the number of VMs up or down with just a few clicks.
Enterprises typically make use of IaaS in the first wave because of the following:
- Variability of resources: The ability to add/remove resources at will, which in turn allows more business agility
- Utility model: IaaS provides basic resources that are rented out on an hourly basis, allowing more predictability and an opex model
Once the enterprises start becoming comfortable with the IaaS, the next wave of adoption comes in terms of adoption of PaaS as part of the application workloads. In this stage, the enterprises start discovering services with the following benefits:
- Platform services replacement: This involves the identification of potential platform features of the enterprise, lifting and shifting the workload, and replacing it with equivalent platform services from the cloud provider. For example:
- Replacing application messaging systems with queuing systems provided by the cloud provider (such as AWS SQS)
- Replacing data stores or relational database management systems (RDMBS) with equivalent managed data services (such as AWS RDS)
- Replacing security or directory services with a managed directory or security services (such as AWS Directory and AWS IAM)
- These services allow the enterprise to do away with all the operational efforts, such as data store backup, availability, scalability, and redundancy, and replace them with a managed service that provides all these features
- Application services replacement: Enterprises discover new services that can replace their own platform or utility services. For example:
- Replacing build and release services or products with equivalent DevOps services from the cloud provider (such as AWS CodePipeline, AWS CodeCommit, or AWS CodeDeploy)
- Replacing application services or products with equivalent application platform services (such as AWS API Gateway, AWS SWF, and AWS SES)
- Replacing analytics workload services with equivalent application analytics services (such as AWS Data Pipeline and AWS EMR)
Once the applications start adopting the platform services, the applications start abstracting out features or functionalities provided by commercial off-the-shelf (COTS) products (such as messaging, notification, security, workflow, and API Gateway) and replacing them with equivalent feature platform services. For example, instead of hosting and running the messaging IaaS, movement to an equivalent platform service means moving to a model where you pay only for the number of messages sent, without incurring any additional operational costs. This model brings significant savings, as you move from renting and operating the product to a model where the product is rented only for the time it is utilized.
Once the enterprise has adopted the PaaS to build the application, the next step is to abstract out the application logic as a series of smaller functions and deploy them. These functions are invoked as a reaction to an event from the user or agent, which results in these functions computing the incoming events and giving a result back. This is the highest level of abstraction, where the application has been divided into a series of functions and these functions are deployed independently of each other. The functions communicate with each other using asynchronous communication models. Cloud computing platforms provide features such as AWS Lambda and Azure Functions for going serverless.
To enable the adoption of the IaaS and PaaS services, a change in how the applications are designed and architected needs to be made.
The model of designing enterprise applications on a base platform (read: application server) meant that the heavy lifting of the application's scalability and availability was the responsibility of the platform. Enterprise developers would focus on using the standardized JEE patterns and developing components (Presentation, Business, Data, and Integration) to build fully functional and transactional applications. The extent to which the application could be scaled was limited by the abilities (node clustering and distributed caching) of the underlying platform:
Monolithic application
A business application built as a monolithic application is typically characterized by the following factors:
- The entire application logic is packaged into a single EAR file
- The application reuse is derived by sharing JARs
- Application changes are planned months in advance, typically in a big push once a quarter
- There is one database that encompasses the entire schema for the application
- There are thousands of test cases that signify the amount of regression
- The application design, development, and deployment requires coordination among multiple teams and significant management
With the advent of social interactions and mobile users, the scale of application users and data started increasing exponentially. Enterprises soon found that the platform was becoming a bottleneck in terms of the following issues:
- Business agility: The operational cost of managing the application platform and making constant changes to the features/functionalities was getting hampered because of the monolithic nature of the application. Even for a small feature change, the entire cycle of regression tests and deployment across server clusters was eating into the overall speed of innovation. The mobile revolution meant that the problem was not just at the channel layers, but also percolated down to the integration and systems of record layers. Unless enterprises fixed the problem across these layers, the ability to innovate and be competitive in the market would be under threat.
- Cost: To handle the increased demand, the IT Operations team were adding new server instances to handle the load. However, with each new instance, the complexity was increasing along with license costs (that depended on the number of cores). Unlike the Facebooks of the world, enterprise cost per user was increasing with every user acquisition.
At this time, enterprises started looking at open source products and how modern applications are getting built in consumer-facing companies serving millions of users, handling petabytes of data, and deploying to the cloud.
Consumer-facing companies encounter these hurdles early in their life cycle. Lots of innovation led to the design and development of new open source products, as well as design patterns for cloud computing.
In this context, the whole premise of service-oriented architecture (SOA) was looked at and enterprises investigated how the application architecture could adopt principles of designing autonomous services that are isolated, discrete, and could be integrated and composed with other services. This has led to the rise of the microservices model, which adapts and integrates very well with the cloud services model, where everything is available as a service and as an HTTP endpoint.
Microservices is a specialization of and implementation approach for service-oriented architectures (SOA) used to build flexible, independently deployable software systems
– Wikipedia
Microservices are designed and developed, keeping in mind that a business application can be built by composing these services. The microservices are designed around the following principles:
- Single-responsibility principle: Each microservice implements only one business responsibility from the bounded domain context. From a software point of view, the system needs to be decomposed into multiple components where each component becomes a microservice. Microservices have to be lightweight, in order to facilitate smaller memory footprints and faster startup times.
- Share nothing: Microservices are autonomous, self-contained, stateless, and manage the service state (memory/storage) through container-based encapsulation models. The private data is managed by a service and there is no contention on the data from any other service. Stateless microservices scale better and start faster than stateful ones, as there is no state to be backed up on shutdown or activated on startup.
- Reactive: This is applicable for microservices with concurrent loads or longer response times. Asynchronous communication and the callback model allow optimal utilization of the resources, resulting in better availability and increased throughput of the microservices.
- Externalized configuration: This externalizes the configurations in the config server, so that it can be maintained in hierarchical structure, per environment.
- Consistent: Services should be written in a consistent style, as per the coding standards and naming convention guidelines.
- Resilient: Services should handle exceptions arising from technical reasons (connectivity and runtime), and business reasons (invalid inputs) and not crash. Patterns, such as circuit breakers and bulk headers, help isolate and contain failures.
- Good citizens: Microservices should report their usage statistics, the number of times they are accessed, their average response time, and so on through the JMX API or the HTTP API.
- Versioned: Microservices may need to support multiple versions for different clients, until all clients migrate to higher versions. There should be a clear version strategy in terms of supporting new features and bug fixing.
- Independent deployment: Each of the microservices should be independently deployable, without compromising the integrity of the application:
Moving from a monolithic to a microservices-based application
The microservices' design, development, and deployment considerations are covered in detail in the subsequent chapters. We will see how to build services for an e-commerce product. I am sure everyone is quite familiar with e-commerce and will understand the product requirements easily.
In order to build a distributed, microservices-based application that can be deployed across cloud providers, engineers at Heroku came up with 12 factors that need to be implemented by any modern cloud-native application:
- Single codebase: The application must have one codebase, tracked in revision control for every application (read: microservice) that can be deployed multiple times (development, test, staging, and production environments). Two microservices do not share the same codebase. This model allows the flexibility to change and deploy services without impacting other parts of the application.
- Dependencies: The application must explicitly declare its code dependencies and add them to the application or microservice. The dependencies are packaged as part of the microservice JAR/WAR file. This helps isolate dependencies across microservices and reduce any side effects through multiple versions of the same JAR.
- Config: The application configuration data is moved out of the application or microservice and externalized through a configuration management tool. The application or microservice will pick up the configuration based on the environment in which it is running, allowing the same deployment unit to be propagated across the environments.
- Backing services: All external resources, access should be an addressable URL. For example, SMTP URL, database URL, service HTTP URL, queue URL, and TCP URL. This allows URLs to be externalized to the config and managed for every environment.
- Build, release, and run: The entire process of building, releasing, and running is treated as three separate steps. This means that, as part of the build, the application is built as an immutable entity. This immutable entity will pick the relevant configuration to run the process based on the environment (development, testing, staging, or production).
- Processes: The microservice is built on and follows the shared-nothing model. This means the services are stateless and the state is externalized to either a cache or a data store. This allows seamless scalability and allows load balance or proxy to send requests to any of the instances of the service.
- Port binding: The microservice is built within a container. The service will export and bind all its interfaces through ports (including HTTP).
- Concurrency: The microservice process is scaled out, meaning that, to handle increased traffic, more microservice processes are added to the environment. Within the microservice process, one can make use of the reactive model to optimize the resource utilization.
- Disposability: The idea is to build a microservice as immutable with a single responsibility to, in turn, maximize robustness with faster boot-up times. Immutability also lends to the service disposability.
- Dev/prod parity: The environments across the application life cycle—DEV, TEST, STAGING, and PROD—are kept as similar as possible to avoid any surprises later.
- Logs: Within the immutable microservice instance, the logs generated as part of the service processing are candidates for state. These logs should be treated as event streams and pushed out to a log aggregator infrastructure.
- Admin processes: The microservice instances are long-running processes that continue unless they are killed or replaced with newer versions. All other admin and management tasks are treated as one-off processes:
12-factor app
Applications that follow the 12 factors make no assumptions about the external environment, allowing them to be deployed on any cloud provider platform. This allows the same set of tools/processes/scripts to be run across environments and deploy distributed microservices applications in a consistent manner.
In order to successfully run microservices, there are certain enabling components/services that are needed. These enabling services can be tagged as PaaS that are needed to support the building, releasing, deployment, and running of the microservices.
In the case of the cloud-native model, these services are available as PaaS services from the cloud provider itself:
- Service discovery: When the application is decomposed into a microservices model, a typical application may be composed of hundreds of microservices. With each microservice running multiple instances, we soon have thousands of microservice instances running. In order to discover the service endpoint, it is pertinent to have a service registry that can be queried to discover all of the instances of the microservice. In addition, the service registry tracks the heartbeat of every service instance to make sure that all services are up and running.
Further, the service registry helps in load balancing the requests across the service instances. We can have two models for load balancing:
- Client-side load balancing:
- A service consumer asks the registry for a service instance
- The service registry returns with the list of services where the service is running
- Server-side load balancing:
- The service endpoint is hidden by Nginx, API Gateway, or another reverse proxy from the consumer
Typical products in this space are Consul and Zookeeper:
- Client-side load balancing:
The service registry
- Config server: The microservice needs to be initialized with multiple parameters (for example, database URL, queue URL, functional parameters, and dependency flags). Managing properties in file or environment variables beyond a certain number can become unwieldy. To manage these properties across environments, all such configurations are managed externally in a configuration server. At boot time, microservices will load the properties by invoking the API on the config server.
Microservices also make use of listeners to listen for any changes to the properties on the config server. Any runtime change of properties can be picked up immediately by the microservices. The properties are typically categorized at multiple levels:
- Service-specific properties: Hold all properties tied to the microservice
- Shared properties: Hold properties that might be shared between services
- Common properties: Hold properties that are common across services
The config server can back up these properties in a source-control system. Typical products in this space are Consul, Netflix Archaius, and Spring Cloud Config server:
The config server
- Service management/monitoring: An average business application typically tends to get decomposed into about 400 microservices. Even if we ran two to three instances of these microservices, we would be talking about managing over 1,000 instances of our microservices. Without an automated model, managing/monitoring these services becomes an operational challenge. The following are the key metrics that need to be managed and monitored:
- Service health: Each service needs to publish its health status. These need to be managed/tracked to identify slow or dead services.
- Service metrics: Each service also publishes throughput metrics data, such as the number of HTTP requests/responses, the request/response size, and the response latency.
- Process info: Each service will publish JVM metrics data (like heap utilization, the number of threads, and the process state) typically available as part of the Java VisualVM.
- Log events as stream: Each service can also publish log events as a set of streaming events.
All of this information is pulled from the services and tied together to manage and monitor the application services landscape. Two types of analysis—event correlation and correction decisions—need to be done. Alerts and actuation services are built as part of the service monitoring systems. For example, if a certain number of service instances need to be maintained and the number reduces (service not available due to health check) then an actuation service can take the event as an indicator to add another instance of the same service.
Further, in order to track the service call flow through the microservices model, there is third-party software available that can help create a request identified and track how the service call flows through the microservices. This software typically deploys agents onto the containers, which weave them into the services and track the service metrics:
Service metrics
- Container management/orchestration: Another key infrastructure piece of the microservice environment is container management and orchestration. The services are typically bundled in a container and deployed in a PaaS environment. The environment can be based on an OpenShift model, a Cloud Foundry model, or a pure VM-based model, depending whether they are deployed on a private or a public cloud. To deploy and manage the dependencies between the containers, there is a need for container management and orchestration software. Typically, it should be able to understand the interdependencies between the containers and deploy the containers as an application. For example, if the application has four pieces—one for UI, two for business services, and one for data store—then all of these containers should be tagged together and deployed as a single unit with interdependencies and the right order of instantiation injected.
- Log aggregation: 1 of the 12 factors is treating logs as event streams. The containers are meant to be stateless. The log statements are typically stateful events that need to be persisted beyond the life of the containers. As a result, all logs from the containers are treated as event streams that can be pushed/pulled onto a centralized log repository. All the logs are aggregated and various models can be run on these logs for various alerts. One can track security and failure events through these logs, which can feed into the service management/monitoring system for further actions:
Log aggregation
- API Gateway/management: The services are meant to be simple and follow the single responsibility model. The question arises: who will handle other concerns, such as service authentication, service metering, service throttling, service load balancing, and service freemium/premium models? This is where the API Gateway or management software comes into the picture. The API Gateway handles all such concerns on behalf of the microservice. The API Gateway provides multiple options for managing the service endpoints and can also provide transformation, routing, and mediation capabilities. The API Gateway is more lightweight, compared to the typical enterprise service bus.
API Management Gateway
- DevOps: Another key aspect is the continuous integration/deployment pipeline, coupled with the automated operations that need to set up the microservice-based applications. As the developer writes code, it goes through a series of steps that need to be automated and mapped with gating criteria to allow the release of regression-tested code:
Development life cycle
Microservice adoption within an enterprise is driven by a common theme of digital transformation, whether they are looking to re-architect the existing monolithic applications in the system of innovations to increase business agility and reduce technical debt, or to develop greenfield applications that allow them to rapidly innovate and experiment with different business models.
Enterprises have been running channel applications built on JEE principles running on clusters of application servers. These applications have accumulated a lot of technical debt over the years and have become a major issue—large, unwieldy, and resistant to constant change.
With the increase in competition in the business environment and the proliferation of the channels, businesses are looking for faster innovation and to provide seamless customer experience. On the other hand, they do not want to throw away the existing investment in the existing applications.
In this scenario, enterprises are undertaking multiple programs to re-factor and re-architect the existing applications into modern, distributed, microservice-based models that provide the currency of rapid iteration and are future-proof.
Enterprises are attacking this problem in a two-prong manner:
- Setting the base platform that provides the core ecosystem as a set of services to deploy and run the microservices. These services include Configuration Management, Service Discovery, Elastic Compute, Container Management, Security, Management and Monitoring, DevOps pipeline, and more. Enterprises typically weigh in between using the public cloud and setting up a private cloud. The choice of cloud platform depends on the industry in question and the maturity of the enterprise strategy.
- The second approach is to chip at the monolithic application, one functional piece at a time, and migrate the core business logic to the microservice model. The GUI part is separately migrated to an SPA model using frameworks such as AngularJS and ReactJS. For example, a lot of e-commerce enterprises have moved their catalogue and search services to elastic cloud providers. Only when the customer clicks the checkout do they bring the customer to the in-house data center.
Once the enterprise has set up the ecosystem with respect to platform services, the ability to add more microservice-based functionality becomes easy, providing the required impetus in terms of business agility and innovation.
We will cover digital transformation in more detail in Chapter 12, Digital Transformation.
In this chapter, we covered what cloud-native programming is and why to go for it. We saw what the various adoption models for enterprises, when it comes to cloud-native applications, are. We covered the 12 factors for distributed applications, along with the usage of microservice-based design for cloud-native enablement. We covered the enablement ecosystem for building microservices-based applications.
As we progress through the book, we will cover how to design, build, and run your cloud-native application. We will also cover cloud-native application development using two cloud provider platforms—AWS and Azure. We will make use of their platform services to build a cloud-native application.
We will also cover operational aspects of the cloud-native application—DevOps, deployment, monitoring, and management. Lastly, we will cover how to transform existing monolithic applications into modern distributed cloud-native applications. In the next chapter, we will dive right into creating our first cloud-native application.
