Home Data Machine Learning in Microservices

Machine Learning in Microservices

By Mohamed Abouahmed , Omar Ahmed
books-svg-icon Book
eBook $35.99 $24.99
Print $44.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $35.99 $24.99
Print $44.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Chapter 1: Importance of MSA and Machine Learning in Enterprise Systems
About this book
With the rising need for agile development and very short time-to-market system deployments, incorporating machine learning algorithms into decoupled fine-grained microservices systems provides the perfect technology mix for modern systems. Machine Learning in Microservices is your essential guide to staying ahead of the curve in this ever-evolving world of technology. The book starts by introducing you to the concept of machine learning microservices architecture (MSA) and comparing MSA with service-based and event-driven architectures, along with how to transition into MSA. Next, you’ll learn about the different approaches to building MSA and find out how to overcome common practical challenges faced in MSA design. As you advance, you’ll get to grips with machine learning (ML) concepts and see how they can help better design and run MSA systems. Finally, the book will take you through practical examples and open source applications that will help you build and run highly efficient, agile microservices systems. By the end of this microservices book, you’ll have a clear idea of different models of microservices architecture and machine learning and be able to combine both technologies to deliver a flexible and highly scalable enterprise system.
Publication date:
March 2023
Publisher
Packt
Pages
270
ISBN
9781804617748

 

Importance of MSA and Machine Learning in Enterprise Systems

In today’s market, the competition has never been fiercer, and user requirements for IT systems are constantly increasing. To be able to keep up with customer requirements and market demands, the need for a shorter time-to-market (TTM) for IT systems has never been more important, all of which has pushed for agile deployment and the need to streamline the development process and leverage as much code reuse as possible.

Microservices architecture (MSA) addresses these concerns and tries to deliver a more competitive, reliable, and rapid deployment and update delivery while maintaining an efficient, stable system operation.

In this chapter, we will learn more details about how microservices help build a modern, flexible, scalable, and resilient enterprise system. The chapter will go over key concepts in MSA and discuss the common enterprise system architectures, how each architecture is different from MSA, why they are different, and what you gain or lose when you adopt one or more architectures over the others.

We will cover the following areas as we go over the chapter:

  • What MSA is and why
  • MSA versus monolithic enterprise systems
  • Service-driven architecture, event-driven architecture (EDA), and how to incorporate that in MSA
  • Challenges of deploying and operating MSA enterprise systems
  • Why it is important to embrace DevOps in building MSA
 

Why microservices? Pros and cons

Microservices is often likened to MSA. MSA refers to the way in which a complex system is built from a collection of smaller applications, where each application is designed for a specific limited-scope function. These small applications (or services, or microservices) are independently developed and can be independently deployed.

Each microservice has an API interface for communicating with other microservices in the system. The way all these individual microservices are organized together forms the larger system function.

In order to understand the value of microservices and the challenges one faces in designing an MSA, it is imperative to understand how microservices communicate and interact with each other.

Microservices can communicate together in a linear or non-linear fashion. In a linear microservices pipeline, each microservice communicates with another microservice, processing data across the system in a sequential manner. The input is always passed to the first microservice, and the output is always generated by the last microservice in the system:

Figure 1.1: Linear microservices pipeline

Figure 1.1: Linear microservices pipeline

Practically, however, most existing systems are formed using a non-linear microservices pipeline. In a non-linear microservices pipeline, data is distributed across different functions in the system. You can pass the input to any function in the system, and the output can be generated from any function in the system. You can therefore have multiple pipelines with multiple inputs, serving multiple functions and producing multiple outputs:

Figure 1.2: Non-linear microservices pipeline

Figure 1.2: Non-linear microservices pipeline

Consider the following diagram of a simplified order fulfillment process in a typical e-commerce system. Each function within the Placing an Order process represents a microservice. Once an order is placed by a customer, an API call is triggered to the Add/Update Customer Information microservice to save that customer’s information or update it if needed. This microservice sole responsibility is just that: manage customer information based on the data input it receives from the API caller.

Another API call is issued at the same time to the Verify Payment part of the process. The call will be directed to either the Process PayPal Payment or the Process Credit Card Payment microservice depending on the payment type of the API call. Notice here how the payment verification process is broken down into two different microservices—each is specifically designed and developed for a specific payment function. This enables the flexibility and portability of these microservices to other parts of the system or to another system if needed.

After payment is processed, API calls are triggered simultaneously to other microservices in the system to fulfill the order:

Figure 1.3: A non-linear microservices pipeline example – customer order

Figure 1.3: A non-linear microservices pipeline example – customer order

The order placement example shows how modular and flexible designing an MSA enterprise system can be. We will often use this example to show some of the advantages and challenges one may face when designing, deploying, and operating an MSA enterprise system.

It is essential that we go over some of the advantages and disadvantages of building enterprise systems using MSA to help decide whether MSA is a better option for your organization or not.

Note that some of the advantages listed next could also be considered disadvantages in other situations (and vice versa).

Advantages of microservices

There is some significant value to implementing MSA. The following are some of the advantages we see applicable to today’s market.

Autonomy

One of the biggest advantages of microservices is their autonomy—it is the keystone for many of the other advantages of MSA. And because of their autonomy, microservices have their own technology stack, which means that each system service can be developed with completely different tools, libraries, frameworks, or programming languages than any other system service, yet they integrate with each other smoothly.

Microservices can be developed and tested independently of any other application within the system, which enables each microservice to have its own life cycle, including quality assurance (QA), change management, upgrades, updates, and so on, which in return greatly minimizes application dependencies.

Portability

Microservices’ autonomy enables them to be portable across platforms, operating systems, and different systems, all independent of the coding language in which these services were written.

Reuse

When reusing microservices, you don’t need to reinvent the wheel. Because of their autonomy, microservices can be reused without the need to add additional coding, changes, or testing. Each service can be reused as needed, which largely increases system flexibility and scalability, significantly reduces the development time, cost, and deployment time, and reduces the system’s TTM.

Loosely coupled, highly modular, flexible, and scalable

Microservices form the main building blocks of an MSA enterprise system. Each block is loosely coupled with the other blocks in the system. Just like Lego blocks, the manner in which these blocks are organized together can form a complex enterprise MSA system building a specific business solution.

The following diagram shows an example of how we can build three different systems with multiple microservices.

The diagram shows nine services, and seven out of these services are organized in such a manner to reuse and build three different systems—system A, system B, and system C. This shows how loose coupling enables flexibility in MSA in such a way that you can reuse each service to build a different system function.

You can build a system with minimal development added to existing microservices either acquired by a third party or previously developed in house. This largely enables rapid system development, new feature releases, very short TTM, and reliable, flexible, and much more stable hot updates and upgrades. All of this increases business continuity (BC) and makes the enterprise system much more scalable:

Figure 1.4: Flexibility and modularity in microservices

Figure 1.4: Flexibility and modularity in microservices

Shorter release cycle and TTM

Because of the individual and independent services features we previously mentioned, the deployment of microservices becomes much easier and faster to perform. Automation can play a great role in reducing time-of-service testing and deployment, as we will discuss later in this chapter.

Fault tolerance and fault isolation

Each microservice has its own separate fault domain. Failures in one microservice will be contained within that microservice, hence it is easier to troubleshoot and faster to fix and bring back the system to full operations.

Consider the order fulfillment example we mentioned earlier; the system can still be functional if the Message/Email Customer microservice—for example—experiences any failures. And because of the nature of the failure and the small fault domain, it will be easy to pinpoint where that failure is and how to fix it. Mean Time to Resolution (MTTR) is therefore significantly reduced, and BC is greatly enhanced.

Architects are sometimes able to build the system with high embedded tolerance to prevent these failures to begin with or have other backup microservices on standby to take over once a failure is detected in the primary microservice. One of the primary objectives of this book, as we will see later, is to be able to design a system with high enough intelligence to provide the desired high resilience.

What software architects have to bear in mind, however, is that, with too many system components in the MSA, too many things can go wrong. Architects and developers, therefore, have to have solid fallback and error handling to manage the system’s resilience.

The communication between the different microservices, for example, can simply time out for whatever reason; it could be a network issue, a server issue, or too many API calls at the receiving microservices or at the event-handling mechanism developed in the system, overwhelming this system component and causing failures or delayed response.

There are many data flow streams and data processing points in the system that all need to be synchronized. A single failure, if not taken care of properly by the system, can create system-cascading failures, and accordingly could cause a failure to the entire system.

How fault tolerance is designed will be a big factor in how system performance and reliability are impacted.

Reliability and the Single Responsibility Principle (SRP)

If you come from the programming world, you are probably familiar with the SRP in object-oriented programming (OOP): A class should have one, and only one, reason to change. Every object, class, or function in the system should have a responsibility over only that functionality of the system, and hence that class, once developed, should only change for the reason it was originally created for. This principle is one of the main drivers of increased system reliability and BC in MSA.

At the initial phases of developing an MSA enterprise system, and during the phase of developing new microservices from scratch, the MSA enterprise system may not be fully tested or fully matured yet, and reliability may still be building up. When the system matures, changes to individual microservices are minimal—if any— and microservices’ code reliability is, therefore, higher, the operation is more stable, fault domains are contained, fault tolerance is high, and the system’s reliability thus becomes much higher than similar systems with a monolithic architecture. Reliability is highly contingent on how well the system is designed, developed, and deployed.

Reducing system development and operational cost

Reusing microservices largely reduces the development efforts and time needed to bring the system to life. The more microservices you can reuse, the lower the development time and cost will become.

Microservices do not have to be developed from scratch; you can purchase already developed microservices that you may need to plug into your MSA enterprise system, cutting the development time significantly.

When these microservices are stable and mature, reliability is higher, MTTR is much shorter, and hence system faults are lower and BC is higher. All these factors can play a major role in reducing the development cost, operational cost, and total cost of ownership (TCO).

Automation and operational orchestration are ideal for microservices; this enables agile development and can also decrease operational costs significantly.

Disadvantages of microservices

Microservices come with a set of challenges that need to be taken into consideration before considering an MSA in your organization. The good news is that many of these challenges—if not all—can effectively be addressed to have in the end a robust MSA enterprise system.

Mentioned here are some of the challenges of microservices, and we will later in this chapter talk about some of the methodologies that help address these challenges.

Complexity

MSA systems contain many components that must work together and communicate together to form the overall solution. The system’s microservices in most cases are built with different frameworks, programming languages, and data structures.

Communication between microservices has to be in perfect synchronization for the system to properly function. Interface calls could at times overwhelm the microservice itself or the system as a whole, and therefore, system architects and developers have to continuously look for mechanisms to efficiently handle interface calls and try to eliminate dependencies as much as they can.

Designing the system to handle call loads, data flows, and data synchronization, along with the operational aspects of it, could be a very daunting process and creates layers of complexity that are hard to overlook.

Complexity is one of the main trade-off factors in implementing and running an MSA enterprise system.

Initial cost

MSA systems usually require a large number of resources to be able to handle the individual processing needs of each microservice, the high level of communication between microservices, and the different development and staging environments for developing these microservices.

If these microservices are being developed from scratch, the initial cost of building an MSA system would therefore be too high. You have to account for the cost of the many individual development environments, the many microservices to develop and test, and the different teams to do all these tasks and integrate all these components. All this adds to the cost of the initial system development.

Tight API control

Each microservice has its own API calls to be able to integrate with other microservices in the system. Any change in the API command reference set—such as updates in any API call arguments, deprecated APIs, or changes in the return values—may require a change in how other microservices handle the data flow from and to that updated microservice. This can pose a real challenge.

Developers have to either maintain backward compatibility (which can be a big constraint at times) or change the API calls’ code of every other component in the system that interacts with the updated microservice.

System architects and developers have therefore to maintain very tight control over API changes in order to maintain system stability.

Data structure control and consistency

The drawback of having independent applications within the enterprise system is that each microservice will have to maintain its own data structure, which creates a challenge in maintaining data consistency across your system.

If we take the earlier example of customer order fulfillment, the Add/Update Customer Information microservice should have its own database totally independent from any other database in the system. Similarly, the Update Item Inventory microservice should be the microservice responsible for the item information database, the Update Orders Database microservice should have the orders database, and so on.

The challenge now is that the shipping database will need to be in sync with the customer information database, and the orders database will have to contain some of the customer information. Also, the Message/Email Customer microservice has to have a way to access customer information (or receive customer information through API calls), and so on. In a larger system, the process of keeping data consistent across the different microservices becomes problematic. The more microservices we have, the more complex the data synchronization becomes.

Once again, designing and developing a system with all that work in mind becomes another burden on the system architects and developers.

Performance

As we mentioned earlier, microservices have to communicate with each other to perform the entire system function. This communication, data flows, error handling, and fault-tolerance design—among many other factors—are susceptible to network latency, network congestions, network errors, application data processing time, database processing time, and data synchronization issues. All these factors greatly impact system performance.

Performance is another major trade-off factor in adopting and running an MSA enterprise system.

Security

Because of microservices’ autonomy and their loose coupling, a high number of data exchanges between the different services is necessary for the MSA to function. This data flow, data storage within each microservice, data processing, the API call itself, and transaction logging all significantly increase the system attack surface and develop considerable security concerns.

Organizational culture

Each microservice in the MSA has its own development cycle and therefore has its silo of architects, developers, testers, and the entire development and release cycle teams, all to maintain the main objective of microservices: their autonomy.

MSA enterprise systems are built from a large number of microservices and mechanisms to manage the interaction between the different system components. Developers have to therefore have system operational knowledge, and the operational teams need to have development knowledge.

Testing such complex distributed environments that one will have in the MSA system becomes a very daunting process that needs a different set of expertise.

The traditional organizational structure of one big development team solely focused on development, one QA team only doing basic testing, and so on is no longer sufficient for the way MSA is structured and operated.

Agile development and DevOps methodologies are very well suited for microservices development. You need agile processes to help maintain the fast development and release cycles MSA promises to deliver. You need DevOps teams who are very familiar with the end-to-end process of designing the application itself and how it fits in the big picture, testing the application, testing how it functions within the entire system, the release cycle, and how to monitor the application post release.

All this requires a cultural shift and significant organizational transformation that can enable DevOps and agile development.

Important note

We rarely see a failure in MSA adoption because of technical limitations; rather, failure in adopting MSA is almost always due to a failure to shift the organization’s culture toward a true DevOps and agile culture.

The benefits outweigh the detriments

The main questions you need to answer now are: Is building an MSA worth it? Can we make it happen given the current organizational culture? How long will it take the organization to transform and be ready for MSA? Do we have the luxury of waiting? Can we do both the organizational transformation and the building of the MSA enterprise system at the same time? Do we have the resources and the caliber necessary for the new organizational structure? Is cost an issue, and do I have the budget to cover that?

Well, first of all, if you are planning to build a large enterprise system, and you have the budget and necessary resources for starting this project, building the system as MSA is definitely worth it. All initial costs endured and time spent will eventually be offset by the long-term cost and time-saving benefits of having an MSA system.

Nevertheless, you are still the one to best address all these previous questions. There are overwhelming and compelling advantages to adopting MSA, but as we have seen, this is not a simple undertaking; so, whether an organization is willing to walk that path or not is something it—and only it—can answer.

Now we know what the advantages of deploying an MSA are, and the challenges that come with MSA adoption, we will now go over different enterprise architecture styles, what they are, and the differences between each other.

 

Loosely versus tightly coupled monolithic systems

Traditional applications back in the day were mostly built using a monolithic architecture, in which the entire application was one big code base. All system components and functions were tightly coupled together to deliver the business solution.

As shown in the following diagram, system functions are all part of the same code, tightly coupled with centralized governance. Each system function has to be developed within the same framework of the application.

In an MSA system, however, each function preserves its own anonymity—that is, loosely coupled with decentralized governance, giving each team the ability to work with its own preferred technology stack, with whichever tools, framework, and programming language it desires:

Figure 1.5: Monolithic versus microservices systems

Figure 1.5: Monolithic versus microservices systems

All functions in the monolithic architecture application are wrapped into the application itself. In the MSA, these functions are developed, packaged, and deployed separately. Therefore, we can run these services in multiple locations’ on-premises infrastructure, in the public cloud, or across both on-premises and the cloud in a hybrid-cloud fashion.

In monolithic systems, and because of the tight coupling, synchronizing the different system function changes is a development and operational nightmare. If one application (for whatever reason) becomes unstable, it could cause a failure to the entire system, and bringing the system back to a stable point becomes a real pain.

In the case of microservices, however, since each of these microservices is loosely coupled, changes and troubleshooting are limited to that particular microservice, as long as the microservice interface does not change.

One large piece of code, in the case of monolithic architecture, is very hard to manage and maintain. It is also hard to understand, especially in large organizations where multiple developers are working together.

In many cases such as employee turnover, for example, a developer may need to troubleshoot someone else’s code, and when the application is written in a single big piece of code, things tend to be complicated, hard to trace and understand, and hard to reverse engineer and fix. Code maintenance becomes a serious problem, while in the microservices case, this humongous line of code is broken into smaller chunks of code that are easier to read, understand, troubleshoot, and fix, totally independent of the other components of the system.

When code changes are needed in monolithic architecture, a single change to part of the code may need changes to many other parts of the application, and accordingly, change updates will likely require a rewrite and a recompile of the entire application.

We can also reuse and package different applications together in a workflow to form a specific service, as shown previously in Figure 1.4.

It is just common sense to break down a complex application into multiple modules or microservices, each performing a specific function in the entire ecosystem for better scalability, higher portability, and more efficient development and operations.

For small, simple, and short-lived systems, monolithic applications may be a better fit for your organization, easier to design and deploy, cheaper to develop, and faster to release. As the business needs grow, MSA becomes a better long-term approach.

Since monolithic systems are tightly coupled, there is no need for API communication between the different system functions; this significantly decreases the security surface of your system, lowering system security risks and increasing the system’s overall performance.

Think of the deployment difference between both monolithic and MSA as the difference between an economy car and a Boeing 787. The car is a better, cheaper, and faster tool for traveling between two cities 50 miles apart, with no need for the security checks you experience in airports before boarding your flight. As the distance increases, however, driving the car becomes more hassle. At 5,000 miles, the Boeing 787 is likely to become a better, cheaper, and faster way to get to your destination, and you will likely be willing to put up with the hassle of security checks you have to undergo to be able to board your flight.

The following is a comparison summary between both monolithic and microservices applications:

Monolithic

MSA

Architecture

Highly autonomous. System functions are split into independent loosely coupled chunks of smaller code.

No autonomy. System functions are all tightly coupled into one big piece of code.

Portability

Highly portable

Very limited portability

Reuse

Highly reusable

Very limited ability to reuse code

Modularity and Scalability

Highly modular and scalable

Limited modularity and hard to scale

Initial TTM

Highly dependent on the readiness of individual system services. The more code reuses, the shorter the TTM is.

If the system microservices are being designed and developed from scratch, TTM is usually longer for monolithic architecture.

Long TTM, especially in large systems. Shorter TTM in small and simple systems.

Release Cycle

Very short release cycle, super-fast to deploy changes and patch updates

Long and usually very time-consuming release cycles and patch updates

Initial Cost

Usually high. Depends on the system size.

The initial cost is offset by operational cost savings.

Usually low. The initial size becomes higher in large enterprise systems.

Operational Cost

Low. Easier to maintain and operate.

High. Hard to maintain and operate.

Complexity

High

Low

API Control

High

Low

Data Structure Consistency

Decentralized databases, hence data consistency is harder to maintain

A centralized database, hence easier to maintain data consistency across the system

Performance

Usually lower

Usually higher

Security

Many security concerns

Lower security concerns

Organizational Adoption

Hard to adopt depending on the organizational structure. Requires adoption of agile development and DevOps. Organizational transformation may be required and may take a long time to achieve.

Easy to adopt. Minimal organizational transformation needed—if any.

Fault Tolerance

Usually higher

Usually lower

Table 1.1: Summary of the differences between monolithic and MSA systems

We covered in this section the different aspects of a monolithic system; next, we go over service-driven architecture and EDA, and how to combine these architectural styles within MSA to address some of the MSA challenges discussed earlier.

 

Service-driven, EDA, and MSA hybrid model architecture

People often get mixed up between MSA and service-driven architecture (aka service-oriented architecture or SOA). Both types of architecture try to break down the monolithic architecture system into smaller services. However, in MSA, the system services decomposition is extremely granular, breaking down the system into very fine specialized independent services. In the SOA, the system services decomposition is instead coarse-grained to the domain level.

All domains, as shown in the following diagram, share the same centralized database and may actually share other resources in between, creating some level of coupling and system dependencies that are non-existent in MSA. Data storage is a key difference between both architectural styles:

Figure 1.6: SOA architecture split into functional domains

Figure 1.6: SOA architecture split into functional domains

In the case of the simplified MSA customer ordering example discussed earlier, there are eight different microservices. A similar implementation in SOA is likely to have all these microservices built together and tightly coupled in a single domain. Other domains within the system could be Cart Handling, Catalog Browsing and Suggestions, and so on.

SOA has a holistic enterprise view, while in a microservice, development looks into the function itself in total isolation of the enterprise system in which the microservice is intended to be used.

EDA is another architectural style that is largely adopted. While MSA’s main focus is on function and SOA emphasizes the domain, EDA instead focuses on system events.

EDA is usually complemented by another main system architecture, such as SOA or MSA. In EDA, services are decoupled at a granularity level determined by its main architecture (MSA or SOA) and then communicate with each other through event-based transactions. In our order placement example, these events could be Order Created, Order Canceled, Order Shipped, and so on.

In order to maintain event synchronization and data consistency across the enterprise system, these events must be handled by a message broker. The message broker’s sole responsibility is to guarantee the delivery of these events to different services across the system. Therefore, it has to be highly available, highly responsive, fault-tolerant, and scalable and must be able to function under heavy load.

When EDA is adopted within the MSA enterprise system, the message broker in that case will be handling events, API calls, and API calls’ responses.

The message broker has to be able to queue messages when a specific service is down or under heavy load and deliver that message whenever that service becomes available.

ACID transactions

Any system with some form of data storage always needs to ensure the integrity, reliability, and consistency of that data. In MSA, systems store and consume data across the workflow transactions, and for individual services to ensure integrity and reliability for the MSA system as a whole, data stored within the entire system have to comply with a certain set of principles called Atomicity, Consistency, Isolation, and Durability (ACID):

  • Atomicity: All-or-nothing transactions. Either all transactions in the workflow are successfully executed and committed or they all fail and are canceled.
  • Consistency: Any data change in one service has to maintain its integrity across the system or be canceled.
  • Isolation: Each data transaction has its own sovereignty and should not impact or be impacted by other transactions in the system.
  • Durability: Committed transactions are forever permanent, even in the case of a system failure.

Saga patterns

One of the main challenges in MSA is distributed transactions, where data flow spans across multiple microservices in the system. This flow of data across the services creates a risk of violating the microservice autonomy. Data has to be managed within the microservice itself in total isolation from any other service in the system.

If you look at our order placement example again, you find that customer data (or part of it) spans across the different microservices in the example, which could create undesired dependencies in the MSA, and should be avoided at all costs.

What if, for whatever reason, the Update Item Inventory service fails, or it just happens that the service reports back that the item is no longer available? The system in that case will need to roll back and update all individual services’ databases to ensure ACID transactions for the workflow.

The saga pattern manages the entire workflow of transactions. It sees all sets of transactions performed in a specific process as a workflow and ensures that all these transactions in that workflow are either successfully executed and committed or rolled back in case the workflow breaks for whatever reason, to maintain data consistency across the system.

A saga participant service would have a local transaction part of that workflow. A local transaction is a transaction performed within the service itself and produces an event upon execution to trigger the next local transaction in the workflow. These transactions must comply with ACID principles. If one of these local transactions fails, the saga service initiates a set of compensating transactions to roll back any changes caused by the already executed local transactions in the workflow.

Each local transaction should have corresponding compensating transactions to be executed to roll back actions caused by the local transaction, as shown in the following diagram:

Figure 1.7: Processing of local and compensating transactions

Figure 1.7: Processing of local and compensating transactions

There are two ways to coordinate transactions’ workflow in a saga service: choreography and orchestration.

In choreography, saga participant services exchange events without the need for a centralized manager. As in EDA, a message broker is needed to handle event exchanges between services, as illustrated in the following diagram:

Figure 1.8: Choreography in a saga service

Figure 1.8: Choreography in a saga service

In orchestration, a saga pattern-centralized controller is introduced: an orchestrator. The workflow is configured in the orchestrator and the orchestrator sends requests to each saga participant service on which local transaction it needs to execute, receives events from saga participant services, checks the status of each request, and handles any local transaction failures by executing the necessary compensating transactions, as illustrated in the following diagram:

Figure 1.9: Orchestration in a saga service

Figure 1.9: Orchestration in a saga service

Orchestrators become the brain of the enterprise system and the single source for all steps that need to be taken to execute a specific system workflow. The orchestrator, therefore, must be implemented in a way to be highly resilient and highly available.

Command Query Responsibility Segregation (CQRS)

It is very common in traditional systems, and especially in monolithic applications, to have a common relational database deployed in the backend and accessed by a frontend application. That centralized database is accessed with Create-Read-Update-Delete (CRUD) operations.

In modern architecture, especially as the application scales, this traditional implementation poses a problem. With multiple CRUD requests being processed on the database, table joins are created with a high likelihood of database locking happening. Table locks introduce latency and resource competition, and greatly impact overall system performance.

Complex queries have a large number of table joins and can lock the tables, preventing any write or update operations on them till the query is done and the database unlocks the tables. Database read operations are typically multiple times more than write operations, and in heavy transaction systems, the problem can multiply.

You can see a comparison of CRUD and CQRS patterns here:

Figure 1.10: CRUD versus CQRS patterns

With CQRS, you simply separate one object into two objects. So, rather than doing both commands and queries on one object, we separate that object into two objects—one for the command, and one for the query. A command is an operation that changes the state of the object, while a query does not change the state of the system but instead returns a result.

In our case, the object here is the system database, and that database separation could be either physical or logical. Although it is a best practice to have two physical databases for CQRS, you can still use the same physical database for both commands and queries. You can, for example, split the database into two logical views—one for commands and one for queries.

A replica is created from the master database when two physical databases are used in CQRS. The replica will, of course, need to be synchronized with the master for data consistency. The synchronization can be accomplished by implementing EDA where a message broker is handling all system events. The replica subscribes to the message broker, and whenever the master database publishes an event to the message broker, the replica database will synchronize that specific change.

There will be a delay between the exact time at which the master database was actually changed and when that change is reflected in the replica; the two databases are not 100% consistent during that period of time but will be eventually consistent. In CQRS, this synchronization is called eventual consistency synchronization.

When applying CQRS design in MSA, database processing latency is greatly reduced, and hence communication between individual services’ performance is greatly enhanced, resulting in an overall system-enhanced performance.

The database used can be of any type, depending on the business case of that particular service in the MSA. It may very well be a relational database (RDB), document database, graph database, and so on. A NoSQL database could also be an excellent choice.

We discussed previously the MSA from a design and architecture perspective. Operating the MSA system is another aspect that the entire organization must consider for a successful business delivery process. In the next section, we discuss DevOps, how it fits into the MSA life cycle, and why it is important for a successful MSA adoption and operation.

 

DevOps in MSA

DevOps revolves around a set of operational guidelines in the software development and release cycles. The traditional development engineer is no longer living in their confined environment where all the focus is to convert functional specifications into code; rather, they should have an end-to-end awareness of the application.

A DevOps engineer would oversee, understand, and be involved in the entire pipeline from the moment the entire application is planned out, converting business functions into code, building the application, testing it, releasing it, monitoring its operations, and coming back with the feedback necessary for enhancements and updates.

That does not necessarily mean that a DevOps engineer would be responsible for all development and operational task details. Individual responsibilities within the application team may vary in a way to guarantee a smooth continuous integration and continuous deployment (CI/CD) pipeline of the application:

Figure 1.11: DevOps CI/CD pipeline

Figure 1.11: DevOps CI/CD pipeline

One of the main objectives of DevOps is to speed up the CI/CD pipeline; that’s why there is a lot of emphasis on automation in DevOps. Automation is essential to efficiently perform the pipeline.

Automation can help at every step of the way. In DevOps, many test cases that are part of your QA plan are automated, which significantly speeds up the QA process. The release management and monitoring of your application are also automated to provide high visibility, continuous learning, and quick fixes whenever needed. All of this will help organizations improve productivity, predictability, and scalability.

DevOps is a holistic view of how the application is developed and managed. It is not a function for only the development team or operational team to adopt; rather, the entire organization should adopt it. It is therefore imperative for the organizational structure and the organization’s vision and goal to all align with the set of procedural and functional changes necessary to shift from the traditional way of developing software.

Just to give you a gist of how traditional and DevOps models differ in terms of application development and release cycles, take a look at the following comparison table:

Traditional

DevOps

Planning

Months

Long time to plan due to the large application size and tight coupling between different application components

Days to weeks

Very short planning time since the application is broken down into small individual loosely coupled services

Development

Months

Days to weeks, and even shorter in the case of patches and fixes

Testing

Weeks to months

Mostly manually intensive QA use case testing, which may sometimes jeopardize the reliability of the test’s outcome

Days

Mostly automated QA use case execution that brings high reliability to the application

Release, Deploy

Days

Usually long manual work and more susceptible to human errors

Hours

Mostly automated

Operate, Monitor

Metrics reporting is mostly manually pulled and analyzed

Metrics are monitored and analyzed automatically and can even fix the problem in seconds. Moreover, machine learning (ML) tools can be used to enhance operations even further.

Table 1.2: Traditional operational style versus DevOps

In traditional development environments, you have a big piece of code to write, maintain, and change when needed. Because of the code size, it is only normal to have a long release cycle, and it can only be feasible to deploy patches or new releases when only major changes or high-severity fixes are needed, as illustrated in the following diagram:

Figure 1.12: Traditional development environment versus MSA DevOps

Figure 1.12: Traditional development environment versus MSA DevOps

In MSA, teams are separated based on applications that do not function. That big chunk of code is split into a collection of much smaller code (microservices), and since teams are split to work independently for each team to focus on a specialized microservice, the development and release cycles are much shorter.

Similarly, in DevOps, the application is broken down into smaller pieces to enable the CI/CD pipeline, which makes DevOps the perfect model that fits MSA.

Why ML?

Using ML tools and algorithms in your MSA enterprise system can further enhance and accelerate your DevOps CI/CD pipeline. With ML, you can find patterns in your tests, monitor phases of your pipeline, automatically analyze where the faults may be, and suggest a resolution or automatically fix operational issues whenever possible.

ML can greatly shorten your MSA enterprise system’s TTM and make it more intelligent, self-healing, resilient, and supportable.

We will in this book discuss two aspects of ML: first, we’ll explain in detail how to add CI/CD pipeline intelligence to your MSA enterprise system, and second, we’ll look at how to build an ML enterprise system with MSA in mind:

Figure 1.13: Using ML in CI/CD pipeline

Figure 1.13: Using ML in CI/CD pipeline

 

Summary

In this chapter, we covered the concepts of MSA and how MSA is different from traditional monolithic architecture. By now, you should also have a clear understanding of the advantages of MSA and the challenges organizations may experience when adopting MSA.

We also covered the key concept of methodologies to consider when designing MSA, such as ACID, the saga pattern, and CQRS. All these concepts are essential to help overcome synchronization challenges and to maintain microservices anonymity.

We now understand the basics of DevOps and why it is important in MSA design, deployment, and operations, as well as how ML integration in MSA enterprise systems can help enhance system operations.

In the next chapter, we will go over common methodologies that organizations pursue to transition from running traditional monolithic systems to MSA systems. We will discuss how to break down the existing system into services that form the new MSA enterprise system.

About the Authors
  • Mohamed Abouahmed

    Mohamed Abouahmed is a Principal Architect and Consultant providing a unique combination of technical and commercial expertise to resolve complex and business-critical issues through the design and delivery of innovative, technology-driven systems and solutions. He is specialized in network automation solutions and smart systems development and deployment. Mohamed's hands-on experience underpinned by his strong project management and academic background, which includes a PMP Certification, Master of Global Management, Master of Business Administration (International Business), Master of Science in Computer Networking, and BSc in Electronics Engineering, helped him develop and deliver robust solutions for multiple carriers, service providers, enterprises, and Fortune 200 clients.

    Browse publications by this author
  • Omar Ahmed

    Omar Ahmed is an experienced Computer Engineer with a demonstrated history of working in the information technology and IoT industry. Skilled in C/C++, Databases, Java, Management, and Microcontrollers. Strong engineering professional with a Bachelor's degree focused in Computer Engineering from Georgia Institute of Technology.

    Browse publications by this author
Machine Learning in Microservices
Unlock this book and the full library FREE for 7 days
Start now