Download code from GitHub
Chapter 1: Designing and Architecting the Enterprise Application
Enterprise applications are software solutions designed to solve large and complex problems for enterprise organizations. They enable Order-to-Fulfillment capabilities for enterprise customers in the IT, government, education, and public sectors. They empower them to digitally transform their businesses with capabilities such as product purchasing, payment processing, automated billing, and customer management. When it comes to enterprise applications, the number of integrations is quite high, and the volume of users is also very high as, typically, applications are targeted at a global audience.
To ensure that enterprise systems remain highly reliable, highly available, and highly performant, getting the design and architecture right is very important. Design and architecture form the foundation of any good software. They form the basis of the rest of the software development life cycle; therefore, it is very important to, first, get the right design to avoid any rework later, which could prove very expensive, depending on the changes required. So, you need a flexible, scalable, extensible, and maintainable design and architecture.
In this chapter, we will cover the following topics:
- A primer on common design principles and patterns
- Understanding common enterprise architectures
- Identifying enterprise application requirements (business and technical)
- Architecting an enterprise application
- Solution structuring for an enterprise application
By the end of this chapter, you will be able to start designing and architecting enterprise applications.
A primer on common design principles and patterns
Every piece of software in the world solves at least one real-world problem. As time goes by, things change, including what we expect from any specific software. To manage this change and deal with various aspects of software, engineers have developed several programming paradigms, frameworks, tools, techniques, processes, and principles. These principles and patterns, proven over time, have become guiding stars for engineers to build quality software.
Principles are high-level abstract guidelines to be followed while designing. They are applicable regardless of the programming language being used. They do not provide implementation guidelines.
Patterns are low-level specific implementation guidelines that are proven, reusable solutions for recurring problems. First, let's start with design principles.
Design principles
Techniques become principles if they are widely accepted, practiced, and proven to be useful in any industry. Those principles become solutions to make software designs more understandable, flexible, and maintainable. In this section, we will cover the SOLID, KISS, and DRY design principles.
SOLID
The SOLID principles are a subset of the many principles promoted by an American software engineer and instructor, Robert C. Martin. These principles have become the de facto standard principles in the OOP world and have become part of the core philosophy for other methodologies and paradigms.
SOLID is an acronym for the following five principles:
- Single-responsibility principle (SRP): An entity or software module should only have a single responsibility. You should avoid granting multiple responsibilities to one entity.
Figure 1.1 – SRP
- Open-closed principle (OCP): Entities should be designed in such a way that they are open for extension but closed for modification. This means the regression testing of existing behaviors can be avoided; only extensions need to be tested.
Figure 1.2 – OCP
- Liskov substitution principle (LSP): Parent or base class instances should be replaceable with instances of their derived classes or subtypes without altering the sanity of the program.
Figure 1.3 – LSP
- Interface segregation principle (ISP): Instead of one common large interface, you should plan multiple, scenario-specific interfaces for better decoupling and change management:
Figure 1.4 – ISP
- Dependency inversion principle (DIP): You should avoid having any direct dependency on concrete implementations. High-level modules and low-level modules should not depend on each other directly. Instead, both should depend on abstractions as much as possible. Abstractions should not depend on details, and details should depend on abstractions.
Figure 1.5 – DIP
Don't Repeat Yourself (DRY)
With DRY, a system should be designed in such a way that the implementation of a feature or a pattern should not be repeated in multiple places. This would result in maintenance overhead, as a change in requirements would result in modifications being needed at multiple places. If you fail to make a necessary update in one place by mistake, the behavior of the system will become inconsistent. Rather, the feature should be wrapped into a package and should be reused in all places. In the case of a database, you should look at using data normalization to reduce redundancy.
Figure 1.6 – DRY
This strategy helps in reducing redundancy and promoting reuse. This principle helps an organization's culture too, encouraging more collaboration.
Keep it simple, stupid (KISS)
With KISS, a system should be designed as simply as possible, avoiding complicated designs, algorithms, new untried technologies, and more. You should focus on leveraging the right OOP concepts and reusing proven patterns and principles. Include new or non-simple things only if it is necessary and adds value to the implementation.
When you keep it simple, you will be able to do the following better:
- Avoid mistakes while designing/developing.
- Keep the train running (there is always a team whose job is to maintain the system, even though they are not the team that developed the system in the first place).
- Read and understand your system code (your system code needs to be understandable to people who are new to it or for people who will use it in the future).
- Do better and less error-prone change management.
With this, we are done with our primer on common design principles; we have learned about SOLID, DRY, and KISS. In the next section, we'll look at some common design patterns in the context of real-world examples to help you understand the difference between principles and patterns and when to leverage which pattern—a skill that's essential for good design and architecture.
Design patterns
While following design principles in the OOP paradigm, you might see the same structures and patterns repeating over and again. These repeating structures and techniques are proven solutions to common problems and are known as design patterns. Proven design patterns are easy to reuse, implement, change, and test. The well-known book, Design Patterns: Elements of Reusable Object-Oriented Software, comprising what is known as the Gang of Four (GOF) design patterns, is considered the bible of patterns.
We can categorize the GOF patterns as follows:
- Creative: Helpful in creating objects
- Structural: Helpful in dealing with the composition of objects
- Behavioral: Helpful in defining the interactions between objects and distributing responsibility
Let's look at these patterns with some real-life examples.
Creational design patterns
Let's take a look at some creational design patterns, along with relevant examples, in the following table:
Table 1.1 – Creational design patterns
Structural design patterns
The following table includes some examples of structural design patterns:
Table 1.2 – Structural design patterns
Behavioral design patterns
The following table includes some examples of behavioral design patterns:
Table 1.3 – Behavioral design patterns
Sometimes, you can become overwhelmed by all these patterns being inside the table. But really, any design is a good design until it violates the basic principles. One rule of thumb that we can use is to go back to the basics, and in design, principles are the basics.
Figure 1.7 – Patterns versus principles
With this, we are done with our primer on common design principles and patterns. By now, you should have a good understanding of the different principles and patterns, where to use them, and what it takes to build a great solution. Now, let's spend some time looking at common enterprise architectures.
Understanding common enterprise architectures
There are a few principles and architectures that are commonly practiced when designing enterprise applications. First and foremost, the goal of any architecture is to support business needs at the lowest cost possible (costs being time and resources). A business wants software to enable it rather than act as a bottleneck. In today's world, availability, reliability, and performance are the three KPIs of any system.
In this section, first, we will look at the issues with monolithic architectures, and then we will see how to avoid them by using widely adopted and proven architectures for developing enterprise applications.
Consider a classical monolithic e-commerce website application, such as the one shown in the following diagram, with all the business providers and functionality in a single app and data being stored in a classical SQL database:
Figure 1.8 – A monolithic app
The monolithic architecture was widely adopted 15–20 years ago, but plenty of problems arose for software engineering teams when systems grew and business needs expanded over time. Let's look at some of the common issues with this approach.
Common issues with monolithic apps
Let's take a look at the scaling issues:
- In a monolithic app, the only way to horizontally scale is by adding more compute to the system. This leads to higher operational costs and unoptimized resource utilization. Sometimes, scaling becomes impossible due to conflicting needs in terms of resources.
- As all the features mostly use single storage, there is the possibility of locks leading to high latency, and there will also be physical limits as to how far a single storage instance can scale.
Here is a list of issues associated with availability, reliability, and performance:
- Any changes in the system will require the redeployment of all components, leading to downtime and low availability.
- Any non-persistent state, such as sessions stored in a web app, will be lost after every deployment. This will lead to the abandonment of all workflows that were triggered by users.
- Any bugs in a module, such as memory leaks or security bugs, make all the modules vulnerable and have the potential to impact the whole system.
- Due to the highly coupled nature and sharing of resources within modules, there will always be unoptimized use of resources, leading to high latency in the system.
Lastly, let's see what the impact on the business and engineering teams is:
- The impact of a change is difficult to quantify and requires extensive testing. Hence, it slows down the rate of delivery to production. Even a small change will require the entire system to be deployed again.
- In a single highly coupled system, there will always be physical limits on collaborations across teams to deliver any features.
- New scenarios such as mobile apps, chatbots, and analysis engines will take more effort as there are no independent reusable components or services.
- Continuous deployment is almost impossible.
Let's try to solve these common problems by adopting some proven principles/ architectures.
Separation of concerns/single-responsibility architecture
Software should be divided into components or modules based on the kind of work it performs where every module or component owns a single responsibility from the entire software's responsibility. Interaction between components happens via interfaces or messaging systems. Let's look at the n-tier and microservices architecture and how the separation of concerns is taken care of.
N-tier architecture
N-tier architecture divides the application of a system into three (or n) tiers:
- Presentation (known as the UX layer, the UI layer, or the work surface)
- Business (known as the business rules layer or the services layer)
- Data (known as the data storage and access layer)
Figure 1.9 – N-tier architecture
These tiers can be owned/managed/deployed separately. For example, multiple presentation layers, such as the web, mobile, and bot layers, can leverage the same business and data tier.
Microservices architecture
Microservices architecture consists of small, loosely coupled, independent, and autonomous services. Let's see their benefits:
- Services can be deployed and scaled independently. An issue in one service will have a local impact and can be fixed by just deploying the impacted service. There is no compulsion to share technology or frameworks.
- Services communicate with each other via well-defined APIs or messaging systems such as the Azure service bus.
Figure 1.10 – Microservices architecture
As you can see in the preceding diagram, a service can be owned by independent teams and have its own cycle. Services are responsible for managing their own data stores. Scenarios demanding lower latency can be optimized by bringing in a cache or high-performance NoSQL stores.
Stateless services architecture
Services should not have any state. State and data should be managed independently from services, that is, externally through a data store such as a distributed cache or a database. By delegating the state externally, services will have the resources to serve more requests with high reliability. The following diagram shows an example of stateful services on the left-hand side. Here, state is maintained in each service through an in-memory cache or session provider, whereas a stateless service, as shown on the right-hand side, manages state and data externally.
Figure 1.11 – Stateful (left) versus stateless (right)
Session affinity should not be enabled as it leads to sticky session issues and will stop you from getting the benefits of load balancing, scalability, and the distribution of traffic.
Event-driven architecture
The main features of event-driven architectures are listed as follows:
- In an event-driven architecture, communication, which is generally known as publisher-subscriber communication, between modules, is primarily asynchronous and achieved via events. Producers and consumers are totally decoupled from each other. The structure of the event is the only contract that is exchanged between them.
- There can be multiple consumers of the same event taking care of their specific operations; ideally, they won't even be aware of each other. Producers can continuously push events without worrying about the availability of consumers.
- Publishers publish events via a messaging infrastructure such as queues or a service bus. Once an event has been published, the messaging infrastructure is responsible for sending the event to eligible subscribers.
Figure 1.12 – Event-driven architecture
This architecture is best suited for scenarios that are asynchronous in nature. For example, long-running operations can be queued for processing. A client might poll for status or even act as a subscriber for an event.
Resiliency architecture
As the communication between components increases, so does the possibility of failures. A system should be designed to recover from any kind of failure. We will cover a few strategies for building a fault-tolerant system that can heal itself in the case of failures.
If you are familiar with Azure, you'll know that applications, services, and data should be replicated globally in at least two Azure regions for planned downtime and unplanned transient or permanent failures, as shown in the following screenshot. In these scenarios, choosing Azure App Service to host web applications, using REST APIs, and choosing a globally distributed database service, such as Azure Cosmos DB, is wise. Choosing Azure paired regions will help in business continuity and disaster recovery (BCDR), as at least one region in each pair will be prioritized for recovery if an outage affects multiple regions.
Figure 1.13 – Resiliency architecture
Now, let's see how to tackle different types of faults.
Transient faults can occur in any type of communication or service. You need to have a strategy to recover from transient faults, such as the following:
- Identify the operation and type of transient fault. Then, determine the appropriate retry count and interval.
- Avoid anti-patterns such as endless retry mechanisms with a finite number of retries or circuit breakers.
If a failure is not transient, you should respond to the failure gracefully by choosing some of the following options:
- Failing over
- Compensating for any failed operations
- Throttling/blocking the bad client/actor
- Using a leader election to select a leader in the case of a failure
Here, telemetry plays a big role; you should have custom metrics to keep a tab on the health of any component. Alerts can be raised when a custom event occurs or a specific metric reaches a certain threshold.
With this, we are done with our coverage of common enterprise architectures. Next, we will look at the requirements of enterprise applications and their different architectures through the lens of the design principles and common architectures that we learned about earlier.
