The notion of software architecture has been around for about 50 years; however, only in the 1990s did it become a part of the computer industry dictionary. Since then, it has undergone some evolution, influenced by other areas of software development and project management. In the Software Architecture and Design chapter of the Microsoft Application Architecture Guide, 2nd Edition document (2009), software application architecture is defined as follows:
"Software application architecture is the process of defining a structured solution that meets all of the technical and operational requirements, while optimizing common quality attributes such as performance, security, and manageability".
Today, the notion of software architecture is typically applied to two major areas of the industry: project development and product development. Project development is characterized by the fact that every project has a beginning and an end, which always has a time frame. At the beginning of the project, there is normally a requirements-gathering process, and the solution that is delivered at the end of the project has to satisfy those requirements. Some requirements describe the business functionality of the solution. Other requirements can specify its availability, extensibility, resilience, security, and many other nonfunctional aspects.
In product development, businesses focus on developing products according to the initial set of requirements or upgrading them according to the subsequent sets of requirements. In a typical process, the software development life cycle (SDLC) has several stages. The stages start from generating an idea of the new product, business analysis, and market research about the coding, testing, delivering to the market, and maintenance. Delivering a new version of a software product can be considered a project itself; however, the entire product development process is cyclic typically, and it does not have a visible end. There are many SDLC methodologies from Waterfall to a whole spectrum of Agile ones; the focus of the architect's role in them is very different from one to another.
In this chapter, we'll consider an architect's approach to a typical software project. We'll discuss the need for an architectural decision framework and sources of information that influence an architect's decisions. We'll talk about evaluating technologies and how organizational context, solution design, implementation, as well as operations have their impact on such an evaluation.
Architects who work on solutions are called solutions architects, and solution architecture will be the main focus of this book. A solutions architect is by no means required to be very technical; however, he/she should possess other capabilities such as understanding organizational dynamics, providing leadership, and quite often, fulfilling the role of a translator between the business and technical teams. Regardless of their involvement in the solution delivery, they always need to make decisions about the technologies that will be used.
Here is where frameworks become very helpful. Frameworks truly give architects a structure to build their decisions on. Without such a structure, there are always gaps; some good decisions would be missed and some simple solutions will be overlooked.
The decision framework that we propose in this book is based on the following principles:
Gather as much information about the current status or the existing solution: For product development, if a new version is to be developed, it consists of the knowledge of the existing product. For project development, this will be the knowledge of how the same or similar problems are being solved in the organization. Gather as many requirements as possible (see the next section, Sources of input, to get more ideas about requirements gathering). Gather as much context information as possible: existing infrastructure, business purpose for the solution, laws and regulations that might apply in the industry, standards, and so on.
Align your decisions with the organizational direction: The next section discusses this principle in detail.
Look for critical and problem areas: The 80/20 principle suggests that 80 percent of time is spent on 20 percent of problems. Try to identify these problems at the beginning of the architectural work. See whether the problems can be solved using some best practices and patterns.
Apply best practices and patterns: True innovations seldom happen in the world of software architecture; most of the solutions have been thought of by hundreds and thousands of other architects, and reinventing the wheel is not necessary.
Capture and evaluate alternatives: Experts are often biased by their previous experience, which blinds them and does not let them consider alternatives. If a few architects with different experiences (for example, in Java and in .NET) got together in a room, each one would have his/her own strong preferences. The architecture work can then get into "analysis paralysis". To avoid this, capture all alternatives, but evaluate them unbiasedly.
Simplify: Simple solutions are the most elegant and, surprisingly, the most effective. Don't use the pattern just because it's cool; don't add a feature if it is not requested; don't use a technology that is not designed for the problem. Use Occam's razor as it removes the burden of proof.
There are several major sources of input that an architect must consider before making a decision. Four of them are crucial to the solution delivery: organizational direction, functional requirements, nonfunctional requirements, and derived requirements.
Software development teams often forget that any solution they build is required only because of the business needs. They don't necessarily understand (and unfortunately, often don't want to understand) the details of these needs. Business people also usually don't want to learn the technicalities of the software solution. Nothing is wrong with that. However, since technical and business people speak different languages, there should be a role of a translator between the two groups. And this is, not surprisingly, the role of the solutions architect.
Every solution starts with a challenge. A business creates these challenges—this is the nature of business, this is its driving force, this is the imminent requirement for the business to survive. The solutions are typically executed as projects, with a start date and an end date, which is limited in time. However, most businesses also do not exist as short temporary activities; they plan their existence and strategies for a long period of time.
Business strategies and long-term plans provide the context for time-framed solutions that have to be delivered in order to solve a specific problem or a set of problems. For organizations with mature IT departments, Enterprise Architecture (EA) frameworks help architects manage this context. Usually, the organizational considerations are outlined in the EA policies and principles.
The next input for the decision-making process is functional requirements. Functional requirements describe the intended behavior of the system. Functional requirements typically come from the business and there are many methods for requirement solicitation, from questionnaires and surveys, to workshops and stakeholder interviews. The requirements can originate in the marketing department or come from existing end users of the product. They can describe the feature baseline necessary for the product to survive competition or can be "nice to haves" produced by the dreamer/owner of the business. The process of gathering, validating, and prioritizing requirements might be quite long and can end up with different artifacts.
When building a solution, architects should pay attention to the priorities assigned to the requirements. Usually, it is impossible to satisfy them all in the first releases of the solution, and the choice of technologies should be flexible enough to extend the solution in the future.
One of the most convenient ways to capture functional requirements is to build use cases. Use cases define a focused specification of interaction between actors and the system. Actors can be end users, roles, or other systems. Usually, use cases are written in a language that is relevant to the domain, and they can be easily understood by non-technical people. A common way to summarize use cases in a structured way is using the UML notation.
Use cases are also used in the validation of proposed architectures. By applying the proposed architecture to the use cases, architects can identify the gaps in the future solution.
Functional requirements analysis should be aligned with the design of major architectural blocks of the solution. Each requirement must be implemented in one of the solution components. Breaking down functional requirements across components or tiers provides us with a good way to validate the proposed solution architecture.
Nonfunctional requirements (NFRs) are often ignored, maybe not completely, but to a significant degree. However, they are as important to the architecture as functional requirements. Moreover, some architects argue that NFRs play a more significant role in the architecture than their functional counterpart. Wikipedia even suggests the following:
"The plan for implementing functional requirements is detailed in the system design. The plan for implementing non-functional requirements is detailed in the system architecture."
We may argue this statement, but NFRs, without a doubt, touch very deep areas of the technology.
There are many different categories of nonfunctional requirements. There is no exact list of these categories; different sources would give you different names, but the major ones would be the following:
Availability
Performance
Reliability
Recoverability
Capacity
Security
Interoperability
Maintainability
Auditability
Usability
Scalability
Expandability
When we discuss the criteria for choosing technologies later in this book, we shall pay very close attention to the NFRs. They will become the major criteria for coming up with a proper solution design.
To summarize the difference between functional and nonfunctional requirements, one can say that functional requirements answer the "what?" questions, and nonfunctional requirements answer the "how?" questions.
Working on the solution architecture, architects might come up with a requirement that was not explicitly stated either as a functional or as a nonfunctional requirement. Architects derive these requirements from initial inputs. The derived requirements have to be validated with the stakeholders and added to the set of functional or nonfunctional requirements.
For example, a functional requirement might state that the system must have real-time monitoring capabilities with an ability to inform the administrator about reaching certain configurable thresholds. To conform to this requirement, a couple more requirements should be added, which are as follows:
The system must be integrated with a communication channel: e-mail, SMS, or a similar channel
The system must have a mechanism (XML files and a database with a UI) to change the configuration
Requirements could be simply forgotten during the requirement-gathering process. For example, a Publish/Subscribe system should have a way to manage subscriptions and subscribers, which sometimes become an epiphany later during the design process.
Gathering requirements is an iterative process. Once the architects start working with the requirements, more requirements can be derived. They should be given back to the business stakeholders for validation. The more complete set of requirements the designers get, the less expensive the system development will be. It is well known that a requirement implemented at the end of the solution development costs much more than if it was suggested at the beginning of the process.
Once the core requirements are set forth, architects can start working on the building blocks for the solution. The building blocks are like high-level patterns; they specify what major components the system might have. For example, for a middle-tier monitoring solution, building blocks might consist of a message-logging system, a reporting system, a notification system, a dashboard, a data maintenance system, and others. The next step should be to look into a lower level of the architecture; each building block requires patterns to be used. Message logging can be done with the usage of the filesystem, a database, SNMP sending log data into another system, or something else. Before the patterns are selected, evaluated, and thoroughly considered, jumping into a product selection would be a grave mistake. It could cause selecting a tool that might not be fit for the task, a tool that is an overkill, or a tool that requires an enormous amount of configuration effort. Sometimes, building a proof of concept might be required to evaluate the patterns implemented by a technology candidate.
There are many books that have been written on generic patterns, especially patterns of the enterprise application architecture. The most respected series is the series with the Martin Fowler signature. We would recommend Patterns of Enterprise Application Architecture, Addison-Wesley Professional, Martin Fowler, and Enterprise Integration Patterns, Addison-Wesley Professional, Gregor Hohpe, Bobby Woolf, as the most relevant to discussions in our book.
In the process of evaluating technologies, we will build criteria in the following four dimensions:
Organizational context: Solutions built to function in an organization should be aligned with business needs and directions. Organizational context is usually provided by the enterprise architecture that builds general IT principles and strategies for the organization.
Solution design: These criteria are relevant to the process of designing the system. The design is typically the step that starts after the core architecture is completed. In the Agile development, the design starts sooner, but the architecture keeps the backlog of unfinished business (the so-called architectural debt) that is being worked on over time.
Solution implementation (development, testing, and deployment): These criteria focus on the next stages of solution delivery from the completed design to the deployment in production. Product development might not have a production deployment stage per se; rather, it would have a need to create installation programs and packaging.
Operations: Surprisingly, this is the area that is neglected the most while the architecture is developed. This is because it is all about the business value that the solution is supposed to provide and was built for. A very typical example is giving low priority to buying (or developing) administration tools. We have seen organizations that buy sophisticated and very expensive monitoring tools but don't provide proper training to their staff, and the tools end up simply not being used. As the most ridiculous example, I remember an organization providing SaaS services that allowed intruders to use a back door to their FTP server for eight months simply because they did not use proper monitoring tools.
Organizational context provides us with a big picture. Every organization has its set of principles, implicit or explicit, and the task of the solutions architect is to build systems aligned with these principles. The following table lists some major principles that are typically developed by the organization enterprise architecture team:
In this section, we look at the characteristics relevant to the overarching design of a solution. The list is certainly not exhaustive, but it provides a good basis for building a second dimension of the framework.
Should design, coding, and other standards be automatically enforced through tooling, or is this a more manual process? Should the source control system be centralized and integrated in a continuous integration model? Should the programming languages be enforced by an organizational policy or be chosen by developers? All these questions belong to the realm of solution delivery. If architects select a technology that cannot be delivered on time or with given skillsets, the entire solution will suffer.
Solution delivery also very much depends on the project management approach. In a modern Agile world, delivery technologies should be chosen to allow for rapid changes, quick prototyping, quick integration of different components, efficient unit testing, and bug fixing. Agile projects are not easier or cheaper than Waterfall projects. In fact, they guarantee rapid and quality delivery but at a cost. For example, it is well known that Agile projects need more skilled (and therefore, more expensive) developers. Some estimate the number of required senior developers is up to 50 percent of the team.
The following table presents some considerations that affect the technology selection:
Even after we have satisfied our design and implementation needs, we absolutely must consider the operational aspects of the proposed solution. Although the project delivery team inevitably moves on to other work after a successful deployment, the actual solution might remain in a production state for years. If we have a grand architecture that is constructed cleanly but is an absolute nightmare to maintain, then we should consider the project failed. There are many examples of solutions like this. Consider, for instance, a system that provides sophisticated calculations, requires high-end computers for this purpose, but has a small number of servers. If an architect suggests that the organization should utilize Microsoft System Center for monitoring, it would create a nightmare for the operations team. The System Center is a very large tool, even formal training for the team would take a week or two, and the learning curve would be very steep. And at the end of the day, maybe only 5 percent of the System Center capabilities will be utilized.
Operational concerns directly affect the solution design. These factors, often gathered through nonfunctional requirements, have a noticeable effect on the architecture of the entire system.
So what do we do with all this information? In each of the "pattern chapters" of this book, you will find us using this framework to evaluate the use case at hand and proposing viable candidate architectures. We will have multiple candidate architectures for each use case and, based on which underlying product is the best fit, go down the path of explaining this specific solution.
So, how do we determine the best fit? As we evaluate each candidate architecture, we'll be considering the preceding questions and determining whether the product that underlies our solution meets the majority of the criteria for the use case. Using the next representation, we'll grade each candidate architecture in the four technology evaluation dimensions. The architecture that is the most compatible with the use case objectives will win.
In the next chapters, we will use the icons presented in the following table to indicate the overall evaluation of the technologies:
A common methodology to evaluate solution requirements against product capabilities will go a long way towards producing consistent, reliable results. Instead of being biased towards one product for every solution, or simply being unaware of a better match in another software offering, we can select the best software depending on its key capabilities for our client's solution.
In the next set of chapters, we'll introduce you to these core Microsoft application platform technologies and give you a taste as to what they are good at. While these primers are no more than cursory introductions to the products, they should give you the background necessary to understand their ideal usage scenarios, strengths, and weaknesses.
Later in this book, when we discuss different Microsoft applications and technologies, we shall build a taxonomy of Microsoft products, which will help architects navigate in the ocean of software tools.
