What is governance? Why is it so critical to the success of an SOA adoption effort. This chapter will introduce you to the concept of governance, using the familiar concept of municipal government, introduce its core components of people, policies, and processes, and then illustrate why these are important to the adoption of SOA within an enterprise.
When you hear the word "governance", what comes to mind? For most people in information technology, it is not a positive image. If you are a typical corporate developer, you are probably envisioning forms to fill out, presentations to prepare, meetings in front of a review board, and more than likely, annoyance in having to take time away from writing code for people that, in your opinion, really don't understand what you're trying to do in the first place. Often, a developer may see the review as nothing more than an opportunity for the reviewers to flaunt their authority. They simply take their lumps in the review, and then everybody goes back to doing what they were doing with no real change in behavior, other than some additional animosity in the organization.
If you are an enterprise architect, you may be on the other side of this equation. You are the one listening to presentations from project teams, trying to provide guidance to ensure that the efforts go beyond the needs of the individual project, but only encountering developers who are more interested in finding opportunities to try the newest technologies than what is needed to meet the needs of the enterprise. Even if the developers are able to be convinced, the required changes then get shot down by a project manager or sponsor who won't accept the resulting change in schedule.
If you are a manager, especially a senior manager, you may have a completely different take on governance. Rather than being about the efforts going on inside a project, it's about getting projects approved. Many organizations even have a committee called the IT Governance Committee, whose job is to review project proposals and determine which efforts will be funded. While there normally isn't as much pain associated with this effort, there's still potential for animosity when managers don't understand the prioritization process used by the committee.
So why do we do it? The fact is that governance is a required and critical part of any organization. It is the combination of people, policies, and processes that are put in place to ensure the organization achieves one or more desired behaviors. When used properly, it can be the difference between success and failure.
The adoption of service-oriented architecture, or SOA, has been touted as an approach that can change the way IT operates, increasing the agility of the organization and achieving a greater degree of alignment between IT and the rest of the business. An effort of this nature represents a fundamental change in the way an organization leverages information technology. It is up to governance to guide the organization through this change.
To better understand governance, let's first look at it from a different context, one that we all deal with on a daily basis, which is municipal government.
The city you live in is a living organization, trying to meet the needs of its constituents and businesses alike. Nearly all cities have a desired behavior of being a safe place where people want to raise their children and businesses want to operate. Cities will likely vary, however, in their approach to growth. At one end, an established city may be landlocked and may have to focus on remaining attractive to both young and old residents, keeping the population base stable. At the other end, areas near urban centers with plenty of open space may be experiencing rapid growth as young professionals seek larger lots with plenty of space for kids to play. In the middle, rural communities may be looking for slow, controlled growth to preserve their rural heritage yet remain attractive to young families.
Regardless of where you live, you are likely to be subjected to many forms of government. Your city or village may have a mayor and a city council. The churches may have a pastor and an associated council of leaders. Your city or village may be part of a regional government, such as a state or province with a governor or other form of provincial leadership. That regional government is likely to be subjected to the oversight of the country's government, which can include a president or prime minister, along with parliament, congress, or some other body of representatives. In addition to these roles, one cannot forget the police force. All of these examples have one thing in common: people who are recognized as authority figures, typically in either a position of establishing, or enforcing, policy.
It should be known, however, that authority does not necessarily imply a dictatorship. In many governments, it is the people that grant the authority figures their powers through the election process, and the people typically have the power to remove those figures from authority. While the typical corporation is not a democracy, there are many lessons to be learned from a democratic style of government. One must not forget that the motto of many police organizations is to serve and protect, while legislators are representatives of the people. The correct message is that governance is a responsibility of everyone, whether formally assigned or not. The degree to which the governed participate in the governance process can have a huge impact on the success or failure of the governance effort.
Simply having people is not enough. While the people may all agree on where they want to go, it is the policies associated with the day-to-day activities of the community that make it happen. The community must look at its desired behavior and determine the right set of policies that will achieve that behavior. For example, does the community want to be a bedroom community, or does it want to be a retail hub for the region? Does it want to focus on attracting medium to large organizations with many employees, or will it focus on smaller businesses? Will the community stay small, or will it be on a path of continued growth, adding property, businesses, and residents over time? Will the community allow a variety of residents and businesses, ranging from low income housing to million dollar mansions and from the local hardware store to a major international company? What kind of education will the community provide for its residents?
In order to ensure that the community realizes the desired behavior, its actions must be guided through policy. These policies will cover a range of things that are required for the community to stay healthy and grow. It involves many different aspects, including the speed limits on city streets, tax rates for residents and businesses, and zoning regulations that guide the types of businesses allowed. There are also polices that influence the activities that take place within the city, such as specifying that a specific percentage of revenue must go towards education versus other needs. It is likely that an IT Governance committee has similar policies that are used in determining which projects get funded.
As the community grows and the policies grow more and more numerous, it will become clear that having people and policies alone are still inadequate for effective governance. While many people will adhere to policies, not everyone will. For some, it may be due to a deliberate action, for others, it may simply be due to lack of awareness. In order to combat this, processes must be put in place to ensure that the community is aware of the policies that have been created by the leaders, as well as processes that ensure that the community is following those policies.
Take, for example, speed limits. In its earliest phases a community may not have had any speed limits on its roads. Over time, as the community grows, a continued increase in the number of automobile accidents may cause the leaders to establish a speed limit on city roads: a policy. However, simply passing this law during a city council meeting is unlikely to change behavior. The first thing the leaders must do is educate the community on the new policy, and they do so by placing speed limit signs on the roads in question. In addition, a driver's education course is created and all new drivers, or drivers that are renewing their licenses, are required to complete it successfully before receiving their new or renewed license. These processes will certainly increase the adherence to the policy, but just as many drivers on the road today ignore speed limit signs and so it may not achieve the levels desired by the leader. To achieve the desired behavior, the city council decides that a police force is necessary to enforce the policies. Through the use of radar guns the police are able to detect when automobiles are out of compliance with the stated policies, and can institute appropriate punishment in the form of warnings, fines, or other loss of privileges.
Processes are frequently the difference between good governance and poor governance. All too often, the negative view of governance is a result of an over-emphasis on policy enforcement. This can frequently result in a command-and-control culture, which can create animosity in an organization. Perhaps, even more important than enforcement processes are communication and education processes. By educating the residents and businesses on the policies first, it is far easier to achieve compliance. Likewise, the authorities must have an open ear, and listen to where policies are actually counter-productive to the goals of the community. Finally, just as the people and businesses are held accountable for adherence to the policies, the authorities must be held accountable for their actions, with the people having the ability to remove leaders that are not acting in the best interest of the constituents or if the desired behaviors are not being achieved.
It is important to realize that no two governments are alike. In communities where the residents have a high degree of trust in the leaders, and agreement on the direction and policies, the community may not need as many enforcement processes as the residents naturally adhere to the policies as it is in their best interest. In communities where the residents do not trust the leaders of the organization, due to corruption or other factors, policies may not be followed, and as a result, the community may have to invest far more heavily in education and more likely, enforcement through the police force.
These aspects are the essence of governance: desired behavior, people, policies, and process. The desired behavior is achieved through a successful combination of people, policies, and processes. People are the leaders that are responsible for establishing the desired behavior of the organization, policies are the rules that express the desired behavior, and process ensures that the policies are followed. Just as no two governments will operate in exactly the same manner, with the same structure, the same holds true for information technology organizations. They will each have their own leadership structure, desired behavior, policies, and processes. If the desired behavior is being achieved, the governance is successful.
While it easy to put governance into the context of municipal or regional governments, it is not limited to this domain. The Sarbanes-Oxley Act increased awareness of the term corporate governance. A key aspect of Sarbanes-Oxley was to ensure that the corporate boards (the people responsible for governance) of publicly-traded companies in the United States take individual responsibility for the accuracy and completeness of financial reports. In addition, there were new standards established for compliance audits of these companies. In order to be compliant, companies had to introduce new policies associated with a variety of corporate activities. On top of that, it was certainly in the company's best interest to perform their own audits and ensure compliance with these policies through internal processes prior to the official audits by an independent auditor. While Sarbanes-Oxley may not touch on all aspects of corporate governance, it certainly serves to demonstrate how people, policies, and processes are an inherent part.
In the case of Sarbanes-Oxley, the primary concern is governing the financial accounting practices, with the desired behavior being articulated as part of it. Another part of corporate governance, however, is the desired behavior of the use of information technology, which is known as IT Governance. Remaining consistent with the earlier definition of governance, IT Governance is defined as the people, policies, and processes that an organization leverages to ensure the appropriate behaviors and outcomes in respect to the organization's utilization of information technology. In many organizations, the face of IT Governance is the review board (people) that make decisions on which efforts receive funding, and which do not. However, IT governance does not end there. Many organizations also have Portfolio Management Organizations, or PMOs, that ensure that the efforts, once funded, are properly prioritized, staffed, and executed in a consistent and appropriate manner. The PMOs must establish policies that define what consistent and appropriate means, and then ensure that the projects are compliant with those policies.
Before we delve into governance within the context of SOA, we first need to define what SOA is. The first step in this is to define what we mean by service. One of the many definitions provided by the Merriam-Webster dictionary (http://www.merriam-webster.com/) for service is a facility supplying some public demand. The key parts of this definition are facility which means that some capability or function is performed, supplying which means that the function is provided to consumers, and public demand which means it's something that one or more consumers actually want. A SOA, therefore, is quite simply, an architecture that utilizes the core concepts of service providers and service consumers to define a system.
Building on our example of a municipality, the community may initially have started as a collection of homes, each with their own well for water, garden for food, and so on. Over time, however, the residents realized the need for some common services. It may have begun with residents each contributing property for a common road that connects their houses. In other areas, it was likely focused on the economies of scale, such as a public school system, a shared source of water, sanitation services, and as technology evolved, communications and media services. As these services evolved, the impact on individual residents varied widely. Some residents had designed their homes in such a way that a transition from their private well to a public water source was an inexpensive effort. Other residents, however, had far greater expenses in adapting their internal plumbing to the fixtures required by the public source. The municipality can be viewed as a collection of these services, with the municipality acting as the provider of the services and its residents as the consumer of the services.
While this definition may seem simple, it captures the essence of what SOA is all about: breaking down a system into a collection of consumers and providers. The key to a successful SOA, however, is ensuring that the right services are provided and that the relationships between consumers and providers are formally established and managed. A city that has a complicated maze of pothole-laden roads, unreliable electricity, poor schools, high taxes, along with a city council that was appointed for life is not going to be a pleasant place to live. Are they providing services? Yes. Are they providing them well? No. Is the relationship between the constituents and city healthy, given that the council members are assured a paycheck for life, regardless of whether any improvements are made? Probably not.
If we compare this to the typical corporate IT department, individual applications are similar to the homesteads provided in a new community. Many of these applications are currently implementing capabilities in their own, private manner, even though there are many applications within the enterprise that implement the same capability. Some of these capabilities will be pure infrastructure, such as security and logging, but others will be business capabilities such as customer management and order processing.
Just as some of our homeowners had a higher cost associated with utilizing the public services, the same thing holds true in the world of corporate information technology. Many applications are hampered by an inflexible design such that the cost of change is now prohibitive. This shouldn't be considered a result of poor decisions taken years ago, but rather the normal course of growth. It is unlikely that all homeowners could have anticipated the changes that would happen over the years, and equally unlikely, if not more, that application designers could anticipate the technology advances that have occurred over the last twenty years.
One key difference between the typical corporate enterprise and typical community, however, is that all things in the enterprise exist for the good of the enterprise, and not as independent entities. When an individual homeowner chose to build in an inflexible manner, the only one impacted by this inflexibility was the homeowner. The community, as a whole, is likely not impacted by this. For the corporate enterprise, however, an inflexible application is another story. As long as that application is still necessary for the enterprise, the cost associated with that inflexibility will grow larger and larger. Just as a community can bulldoze a dilapidated property, an enterprise can choose to scrap an application and rewrite, but that comes at a large expense.
In order to prevent the continued cycle of inflexibility, an enterprise must move away from today's state where the information technology assets are largely viewed as a collection of individual applications and their data to a state where the assets are viewed as a collection of capabilities provided as services. This is a very important distinction, because many enterprises have simply taken existing applications, rewritten sections of them as services, and think that they're adopting SOA. When it comes down to it, however, they still have the same applications, and those applications still have the same integration challenges. For example, the typical enterprise has a collection of applications as shown in the following figure:
When the need arose for these applications to communicate, the generally accepted approach was to create an adapter that acts as the glue that connects the two applications. For each new pair of applications that need to be integrated, a new adapter would be created, adding more and more complexity over time.
To get out of this endless cycle of adding more and more adapters in the middle, which adds complexity, the enterprise needs to move away from application oriented architecture. Application oriented architecture is where the core unit used to describe the enterprise is an application. It therefore follows that SOA, simply stated, is an architecture whose core unit of composition is a service. If we take the diagrams above and eliminate the boundaries of the application, we get a picture that looks like the following:
When these boundaries are eliminated, the enterprise can now be viewed as a collection of service consumers and service providers that are expected to operate as a community. This is instead of being viewed as a collection of individual applications that have no clear indication of where capabilities are shared, and inconsistent internal structures that do not support future change or integration needs. User interface components and all business logic services are built in a consistent, composable manner, and all data resources are exposed in a consistent, composable manner as shown in the following figure:
This approach doesn't prevent individual services from being highly customized for a particular need. What it does do, however, is to ensure that we still build for agility. If the end result is that a particular business service only has one consumer, that's still okay.
Adopting SOA and moving away from application oriented architecture will allow information technology to lead the enterprise to progress into the future, rather than being perceived as the anchor holding the enterprise back.
Given the understanding we now have of governance in general, and of service oriented architecture and the desired behaviors it intends to achieve, what is SOA governance and why is it important? SOA governance is the combination of people, policies, and processes within your organization that will ensure that the desired behaviors of your strategic SOA initiative are achieved.
It includes the traditional areas associated with IT Governance, which is the selection and funding of IT projects. These projects define the initial scope for technology utilization and can either help or hinder the SOA effort, based upon the scope chosen.
The SOA effort only gets executed through projects, and if the execution is poor, the SOA effort will be poor. Therefore, the project governance activities of an organization must be adjusted to include policies associated with achieving the desired behaviors associated with SOA adoption.
However, it doesn't stop there; the behavior of the IT solutions and the teams that support them may also require changes. The redefinition of boundaries associated with technology solutions can result in new operational activities and a greater need to respond quickly to changes without having a ripple effect through the systems. Run-time governance must be leveraged to ensure the systems and the organizations supporting them operate as efficiently as possible.
The people involved with SOA governance will mainly include the same people involved with your current IT and Project governance efforts. New artifacts may be required to help establish appropriate policies for SOA success. If the existing governance efforts are poor or unsuccessful, it may be necessary to bring in new people or a new approach to your governance efforts. Individuals involved will include senior leadership from both within and outside of IT, enterprise architects, technology architects, development managers, business architects/senior business analysts, and more. The SOA effort cannot be viewed as just an IT operation. While the IT department may oversee a significant portion of the SOA efforts, if those efforts are not in alignment with the desired behavior of the organization as a whole, conflict will arise. This would also be the case if a city was out of alignment with the state or province, or if the state or province was out of alignment with the federal government.
The constituents of the organization are both the IT staff involved in the construction of services and their consumers, the business users that utilize the IT solutions, and in many cases, an organization's partners that may interact via services.
The policies that the people involved with the SOA governance effort create must guide the organization towards the desired behavior of the SOA effort. If the effort is intended to reduce the costs associated with the typical IT development effort, what are the policies that will result in that outcome? If the effort is intended to reduce the time required to make system changes in response to regulatory changes that occur on a regular basis, what are the policies that will result in the outcome? What are the policies that will ensure that the SOA adoption effort won't have a negative effect on the organization by the resulting increase in moving parts putting new strains on inefficient operational processes? If the organization simply constructs services without any change in their behaviors, it is unlikely that the desired outcome will be achieved, and the organization will be at risk of having SOA be seen as yet another overhyped IT effort that promised to save the world and failed.
Finally, SOA governance must involve processes for policy creation, communication and education, enforcement, and measurement. It must focus on communication and collaboration first, so it is not seen as a heavy-handed dictatorship. If it is understood and embraced, enforcement will be a simpler task because the staff will want to be compliant, rather than being forced to be compliant. Finally, the effort must be continually measured so that the leaders themselves are held accountable, and changes can be enacted if the desired behaviors are not being achieved.
But why is all of this needed? Aren't we already choosing what applications to build, funding those projects, and designing and deploying those solutions? Before you embark on your SOA journey, there is one critical question that must be asked. Does your organization need to change? If the answer is no, then there's no reason to keep reading. If the answer is yes, based on the fact that you're reading this book, which is probably the case, then you need SOA governance. Simply viewing SOA as a technology-based solution where you choose to use XML, SOAP, and HTTP technologies in place of Enterprise Java Beans, DCOM, CORBA, or any other distributed computing technology is not a change in behavior. Your organization will still be building the same solutions it always has, just with a different set of technologies.
The real challenge behind SOA is not choosing technologies appropriately; it is changing the behavior of the organization so that it can improve. Governance is all about guiding behavior of an organization, so it is understandable that the key to being successful with SOA is governance.
Without SOA governance, the information technology assets of an enterprise are, at best, a collection of independent solutions that are primarily related through proximity. Integration is seen as something to be avoided as much as possible, and where required, is done so as a necessary evil, frequently in a suboptimal manner. With SOA governance, the information technology assets of an enterprise are a collection of interoperating solutions that through effective people, policies, and processes, collectively meet the needs today and of the future for the business organization as a whole.
In this chapter, we've learned that governance is necessary to ensure that the organization achieves its desired behaviors, whether that organization is a city, a country, a business, or an IT department. This is done through a combination of three things: people, policies, and processes. People are the leaders that are responsible for establishing the desired behavior of the organization, policies are the rules that express the desired behavior, and process ensures that the policies are followed.
We learned that SOA is a new approach where the information technology assets are viewed as a collection of services and consumers. These services and their consumers are expected to operate as a community from day one, rather than being viewed as a collection of individual applications that lack clarity on where capabilities are shared, and have inconsistent internal structures that do not support future change or integration needs.
This shift from application oriented architecture to SOA is a fundamental change to the way that IT operates and likely a change in the way that the business interacts with IT. A change of this scope represents new behaviors, and the way to achieve these desired behaviors is governance.
In the next chapter you will be introduced to the fictional company and its employees that will be used throughout this book and learn about their initial efforts in adopting SOA, and how governance, or the lack thereof, played a role in their successes and their failures.