SOA Cookbook

By Michael Havey
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. SOA Survival Guide

About this book

SOA Cookbook covers process-oriented SOA. BPEL is the best-known language in this area, and this book presents numerous BPEL examples. It also studies proprietary vendor process languages such as TIBCO's BusinessWorks and BEA's Weblogic Integration. If you are building SOA processes in the field, chances are you are using one of the languages discussed in SOA Cookbook. The book assumes that the reader is comfortable with XML and web services.

Author Michael Havey works with SOA in the field for TIBCO (and previously for IBM, BEA, and Chordiant). SOA Cookbook is Michael's second book. Essential Business Process Modeling, his first book, was published in 2005.

Publication date:
September 2008
Publisher
Packt
Pages
268
ISBN
9781847195487

 

Chapter 1. SOA Survival Guide

There's something decidedly off-putting in the language of SOA. The name Service-Oriented Architecture, we must admit, is a trifle pompous. It's a general-sounding name. Whoever coined the term must also have contemplated writing the book On the Nature of Reality, or selling the instructional DVD A General Solution to Problems. Whatever a service is, it will help us emerge from our current monoliths, stovepipes, and silos, so goes the hype. At its brashest, SOA dismisses OO, MOM, EAI, and distributed objects as ineffectual forerunners that never quite understood the problem at hand. With services, we awaken to the 'glorious mental dawn' (to quote Hegel) that will transform our enterprise.

When presented this way, SOA is such a cheap sales pitch that we're tempted to call it snake oil architecture. We are genuinely amused to hear the pejorative usage same-old architecture. Indeed, it's not difficult to defend the position that SOA and EAI refer to the same concept. Both terms smell like snake oil, though the concept to which they refer is a reasonable, useful concept. (We discuss why presently.) Perhaps the best name is the one that puts the salesman on the defensive. If we call it So?, we're demanding proof of the value and originality of the approach. If the salesman convinces us and we sign off on it, we might consider renaming it Go!

Sadly, most advocates of SOA, when asked to drill-down technically into the method, present it as a kind of simple object access without the protocol (SOAP without the P). Forget for the moment whether it's MOM or web services, whether it's XML or flat file, whether it's Java or .NET. The whole point of services is to look at your business, decide what functions it should offer, and make those functions accessible through technology channels. A bank, for instance, needs deposit(), withdraw(), and transferFunds() functions. Well, the bank already has programs that perform these functions, but they're buried in the mainframe. Every time the bank builds a new customer channel, it reinvents the mainframe access method. The value of services is to build these functions once, for all channels, such as in the following web service:

BankService {
                Deposit() {
                Call mainframe
                }
                Withdraw() {
                Call mainframe
                }
                TransferFunds() {
                Call mainframe
                }
                }
            

Wrap the mainframe in something a little newer and prettier like SOAP over HTTP and you're onto the something! It could also be SOAP over JMS, or some other contemporary mechanism. What matters is that the bank finally has the service it always needed.

When this sort of service is taught in an SOA training lab, the mainframe is replaced with a lightweight classroom database, which might even run directly on the student's desktop or laptop. The bank service is reduced to the following:

BankService {
                Deposit() {
                Call laptop database
                }
                Withdraw() {
                Call laptop database
                }
                TransferFunds() {
                Call laptop database
                }
                }
            

Simple object access indeed! For years, SOA has been pitched as an elaborate way to wrap functionality that is already there. Services are atomic and stateless, according to this view. If there is any complexity in the implementation, it is in legacy connectivity, transactions, security, or data transformation. Organizations that have learned the hard way how gruelling application development projects are can rest easy. Services are a quick win.

Let's get organized! If services are so quick and easy, how do they deliver value? The answer from SOA experts is that if an organization has never before published critical business functions as a WSDL interface, it has never systematically organized its thoughts on what those functions are. Or, it might have put down this structure on paper, but never successfully built the technology interface. Many organizations are in this predicament, and for them SOA is an easy sell. There are three parts to any SOA engagement:

  1. 1. Meet with the business to decide what services are required.

  2. 2. Install the chosen vendor's SOA platform.

  3. 3. Develop the services on this platform.

In step 1, high-priced consultants jot down the required services as a bullet list on a white board. In step 3, headstrong developers anticipate that building these services means a little XML, a little WSDL, a little mainframe connectivity, and little else. But the real 'glorious mental dawn' occurs when everyone discovers the capabilities of the vendor platform installed in step 2. SOA Cookbook awakens to this dawn!

The flow's the thing. Customers who buy SOA platforms from major vendors—such as IBM, BEA, TIBCO, and Oracle—know that building SOA services means constructing not wrappers but processes. In these tools, as we discover below, services are boxes and arrows on a canvas. The boxes themselves are not difficult to master, but assembling them in a control flow can be considerably more challenging. Services that consultants jotted down as bullets on a whiteboard evolve into complex life forms during development. The consultant should have sketched processes instead, because that is what services really are. The BankService code snippet presented above is textbook material only. Mastering SOA means mastering processes. With apologies to Shakespeare, the flow's the thing wherein we'll catch the conscience of SOA.

SOA from 184 Billion Feet

If you wanted to learn the latest on distributed enterprise architecture in the late 1990's, the Martian book was the book to read. Officially named Client/Server Survival Guide (R. Orfali, D. Harkey, J. Edwards, Client/Server Survival Guide, Third Edition. Wiley, New York, 1999), the 700-page book by Robert Orfali, Dan Harkey, and Jeri Edwards was known for its astonishing breadth, its advocacy of CORBA, and its predilection for Martian cartoons. (The best of these cartoons appears on the cover, as shown in the following figure.):

Regarding breadth, the Martian book covers, frankly, everything one needs to know about enterprise technology. The implication is that a good enterprise architect is a sort of universal man who understands, and can speak convincingly about, all things technical: networks, desktop operating systems, server operating systems, network operating systems, security, the nature of user interfaces, transactional databases, object-oriented databases, data warehouses, SQL, transactions, TP monitors, email, workflow, distributed objects, the web, XML, system management, and numerous other topics. This knowledge is not an end in itself but necessary background for building a quality enterprise application. The same is true of SOA today. If we've been sold on the idea that SOA is just point-and-click, we should give vindication to our Martian visitors: building an SOA requires a holistic vision of the overall technology environment.

The Martians obviously lacked prescience, though, in betting on CORBA, and no doubt they've retreated to their planet since the demise of that technology. In Client/Server Programming with Java and CORBA (R. Orfali, D. Harkey, Second Edition. Wiley, New York, 1998), Orfali et al also manage to work Java applets (now long forgotten) into the architecture, resulting in the three-tier arrangement shown in the next figure. The emphasis at the time was on objects. Applets made it possible to develop web-enabled object-oriented user interfaces in Tier 1, a tier that others in the industry called 'Presentation' but for the Martians was 'View Objects'. An applet could rather easily be coded as a CORBA client, using standard Java CORBA classes to converse with a CORBA object over IIOP (Internet Inter-Orb Protocol, a way of calling CORBA over HTTP), and thus could readily provide a view of that object. CORBA objects themselves, which were deployed to an ORB ( Object Request Broker, a kind of object bus) in Tier 2, exposed an object interface of attributes and methods. A CORBA object was uniquely identified by a name and had state; in the same ORB might reside numerous objects having the same interface, distinguishable by name. Objects are integrated with the legacy systems of Tier 3 to implement their interface.

In SOA, the three-tier model does not suffice, because it does not depict communications between services, which are just as important as client communications with services. Other differences are that clients seldom use applets nowadays, the ORB is now called an ESB, and significantly, the components on the bus are not stateful objects but stateless services. Or are they stateless? In the next section, we argue that in practice services are invariably developed as processes, which are stateful and can span multiple partner invocations. This revelation, we argue, is SOA's best-kept secret.

The Martians, for their part, did not pick up on the importance of processes. The samples provided in Client/Server Programming with Java and CORBA implement CORBA object methods in Java as one or two lines of code, much like the bank service wrapper presented above. In the survey of technology in Client/Server Survival Guide, workflow is covered with reasonable thoroughness, but it is pigeonholed as groupware, alongside email, conferencing, and imaging. In the late 1990's, this was an appropriate classification, but the Martians ought to have gazed further into the light cone, and foreseen the emergence of BPM and SOA process architectures.

 

SOA from 184 Billion Feet


If you wanted to learn the latest on distributed enterprise architecture in the late 1990's, the Martian book was the book to read. Officially named Client/Server Survival Guide (R. Orfali, D. Harkey, J. Edwards, Client/Server Survival Guide, Third Edition. Wiley, New York, 1999), the 700-page book by Robert Orfali, Dan Harkey, and Jeri Edwards was known for its astonishing breadth, its advocacy of CORBA, and its predilection for Martian cartoons. (The best of these cartoons appears on the cover, as shown in the following figure.):

Regarding breadth, the Martian book covers, frankly, everything one needs to know about enterprise technology. The implication is that a good enterprise architect is a sort of universal man who understands, and can speak convincingly about, all things technical: networks, desktop operating systems, server operating systems, network operating systems, security, the nature of user interfaces, transactional databases, object-oriented databases, data warehouses, SQL, transactions, TP monitors, email, workflow, distributed objects, the web, XML, system management, and numerous other topics. This knowledge is not an end in itself but necessary background for building a quality enterprise application. The same is true of SOA today. If we've been sold on the idea that SOA is just point-and-click, we should give vindication to our Martian visitors: building an SOA requires a holistic vision of the overall technology environment.

The Martians obviously lacked prescience, though, in betting on CORBA, and no doubt they've retreated to their planet since the demise of that technology. In Client/Server Programming with Java and CORBA (R. Orfali, D. Harkey, Second Edition. Wiley, New York, 1998), Orfali et al also manage to work Java applets (now long forgotten) into the architecture, resulting in the three-tier arrangement shown in the next figure. The emphasis at the time was on objects. Applets made it possible to develop web-enabled object-oriented user interfaces in Tier 1, a tier that others in the industry called 'Presentation' but for the Martians was 'View Objects'. An applet could rather easily be coded as a CORBA client, using standard Java CORBA classes to converse with a CORBA object over IIOP (Internet Inter-Orb Protocol, a way of calling CORBA over HTTP), and thus could readily provide a view of that object. CORBA objects themselves, which were deployed to an ORB ( Object Request Broker, a kind of object bus) in Tier 2, exposed an object interface of attributes and methods. A CORBA object was uniquely identified by a name and had state; in the same ORB might reside numerous objects having the same interface, distinguishable by name. Objects are integrated with the legacy systems of Tier 3 to implement their interface.

In SOA, the three-tier model does not suffice, because it does not depict communications between services, which are just as important as client communications with services. Other differences are that clients seldom use applets nowadays, the ORB is now called an ESB, and significantly, the components on the bus are not stateful objects but stateless services. Or are they stateless? In the next section, we argue that in practice services are invariably developed as processes, which are stateful and can span multiple partner invocations. This revelation, we argue, is SOA's best-kept secret.

The Martians, for their part, did not pick up on the importance of processes. The samples provided in Client/Server Programming with Java and CORBA implement CORBA object methods in Java as one or two lines of code, much like the bank service wrapper presented above. In the survey of technology in Client/Server Survival Guide, workflow is covered with reasonable thoroughness, but it is pigeonholed as groupware, alongside email, conferencing, and imaging. In the late 1990's, this was an appropriate classification, but the Martians ought to have gazed further into the light cone, and foreseen the emergence of BPM and SOA process architectures.

 

The Model Stack


Three-tier CORBA-based architecture has evolved into the SOA architecture, depicted in next figure. The ORB has been replaced by an Enterprise Service Bus (ESB), which acts as a service broker for the organization, routing messages between participants. IIOP is a thing of the past. External clients and partner services now use SOAP over HTTP to access services through the bus; internal ones typically use MOM queues as their access mechanism. Services are orchestration processes, which run either in a single burst (accepting the client request, running through a short string of processing steps, and sending back the response) or in multiple bursts (accepting multiple requests over time, and interacting with partner services in a conversation). (Refer to Chapter 5, a comparison of short-running- and long-running processes, for a precise definition of 'burst'.)

BPM, absent in the CORBA architecture, is a vital part of the SOA stack. The contemporary use case has both human and system activities, which when developed on this architecture are split between BPM processes and SOA orchestration processes. (Chapter 3 discusses BPM/SOA 'splitting' techniques.) Finally, like CORBA objects, SOA services integrate with legacy systems.

SOA vendors that sell platforms to support this architecture divide their stack into three pieces, shown in the next figure. At the bottom of the stack is the ESB, which is often bundled with the vendor's own MOM implementation; if the vendor does not have MOM, the ESB builds in support for third-party MOMs. Next up is an engine to run orchestration processes. This piece is sometimes referred to as the process integration engine, and it typically includes a variety of controls to help integrate with the backend. These controls simplify database access, data transformation, MOM integration, and other common integration needs. Some vendors also sell adapters (for example, mainframe adapter and SAP adapter) that plug-in to the process integration engine, providing orchestration processes with simplified access to major systems. At the top of the stack is the BPM engine, which executes BPM processes. The vendor builds in a bidirectional interface between the BPM engine and the process integration engine, allowing orchestration processes to call BPM processes and vice versa. The following two pieces are the 'SOA' part of the stack.

The following table divides the four leading SOA vendor suites into BPM, process integration, and ESB parts.

Vendor

BPM

Process Integration

ESB

TIBCO

iProcess

BusinessWorks

ActiveMatrix

BEA

AquaLogic BPM

Weblogic Integration

AquaLogic Service Bus

Oracle

'Fusion' BPA Suite

'Fusion' BPEL Process Manager

'Fusion' Enterprise Service Bus

IBM

Websphere Process Server, FileNet

Websphere Process Server, Websphere Interchange Server

Websphere Enterprise Service Bus, Websphere Message Broker

Sometimes smaller niche vendors combine forces to offer this stack. An example is the Lombardi's partnership with Progress; Lombardi provides BPM with Teamworks, which sits atop Progress' Sonic ESB.

A vendor with a good MOM implementation has street cred with customers. SOA might be the flavor of the month, but the vendor must demonstrate that it can solve the age-old problem of moving events swiftly and reliably at high volumes. IBM boasts that its ESB runs on MQ/Series, and TIBCO wins many deals because its SOA solution uses workhorse MOM products—Rendezvous and the Enterprise Message Service (EMS). BEA, despite having one of the industry's leading TP monitors in its arsenal (Tuxedo), insists that it's the application server that matters most, and brags that it has the best application server. Oracle has street cred in the database realm, and tells customers that database street cred matters most for SOA.

Standards, particularly web services standards, influence the design of the stack. Service interfaces are published as WSDLs. Messages are transported as SOAP over HTTP. Message payload is XML, which can be described with XSD and manipulated with XPath, XQuery, or XSLT. Services are registered with a UDDI registry. Service handling is subject to security (WS-Security, SAML), transactional (WS-Transaction), and reliability (WS-RM) considerations. On the process front, there is a standard visual notation for modeling (BPMN) and a standard programmable form for execution (BPEL). Standards are both a blessing and a curse for vendors. Some of them genuinely help make a better design. Others are useless but need to be supported to win customer deals.

 

The Secret They Couldn't Hide—SOA is Process-Oriented


The three-layer stack that leading SOA vendors offer is unmistakably process-oriented. There are processes at each layer. In the BPM layer, the process character is obvious. ESB processes are single-burst, stateless processes, whose principal responsibility is to transform and route messages between parties. In the industry, these processes are known as 'message flows' or 'mediation flows'. The following figure shows an ESB process developed in IBM's Websphere Message Broker, which gets a message from an enrollment queue (EnrollmentIn), transforms it into the formats expected by the CRM and legacy enrollment systems (CRMAdjustment and LegacyAdjustment), and routes it to those systems by placing it on their inbound queues (CRMIn, LegacyIn), keeping a trace along the way of the messages that move through the flow (Trace1, Trace2, Trace3).

The next figure shows an ESB process built in BEA's AquaLogic Service Bus. When the web service loanGateway3 is invoked, it validates the request message. The validation logic is performed in validate loan application, which sits on the leftmost path of PipelinePairNode1, where actions are performed on the request in a request-response service. If the request is valid, the process routes the request to a business service for processing (Route to Normal Loan Processing Service).

Orchestration processes, unlike ESB processes, can be either stateless or stateful, and have a broader mandate than simple message processing. Most use cases today require both system and human actions, and they are invariably written in the form of processes. Business analysts are trained to conceive of use cases as box-and-arrow flow charts. A use case provides a business function; the use case document does not simply state what the function is, but elaborates how it plays out step-by-step. Significantly, the use case document is not elaborated technically. Indeed, many use cases are written before the organization has even selected a vendor platform! When the platform is selected, the technical architect decides how best to build the use case on that platform. The architect sensibly ties human actions in the use case to the BPM engine of the vendor's stack. As for system actions, well, the architect reasons, they obviously belong to the 'service layer'. The architect expects the service layer to be a tool for creating the sorts of wrappers (such as BankService) we encountered previously. Imagine her surprise when she sees actual examples of services on the platform. Boxes and arrow everywhere! Processes!

The following figure shows a service developed in TIBCO BusinessWorks. Whereas ESB services are tasked with basic routing and transformation, this service, which is discussed in detail in Chapter 5, is the functional implementation of actual business requirements. Specifically, this service handles an account holder's request to perform a transfer of funds by email to another party.

The next figure shows a fragment of an orchestration process developed in Oracle's service engine (BPEL Process Manager). The process manages the complex interactions between customer, retailer, and distributor in electricity enrollment. Chapter 4 discusses this process in detail.

Orchestration processes can either drive the end-to-end flow of the use case or simply provide the individual system steps. In either case, we build processes, not wrappers. Granted, in the latter case the processes are simpler. Still, what the analyst documents as one step in the use case is never a single step in the technical implementation. In the best case, it means a single system call, but even a single call requires accompanying error handling, transaction boundaries, input and output message processing, and possibly logging and auditing. In most cases, the service makes multiple system calls, possibly requiring multiple transactions, compensation, and the necessity to abort if responses are not obtained within a specified SLA.

It is no accident that vendors designed their service layer as a process layer. It is easier for developers to model a service as a process than to code it explicitly in a third-generation language. 'You don't need to be a Java guru to do SOA', vendors tell their customers. True, but to design processes is also not as easy as advertised. It is very easy to design bad processes. In fact, most SOA developers, even the Java gurus among them, are poor process modelers. Paradoxically, BPM modelers and business analysts are more comfortable in the process idiom than technical developers. Developers need to catch up. Bad processes mean bad SOA.

SOA need not be programming in the small. Too often SOA is seen as a technical infrastructure across use cases. With such strong process support by vendors, there's no reason why SOA cannot, in a sense, own the use case implementation. In that case, the job of SOA is not merely to assemble existing pieces but to build challenging new applications.

 

Summary


SOA is sold as an opportunity for the business to review its essential functions and for IT to build those functions through technology. The technology implementation is thought to be straightforward; services do not create new functionality but simply wrap existing functionality.

Vendor platforms promote the idea of services as processes. Use cases are process-oriented, so it's not unexpected to see process-oriented characteristics in an SOA implementation. Even a service whose job is to make a single backend call requires several steps (for error handling, logging, transactional boundaries, and so on), and is itself a small process. Vendors tell organizations not to code such complexity in a third-generation language such as Java. Organizations agree, and follow the process-oriented approach too.

The model stack, provided by the leading vendors, has BPM and SOA layers, and SOA is divided into process integration and ESB. This stack differs markedly from the CORBA-based architecture from the days of the Martian book.

SOA Cookbook teaches techniques in the modeling of orchestration processes, which belong to the process integration layer of the model stack.

About the Author

  • Michael Havey

    Michael Havey is an architect with thirteen years experience in integration, SOA, and BPM. A consultant in TIBCO's financial services practice, Michael previously worked as a consultant for IBM, BEA, Chordiant, and eLoyalty. Michael is the author of two books and several articles. Michael lives near Ottawa, Canada.

    Browse publications by this author