We have been doing integration for many decades in proprietary or ad-hoc manner. Today, the buzz word is SOA and in the integration space, we are talking about Service Oriented Integration (SOI). Let us look into the essentials of SOA and see whether the existing standards and APIs are sufficient in the integration space.
Why We Need SOA
We have been using multiple technologies for developing application components, and a few of them are listed as follows:
- Remote Procedure Call (RPC)
- Common Object Request Broker Architecture (CORBA)
- Distributed Component Object Model (DCOM)
- .NET remoting
- Enterprise Java Beans (EJBs)
- Java Remote Method Invocation (RMI)
One drawback, which can be seen in almost all these technologies, is their inability to interoperate. In other words, if a .NET remoting component has to send bytes to a Java RMI component, there are workarounds that may not work all the times.
Next, all the above listed technologies follow the best Object Oriented Principles (OOP), especially hiding the implementation details behind interfaces. This will provide loose coupling between the provider and the consumer, which is very important especially in distributed computing environments. Now the question is, are these interfaces abstract enough? To rephrase the question, can a Java RMI runtime make sense out of a .NET interface?
Along these lines, we can point out a full list of doubts or deficiencies which exist in today's computing environment. This is where SOA brings new promises.
What is SOA
SOA is all about a set of architectural patterns, principles, and best practices to implement software components in such a way that we overcome much of the deficiencies identified in traditional programming paradigms. SOA speaks about services implemented based on abstract interfaces where only the abstract interface is exposed to the outside world. Hence the consumers are unaware of any implementation details. Moreover, the abstract model is neutral of any platform or technology. This means, components or services implemented in any platform or technology can interoperate. We will list out few more features of SOA here:
- Standards-based (WS-* Specifications)
- Services are autonomous and coarse grained
- Providers and consumers are loosely coupled
The list is not exhaustive, but we have many number of literature available speaking on SOA, so let us not repeat it here. Instead we will see the importance of SOA in the integration context.
SOA and Web Services
SOA doesn't mandate any specific platform, technology, or even a specific method of software engineering, but time has proven that web service is a viable technology to implement SOA. However, we need to be cautious in that using web services doesn't lead to SOA by itself, or implement it. Rather, since web services are based on industry accepted standards like WSDL, SOAP, and XML; it is one of the best available means to attain SOA.
Providers and consumers agree to a common interface called Web Services Description Language (WSDL) in SOA using web services. Data is exchanged normally through HTTP protocol, in Simple Object Access Protocol (SOAP) format.
WSDL is the language of web services, used to specify the service contract to be agreed upon by the provider and consumer. It is a XML formatted information, mainly intended to be machine processable (but human readable too, since it is XML). When we host a web service, it is normal to retrieve the WSDL from the web service endpoint. Also, there are mainly two approaches in working with WSDL, which are listed as follows:
- Start from WSDL, create and host the web service and open the service for clients; tools like wsdl2java help us to do this.
- Start from the types already available, generate the WSDL and then continue; tools like java2wsdl help us here.
Let us now quickly run through the main sections within a WSDL. A WSDL structure is as shown here:
<?xml version="1.0" encoding="UTF-8"?>
<!-- other code goes here -->
<!-- other code goes here -->
<!-- other code goes here -->
We will now run through the main sections of a typical WSDL:
- types: The data types exchanged are expressed here as an XML schema.
- message: This section details about the message formats (or the documents) exchanged.
- portType: The PortType can be looked at as the abstract interface definition for the exposed service.
- binding: The PortType has to be mapped to specific data formats and protocols, which will be detailed out in the binding section.
- port: The port gives the URL representation of the service endpoint.
- service: Service can contain a collection of port elements.
Since JBI is based on WSDL, we can deal with many WSDL instances.
In web services, data is transmitted over the wire in SOAP. SOAP is an XML-based messaging protocol. SOAP defines a set of rules for structuring messages that can be used for simple one-way messaging and is useful for performing RPC-style interactions. Even though SOAP is not tied to any particular transport protocol, HTTP is the popular one.
A SOAP-encoded RPC dialogue contains both a request message and a response message. Let us consider a simple service method that takes a String parameter and returns a String type:
public String hello(String param);
The respective SOAP request is shown as follows:
<?xml version="1.0" encoding="UTF-8"?>
Similar to the request, the SOAP response is shown below:
<?xml version="1.0" encoding="utf-8"?>
Return From Server
As is the case with WSDL, we will be dealing with many instances of SOAP requests and responses. Let us not delve into the intricacies of WSDL and SOAP, as there are many text books doing the same. Let us continue with our discussion on integration.
Service Oriented Integration (SOI)
We need to identify ESB as an architectural pattern different from traditional integration architectures, which support standards-based services for integration. Gartner originally identified ESB as the core component in an SOA landscape. Gartner says:
SOA will be used in more than 80% of new mission-critical applications and business processes by 2010
Another market strategy study, conducted by Gartner in SOA reveals that:
The ESB category grew by 160% from year 2004 to year 2005
As is true with SOA, ESB-based integration has been increasingly using web services standards and technologies for service description, hosting, and accessing. The advantage is that we not only reap all the benefits of ESB architecture, but also make sure that the services exposed at the bus are complying with industry standards. This means consumers and providers makes use of existing toolsets and frameworks to interact with the ESB. Most of the current SOA toolsets support web services and related technologies. This opens up a whole lot of opportunities in the integration space too, which leads to SOI. When we virtualize services in the ESB, it is very critical to provide a uniform ecosystem for all kind of services, whether it is written in Java, or .NET, or C++.
If the services are as per the web service standards, ESB can provide the mediation services which will help to interconnect these services. Externalizing mediation logic out of services to an ESB will remove in-code coupling between services. To sum up, our current SOA infrastructure still exists which hosts services and takes care of service management and governance. Also an ESB provides SOI in the form of mediation, which provides communication (along with other features) between services.
JBI in J2EE—How they Relate
The Java 2 Enterprise Edition (J2EE) platform provides containers for client applications, web components based on servlets and Java Server Pages (JSP), and Enterprise JavaBeans (EJB) components. These J2EE containers provide deployment and run-time support for application components. They also provide a federated view of the platform-level services, provided by the underlying application server for the application components. In this section, we will look at where JBI is positioned in the J2EE stack.
Servlets, Portlets, EJB, JCA, and so on
The world of Java is fabulous. Java speaks about portable code (.class files), portable data (JAXP and XML), portable components (EJB), and now portable services (JAX-WS). However, untill now, there was no standard way by which we could do business-level integration across application servers from multiple vendors.
Business integration can be done with business components or with business services. We know that EJB components can interoperate across multiple vendor's application servers, and similar is the case with web services. We have been doing integration within our traditional J2EE programming paradigm in the pre-ESB era. The challenge of accessing, integrating, and transforming data has largely been done by developers using manual coding of J2EE components like POJO, EJB, JMS, and JCA. These J2EE APIs are optimized for business purpose programming, and not for integration. The after effect is that the J2EE programmers have to use low-level APIs to implement integration concerns.
Programmers create integration code having hard coded dependencies between specific applications and data sources, which are neither efficient nor flexible. Hence, a way to do integration in a loosely coupled manner and to port the integration solution as a whole is missing. Components and services are only a part of the integration solution. It also includes the strategy to integrate, the patterns adapted to route and transform messages and similar artifacts. All these differ for different vendor's products and JBI is trying to bring standardization across this space.
JBI and JCA—Competing or Complementing
The J2EE Connector Architecture (JCA) defines standards for connecting the J2EE platform to heterogeneous EIS systems. Examples of EIS systems include CICS, IMS, TPF systems, ERP, database systems, and legacy applications not written in the Java programming language. The JCA enables the integration of EISs with application servers and enterprise applications by defining a developer-level API and a vendor-level SPI. The SPIs enables an EIS vendor to provide a standard resource adapter for its EIS. The resource adapter plugs into an application server, thus providing connectivity between the EIS, the application server, and the enterprise application.
If an application server vendor has extended its system to support the JCA, it is assured of seamless connectivity to multiple EISs. So following JCA, an EIS vendor needs to provide just one standard resource adapter, which has the capability to plug-in to any application server that supports the JCA. He can now be assured that his EIS will plug into the J2EE continuum.
JCA-based adapters are usually used to integrate compliant ESBs with EIS. This means, by using JCA, an EIS vendor can be assured that his EIS can integrate with compliant ESBs. JCA provides the most efficient way of resource pooling, thread pooling, and transaction handling on JMS and other resource adapters. We have already seen the similarities between a message bus and a service bus. Along those lines, we can understand how important is the MOM technology such as JMS. Hence, the importance JCA has got in integration.
We also need to understand one subtle difference between JBI and JCA here. JCA is designed to support the traditional request-response model, but fails to support complex long running transactions and integration scenarios. Much back-end integration does not always fit well with the synchronous request-response model and JBI has got something else to offer here. JBI is based on a mediated message exchange pattern. That is, when a component sends a message to another component, the message exchange is mediated by the JBI infrastructure. As we will see shortly, JBI supports multiple message exchange patterns. Moreover, JBI-based ESBs will give more functionalities such as service composition, BPM, etc., which makes sense in handling back-end integration and long running transactions.
JBI—a New Standard
Every organization would like to leverage existing IT staff to run their integration effort similar to their daily IT development and operations. However, integration is a different art than normal software development and that is why we have integration architects. That doesn't mean integration experts have to be experts in vendor X's product, and are back to novice-level in vendor Y's product. If this has to happen, we need to do similar activities across multiple vendor's products.
As we already discussed, integration is not that simple as it involves integration architectures, integration patterns, and MOM expertise. We need to build the extra intelligent adapters around services and endpoints. These adapters are to work in tandem with the platform provider to give low-level features like communication, session, state, transport, and routing. Thus, business integration is not limited by just application programming, but involves components and engineering processes at the application and platform interface-level. Hence, to make integration artifacts portable, even the integration vendors (who deal with platform services) have a big part to play. This is where we need standardization as components and adapters should be portable across several vendor products.
Let me try to explain this point a bit further, as this is the prime concern in today's integration initiatives. Since the inception of J2EE, we have been using multiple J2EE containers from different vendors. Most of these J2EE containers also have an integration stack, which can be either plugged into their existing J2EE containers or can be run as standalone. Websphere Business Integration (WBI) message broker and BEA Aqualogic Service Bus (ALSB) are just a few of them in the list. It is true, that most of these integration stacks too support standard-based integration, following SOA principles and patterns. What about the integration artifacts as such?
When I say integration artifacts, I refer to any integration libraries available, or any custom configuration the developers do to the above libraries, the packaging and deployment artifacts they generate. In other words, if we create a .jar or .ear file containing some integration solution in ALSB; can we port it to WBI later? I leave this question open for the user community of these frameworks to answer. So, what has happened here? As always, the evolution of standards lags behind the evolution of technology. Hence, even before JBI, we had multiple programming paradigms to solve integration issues, but then they were tied to the vendor's environment. The promise of JBI is to bring portability across different JBI containers for the integration artifacts.
JBI in Detail
ESB architecture can facilitate the collaboration between services. JBI provides a collaboration framework which provides standard interfaces for integration components and protocols to plug into, thus allowing the assembly of SOI frameworks.
JSR 208 is an extension of J2EE, but it is specific for JBI Service Provider Interfaces (SPI). SOA and SOI are the targets of JBI and hence it is built around WSDL. Integration components can be plugged into the JBI environment using a service model based on WSDL. Service composition is a major target in ESB architecture and the JBI environment aggregates multiple service definitions in the WSDL form into the message infrastructure.
In the context of a larger service composition, we have multiple partners (service providers or service consumers) and the metadata for interaction of these individual partners are termed as the business protocol. The metadata of choreography played by a business process in a business protocol is termed as the abstract business process. Partner processes interact with each other by looking at abstract business process; it is the ESB's job to realize this abstract business process. JSR aims to make this abstract business process definition portable. This means the wiring details between components in a service composition scenario can be extracted into a separate layer and it is this layer which we are speaking about JSR 208. Thus, JSR 208 or JBI is a foundation for realizing SOI.
JSR 208 mandates the following for JBI components:
- Portable: Components are portable across JBI implementations.
- Manageable: Components can be managed in a centralized manner.
- Interoperable: Components should be able to provide service to and consume services from other components, despite the fact that they come from different sources and transport protocols.
This section helps us to understand the major components in JBI architecture and their roles with reference to the following figure:
The major components are explained here:
- JBI environment: A JBI environment is a single Java Virtual Machine (JVM) where we can deploy integration artifacts. This JBI can be a standalone ESB or an ESB embedded in the JVM of an application server. In the latter case, even an EJB component deployed in an application server can function as a provider or consumer to the ESB, thus further narrowing down the bridge between traditional J2EE application servers and the relatively new ESB.
- Service Engine (SE): SEs are service providers or service consumers deployed locally within a JBI environment. They provide the actual business logic like transformation. Transformation service can be done with the help of an XSLT engine by using a stylesheet. Another engine may use JCA to give a data access service, or Business Process Execution Language (BPEL), or even custom logic to integrate legacy code like that in CICS or mainframe.
- Binding Components (BC): BC provide communications protocol support and they are normally bound to components deployed remotely from the JBI run time. In fact, nothing prevents a user from defining a binding for a local service in the case where it closely resembles SE. Thus BC provides remote access to services for remote service providers and consumers.
The distinction between SE and BC is important for various pragmatic reasons. Mainly, it separates service logic from binding protocol. This facilitates reusability.
- JBI Container: Similar to the container in an application server, a JBI environment by itself is a JBI container. This container hosts SE and BC. The interesting part is a SE is again a container for engine specific entities like XSLT engine, stylesheets, rules engines, and scripts. Thus, a JBI environment is a container of containers whereas a service engine is a container for hosting WSDL defined providers and consumers local to the JBI.
- Normalized message: A normalized message consists of two parts—the actual message in XML format, and message metadata which is also referred to as the message context data. The message context data helps to associate extra information with a particular message, as it is processed by both plug-in components and system components in the bus.
- Normalized message router (NMR): The nerve of the JBI architecture is the NMR. This is a bus through which messages flow in either direction from a source to a destination. The specialty of this router is that messages are always in a normalized format, irrespective of the source or destination. NMR thus acts as a lightweight messaging infrastructure which facilitates actual message exchange in a loosely coupled fashion. NMR provides varying QOS functionalities and the three levels of message delivery guarantee provided by NMR are listed as follows:
Best effort: Message may be delivered only once or more than once, or even the message can be dropped.
At Least Once: Message has to be delivered once or more, but cannot be dropped. Hence, duplicates can exist.
Once and only once: It is guaranteed that messages will be delivered once and only once.
- Pluggable components: The loosely coupled feature of ESB is due to the pluggable architecture for service components. There are two broad categories of pluggable components called SE and BC. Pluggable components can play the role of service provider, service consumer, or both. Pluggable components are connected to the NMR through a Delivery Channel.
- Service providers and service consumers: There are mainly two roles played by pluggable components within an ESB, the service provider and service consumer. Components can act as both a provider and a consumer at the same time. Service definition is available in the form of WSDL and this is the only shared artifact between providers and consumers. Since endpoint information is not shared between providers and consumers in the ESB, loosely coupled integration is facilitated.
It is to be noted that a single service definition may be implemented by multiple providers. Similarly, a consumer's service definition of interest might be provided by multiple providers too, in ESB architecture. Moreover, the role of both provider and consumer can be played by the component either directly or through a proxy for a remote service.
- Delivery Channel (DC): DC connects a message source and a destination. The messaging infrastructure or the NMR is the bus for message exchange for multiple service providers and consumers. Hence when a service provider has got information to communicate, it doesn't just fling the message into the NMR, but adds the message to a particular DC. Similarly, a message consumer doesn't just pick it up at random from the messaging system. Instead, the consumer receives the message from a particular DC. Thus DCs are logical addresses in the ESB.
JBI is the new integration API introduced in the J2EE world. It is a great enabler for SOA because it defines ESB architecture, which can facilitate the collaboration between services. It provides for loosely coupled integration by separating out the providers and consumers to mediate through the bus.
In part 2 of the article, we will look at the Provider—Consumer Contract and Message Exchange Patterns. We will also consider the different options provided.