Identifying Standard Message Exchange Patterns
When we talk about Message Exchange Patterns, or MEPs, we're considering the direction and timing of data between the client and service. How do I get into the bus and what are the implications of those choices? Let's discuss the four primary options.
This is probably the pattern that's most familiar to you. We're all comfortable making a function call to a component and waiting for a response. When a service uses this pattern, it's frequently performing a remote procedure call where the caller accesses functionality on the distant service and is blocked until either a timeout occurs or until the receiver sends a response that is expected by the caller.
As we'll see below, while this pattern may set developers at ease, it may encourage bad behavior. Nevertheless, the cases where request/response services make the most sense are fine-grained functions and mashup services. If you need a list of active contracts that a hospital has with your company, then a request/response operation fits best. The client application should wait until that response is received before moving on to the next portion of the application. Or, let's say my web portal is calling an aggregate service, which takes contact data from five different systems and mashes them up into a single data entity that is then returned to the caller. This data is being requested for immediate presentation to an end user, and thus it's logical to solicit information from a service and wait to draw the screen until the completed result is loaded.
BizTalk Server 2009 has full support for both consuming and publishing services adhering to a request/response pattern. When exposing request/response operations through BizTalk orchestrations, the orchestration port's Communication Pattern is set to Request-Response and the Port direction of communication is equal to I'll be receiving a request and sending a response. Once this orchestration port is bound to a physical request/response receive port, BizTalk takes care of correlating the response message with the appropriate thread that made the request. This is significant because by default, BizTalk is a purely asynchronous messaging engine. Even when you configure BizTalk Server to behave in a request/response fashion, it's only putting a facade on the standard underlying plumbing. A synchronous BizTalk service interface actually sits on top of a sophisticated mechanism of correlating MessageBox communication to simulate a request/response pattern.
When consuming request/response services from BizTalk from an orchestration, the orchestration port's Communication Pattern is set to Request-Response and the Port direction of communication is equal to I'll be sending a request and receiving a response. The corresponding physical send port uses a solicit-response pattern and allows the user to set up both pipelines and maps for the inbound and outbound messages.
One concern with either publishing or consuming request/response services is the issue of blocking and timeouts. From a BizTalk perspective, this means that whenever you publish an orchestration as a request/response service, you should always verify that the logic residing between inbound and outbound transmissions will either complete or fail within a relatively brief amount of time. This dictates wrapping this logic inside an orchestration Scope shape with a preset timeout that is longer than the standard web service timeout interval.
For consuming services, a request/response pattern forces the orchestration to block and wait for the response to be returned. If the service response isn't necessary for processing to continue, consider using a Parallel shape that isolates the service interaction pattern on a dedicated branch. This way, the execution of unrelated workflow steps can proceed even though the downstream service is yet to respond.
This is your straightforward fire and forget pattern. The message is sent unidirectionally and asynchronously to a waiting receiver. If you grew up building components with fine-grained functional request/reply interfaces, this idea of throwing a message out of your application and not expecting anything back may seem a bit useless. However, this manner of service communication is a powerful way to build more event-driven applications and embrace non-blocking service invocation patterns. A one-way service interface may send a message to a single destination (a point-to-point solution), a defined list of recipients (multi-cast solution) or be a general broadcast (pub/sub solution). The key is, the caller remains unaware of the journey of the message once it is swallowed up by the service.
In scenarios where the sender and receiver may not both be online or active at the same time, a one-way service pattern offers a way to buffer this communication. For instance, I can build a service that offers an operation called PublishCustomerChange which takes a Customer entity possessing modified data attributes. The service itself may decide to queue up requests and only update the legacy Customer Management application during scheduled intervals throughout the day. However, the service may still receive requests all day, but because there is no expectation of a response to the submitter, the service can decide to prioritize the processing of the request until a more convenient time.
While the communication between endpoints may appear to be one-way, the default behavior of a WCF service returning void is to still provide a passive acknowledgement (or negative acknowledgement) that everything has run successfully. To prevent this completely and have a truly asynchronous service operation, set the IsOneWayproperty of an OperationContractattribute within the WCF service.
BizTalk Server 2009 is at its finest when working with one-way messaging patterns. Messages flow into the MessageBox and inherently cascade down to many places. When freed from the restraints of an expected service response, BizTalk Server can more powerfully chain together data and processes into far-reaching solutions. Consider a message sent to BizTalk via a one-way service interface. The loosely-coupled orchestrations and send ports can simply subscribe to this message's data, type, or attributes and act upon it. There is no concern for doing anything else but acting upon the data event. When a request/response receive port is designated as the service publisher, there can feasibly be only a single subscriber (and responder) to the request.
While all the BizTalk WCF adapters support one-way patterns, only the WCF-NetMsmq adapter requires it. MSMQ was designed for disconnected systems, and thus do not expect publishers to the queue to wait for any business data response.
There are cases where the caller of a service wants the benefits of non-blocking asynchronous service invocation, but also needs an actual data response from the service. In this case, a callback pattern can best fit your needs. In this situation, the caller acts as both a service consumer and a service provider. That is, the caller must be able to both send a message, and host an endpoint that the service can send a subsequent response to. You can consider this an asynchronous request/response pattern.
This can be a complicated pattern for service callers to accommodate. The caller has to design some intelligent strategies to correlate out-of-band responses with the original request made. During request/response invocations, the caller is blocked and doesn't proceed until the response has arrived. A sequence of processing is preserved. In a callback scenario, the client receives service responses well after the initial request. They have to make sure that:
- The response corresponds to a given request
- The response remains relevant to the application
Additionally, the client application should take into account the fact that a response may never actually arrive.
How does the service know where to send its response message back to? Typically the inbound request contains a pointer to the callback URI. The SOAP header is a good place to cram this sort of metadata instead of polluting the actual data message with context information.
WCF has some fairly rich support for callbacks through bindings such as the NetTcpBinding and NetNamedPipeBinding. While HTTP is inherently stateless and doesn't naturally support bidirectional communication, WCF provides a special HTTP binding named WSDualHttpBinding, which creates a matching pair of directional channels under the covers. A WCF developer can add callback contracts to a service contract and allow WCF clients to seamlessly call services and have an event raised when the service response is eventually received.
Sounds great, right? Unfortunately, the BizTalk WCF adapters do not openly support the WSDualHttpBinding or callback contracts in general. Instead, we need to get creative through the use of dynamic ports or polling strategies to implement a general purpose, cross platform callback patterns and thus you CAN effectively use the WSDualHttpBinding within a BizTalk receive location, which is out of scope of this article.
This final MEP is actually an extension of the one-way MEP. Instead of a sender and receiver of a service, consider the parties to be a publisher and subscriber. In a publish/subscribe MEP, data objects are sent to an endpoint where a dynamic set of interested entities yank a copy of the data for their own purpose. There is a one-to-many relationship between the publisher and subscribers. The data is published to the service in an asynchronous fashion with no expectation of a direct response.
BizTalk excels at the publish/subscribe service pattern. BizTalk can optionally subscribe on three distinct pieces of information:
- Message context: Each message that arrives into BizTalk Server has a property bag called context attached to it. Regardless of whether the data is in a structured data format (such as XML) or a binary blob (such as a PDF file), any object reaching the BizTalk MessageBox has context attached. Message subscribers can decide to register interest in topics associated with the message metadata found in context. For instance, a subscriber might choose to listen for all messages that arrived at a specific file location. Hence, their send port subscription would equal BTS.ReceivedFileName.
- Message Type: A message with a structured data format can be typed by the BizTalk endpoint prior to publication to the bus. The type value is typically equal to the namespace of the XML message plus the name of the root node. BizTalk subscribers can choose to pull any message that matches the data type they are interested in.
- Message Content: Developers can single out fields in a message schema for promotion which means that they are available for routing decisions within the BizTalk bus. When a message arrives into BizTalk, and a matching schema is found, the data elements designated for promotion have their values yanked out of the payload and put into context. Downstream subscribers can now specify data-level subscription topics. Hence, instead of pulling all new employees from the bus (based on a message type subscription), we could extract only those where EmployeeCountry == Germany using a content based subscription.
While BizTalk Server offers a variety of subscription topic mechanisms (context, type, or content), you are limited by the subscription operators available. For instance, you cannot create a subscription where "Organization CONTAINS Hospital". That is, you do not have options for wildcard searches or dynamic subscriptions in a send port. To achieve such a capability, you'd have to rely on orchestrations or custom pipeline components.
This manner of service invocation shifts the idea of a service from being just a functional component in a fancy SOAP wrapper to being an on-ramp to a distributed computing bus, where the publisher relinquishes knowledge and control of the data's processing path.
Types of services
There are multiple ways to look at types of services, and I've chosen to consider the types of services based on the category of message they accept.
A Remote Procedure Call (RPC) is a means of executing a function on a distant object. These remote invocations are meant to appear as if they were happening locally and typically following the request/response communication pattern. If you've written DCOM or CORBA components before, then this is a familiar concept. As SOAP gained traction, this was initially seen as just another way to execute remote functions but in a more interoperable way. However, this encourages a very point-to-point mentality.
RPC-style services follow the "Gang of Four" Command pattern where objects (in our case, messages) correspond to the action you wish to perform. In essence, you are sending a message that tells the remote service what to do. For example, the payload of an RPC-style service request meant to create a new customer entity would look like this:
While this type of command message may be acceptable for service requests that return a specific response (GetCustomerDetails or VerifyAddress), it is not a good practice for messaging solutions. Why? Architecturally, this is fairly tight coupling where the client confidently knows exactly what the service is supposed to do with their request and by the nature of the request, is demanding such behavior (DeleteCustomer). That's fairly presumptive, no? If the client truly had the ability to do this action themselves, they wouldn't need the service at all! In truth, the caller is only capable of making polite requests to the service where it remains the prerogative of the service to handle that request as it sees fit. For example, a demand to AddNewCustomer may not be a simple, synchronous event. The service may require human validation of the customer entry, or even decide that it doesn't want to add this new customer for reasons such as content duplication, failure of business rules, or simply irrelevance to the system.
Where does RPC play in BizTalk Server solutions? The classic ASMX service generator had a choice of producing bare or wrapped services that dictated whether the service operation followed an RPC style or not. The default style, wrapped, would enclose the message payload with the name of the service operations.
The BizTalk WCF Service Publishing Wizard offers no choice of messaging style. In fact, any WCF service generated either from schemas or orchestrations will follow the document-style outlined below. The name of the service operation is not part of the SOAP payload. If you wish to promote an RPC style service and aren't satisfied with the metadata generated by the BizTalk, you have the option to author your own RPC-oriented WSDL and provide that as the primary facade definition. Either way, the adapter is quite forgiving with each format, as it can either strip out nodes from parent elements, or conversely, add a wrapper to an inbound data object.
A document-style service is one in which the payload represents an encapsulated data structure devoid of any sort of instruction of what to do with it. As you can imagine, this type of service operation is much more coarse-grained and accepts plumper messages that are more self-describing. The service has all it needs to perform its action and doesn't rely on a stateful series of actions that provide context to the operation. If we re-factored our above RPC-style payload to be more document-centric, it might look like this:
Notice that instead of having a functional directive as the focus of the message payload, we are highlighting the data entity that is travelling across the wire. Either the single recipient or the broad list of subscribers can decide how they handle a NewCustomer and what they do with the data to fit their needs.
While this type of service can be used for request/response operations, they typically make lots of sense for scenarios where one application simply needs to exchange data with another application. These services can follow a one-way MEP when data is only being shared in a point-to-point manner, or could also be a one-way MEP with a publish/subscribe flavor where any interested party could have at the data.
Does BizTalk Server support this document-centric service type? It certainly does. As mentioned above, all BizTalk-generated WCF service endpoints define a document-centric contract.
Finally, let's consider the step-brother of the document-style service: the event service. For event services, when something of note occurs, the service is invoked and a message explaining the event is distributed. Unlike the document message, the event message typically contains a limited amount of information. You're basically telling the service that something important happened, but not sharing too much data or instructing it as to what to do next.
In these scenarios, it's quite possible that the service recipient will have to call back to the source system to actually get the relevant data set. Consider the NewEmployee event notification. When a new employee is added to the HR system, our service is called and the NewEmployeeEvent message is transmitted. Let's say that BizTalk fans this message out to all downstream systems that care about employee data. Our French office's security badge application has a service exposed which accepts the NewEmployeeEvent message and then peeks to see which country the new employee is associated with. If that country is France, then their service calls back to the source HR system to retrieve the full employee profile. What exactly is in this truncated event message if not the full document? Stay tuned for when we dissect the event message and consider how best to use it.
In the BizTalk sense, this sort of service is a very nice use of the event-driven message bus and is a great fit for one-way or pub/sub MEPs. Assuming that BizTalk accepts these event messages via its inbound service adapters, what are the possible outcomes? Three options are:
- Fan the message out to downstream services that have their own event-handling logic.
- Spawn a new orchestration workflow that performs event processing logic.
- Generate additional event messages within the bus based on content in the original message.
While event messages are typically asynchronous in nature, the life cycle of event processing can actually exploit all three service types identified in this article. In the sequence diagram below, I show how an initial event message (Order Event) may give way to a command request (GetOrderDetails) and a document-style response (OrderDetails).
The thought of using BizTalk Server for this sort of event-style routing may be counter to classic impressions of BizTalk as a simple point-to-point broker. We've moved beyond that antiquated concept and should consider BizTalk a serious option when architecting robust service-oriented event processing systems.
We covered a lot of ground in our quest to discover the key principles underpinning service-oriented design and evaluating how to champion these concepts within BizTalk Server 2009. Proper planning of a service oriented solution goes a long way towards ongoing agility and future return on investment that is critical to defining the success of a SOA.
If you have read this article you may be interested to view :
- The core principles of a service-oriented architecture with BizTalk Server 2009
- New SOA Capabilities in BizTalk Server 2009: WCF SQL Server Adapter
- Consuming the Adapter from outside BizTalk Server
- New SOA Capabilities in BizTalk Server 2009: UDDI Services