Chapter 10. Advanced Integration with Powerful, Scalable Service Bus in the Cloud

Azure Service Bus is an asynchronous messaging service based on Platform as a Service (PaaS) architecture. It is designed as a highly scalable and reliable messaging system used to connect applications, services, and devices. Being a lightweight, messaging-only architecture, it is also used as the backbone communications channel between many cloud-based solutions.

Service Bus supports two types of message exchange patterns, relayed messaging, and brokered messaging. The main distinction between the two exchange patterns is that relayed service requires both the service consumer and provider to be online simultaneously, whereas with the brokered service, either the consumer or provider may be taken offline at any time without affecting the communication channel.

In this chapter, you will learn the following topics:

• What is Azure Service Bus and where it sits in the integration use cases

• Publishing and subscribing...

Service Bus types

Azure Service Bus offers a collection of four different types of messaging technologies:

• Queues: This type provides asynchronous one-way messaging

• Topics/Subscriptions: This type provides a publish/subscribe architecture style of integration

• Event Hubs: This type is used to ingest event-type messages on a massive scale

• Relays: This type allows the relaying of messages through a common channel

Each type has distinctive characteristics that make it suitable for different messaging scenarios. In the following sections, we will discuss queues, topic/subscriptions, and in Chapter 11, Connecting to Event Hubs and an Introduction to IoT Hubs, we will cover event hubs.

Service Bus tiers

Azure Service Bus is available in three tiers - Basic, Standard, and Premium. The Basic tier provides only very basic queuing, whereas the  Standard tier provides many more features such as topics, transactions, de-duplication, sessions and forwardTo/SendVia. The Premium tier is based on the existing Standard tier but with the added enhancements consisting of isolated CPU and memory resources. This isolation from other tenants provides scalability, greater predictability, performance, and a larger message payload.

Normally, you would use the Premium tier for systems that are latency sensitive; if you are receiving server busy errors from Azure, you need to scale your systems on a massive scale and you will send more than 3000 messages to a topic per second.

Pricing for the Premium tier is priced at a daily flat rate per the messaging unit purchased. There are no other charges included.

Note

A Messaging Unit is a set of dedicated resources exclusively reserved for the Premium...

Service Bus Queue characteristics

Service Bus Queues provide a one-way message exchange pattern to pass messages between distributed and loosely coupled applications using the Azure SDK, WCF, AMQP, or HTTPS endpoints. These queues provide load leveling, by default, where the receiver processes messages off the queue at its own pace.

Load leveling can also be easily implemented by reading the queue length property and spinning up more processes to read the queue concurrently. As the queue length diminishes, the extra processors can be taken offline. When using multiple processors to read messages from the same queue concurrently, careful consideration must be given to the type of read lock you wish to perform. More about this is explained in the section Retrieving messages off a Queue.

There are many uses for this type of coupling pattern:

• Hybrid applications, which allow you to connect an on-premises application to a cloud-hosted service or application

• Loosely coupled applications, where Service...

Retrieving messages off a queue

Reading a message off the queue may be requested using one of two modes. PeekLock, which is the default mode, reads the message of the queue and places a lock on the read message. This makes the message invisible to other consumers of the queue. The message will reappear on the queue if the consumer does not issue a Complete command within the specified VisibilityTimeout period. The message can also reappear if the consumer calls the Abandon method. This type of processing is ideal for when guaranteed processing of a message is mandatory.

The following process flow describes reading a message from the queue using the PeekLock method. Note the two scenarios when a message is placed on the dead-letter queue. First one is when the message is not read within the specified TTL period, and the second scenario is when the de-queuing count equals zero.

The second option is the ReceiveAndDelete mode. This is when once the message is read, it will immediately be deleted...

Deferring message processing

Deferring messages allows the processing of higher priority messages first and then servicing the lower priority messages at a later time.

To use this feature, the receive mode must set to PeekLock. The client receiving the message then has the option to mark the message as being deferred. When the client marks the message to be deferred, they must keep track of the message sequence number in a durable store to be retrieved later.

When a message has been deferred, it will stay in the queue until it is retrieved and the Complete() method has been called. The message will also be removed from the Queue if a message TTL expiry timeout occurs.

The message is read from the queue using the QueueClient class available in the Azure Service Bus SDK and passing the message sequence number that was persisted previously.

Security

Clients access Service Bus resources by presenting an access token. The token specifies the URI to be accessed and an expiry time of the token.

Note

The Service Bus Namespace owner account (RootManageSharedAccessKey) should not be shared or embedded in code. This is a highly privileged account that gives rights to delete and create Azure artifacts and should only be used for administrative tasks.

There are two options available for clients to be authenticated to allow access to the Service Bus:

• Shared Access Signature (SAS): This provides authentication using a shared key configured on the namespace or on a Service Bus entity (Queues, Topics, and Subscriptions) with specific rights for listening, sending, or managing. The key is then used to generate a SAS token, which the client uses to authenticate with the Service Bus.

• Access Control Service (ACS): This provides identity federation with various providers. To access the Service Bus entities, the client requests a Simple Web Token...

Managing Service Bus outages and disasters

One of the items that is overlooked when using cloud services in general is disaster recovery. Many IT professionals overlook the fact that most services are not geo replicated by default to other data centers unless you specifically configure it. Azure Service Bus does not provide any configuration settings for geo replication; therefore, you must implement your own DR strategy.

There are two types of outages that can occur-message store failures and data center outages. Message store failures occur when the underlying storage subsystem fails due to hardware faults. Data center outages may occur due to a failed update deployment to the subsystem services, power outages due to a power supply failure or backup generators and network connectivity failures.

To mitigate against message store failures, you will need to create a queue with multiple partitions. These partitions are guaranteed to be created on different storage systems within the same data...

Service Bus topics

Topics are another communication model that provides a publish/subscribe architecture pattern. This allows a message to be consumed by many subscribers, and each subscriber is able to process the message independently.

The following image shows a component publishing a message to a topic with two subscribers. Each subscriber setting up a Subscription filter on the purchase amount.

Subscriptions

To receive messages from a topic queue, you would create a subscription. A subscription resembles a virtual queue that receives a copy of the message sent to a topic queue.

Note

Note that a single subscription cannot be used for multiple topics. Instead, you need to create multiple subscriptions.

Subscription rules

In some scenarios, it would be helpful to filter message properties and only receive messages that satisfy the filter condition. Currently, if a subscriber creates a subscription to a topic, all messages arriving at that topic are made available to the subscriber. This is because the default filter MatchAll is applied when no filters are specified on the creation of a subscription.

Azure Service Bus has the concept of subscription rules, which allows you to define filters and actions that are applied to a Topic.

Note

More information on creating subscriptions from Microsoft can be found at: http://msdn.microsoft.com/en-us/library/microsoft.servicebus.namespacemanager.createsubscription...

Partitioned queues/topics

Without partitions, a queue or topic is handled by a single message broker and stored in a single messaging store, which can constrain performance. Using partitions, a queue or topic can be spread across multiple brokers and stores, thereby providing a higher throughput rate than a single message broker and store. These partitions contain all the features of a no partitioned queue or topic, such as transactions and sessions.

When messages arrive at the queue or topic, they are distributed in a round-robin fashion to all the fragments of a partitioned queue or topic if no partition key has been defined.

To control which fragment receives what message, the properties SessionId, PartitionKey, and MessageId may be used as partition keys. All messages received using the same partition key will be processed by that specific fragment. If that fragment is temporarily unavailable, an error will be returned.

When a client reads a message from a partitioned queue or topic, the...

Administration tools

Being able to view and administer messages in a production environment is mandatory, especially when things go wrong. Fortunately, there is a tool available named Service Bus Explorer, which allows you to connect to a Service Bus namespace and administer the message entities. It can be downloaded at https://code.msdn.microsoft.com/windowsazure/Service-Bus-Explorer-f2abca5a .

Scenario

In this code sample, we will use our fictitious company Sunny Electricals to add purchase orders received from our website to a Service Bus via a Logic App.

The Service Bus will be configured with two topics-one for purchase orders over $5000 and another topic for orders under $5000 to receive message from the Logic Apps.

Let's go ahead, step by step, to create the necessary artifacts for the Sunny Electricals purchase order solution. We will start with creating a new instance of Azure Service Bus namespace in the Azure Portal.

Log in to Azure portal using your official or personal Azure subscription...

Summary

Azure Service Bus provides a robust and scalable messaging infrastructure. It provides the framework to connect on-premise and cloud-based applications/services together.

Using topics with the Service Bus provides the framework to build scalable publish/subscribe message exchange patterns.

You learned how to wire up a Logic App to a Service Bus topic and how to subscribe to the messages using another Logic App.

In the next chapter, we will look at Event Hubs and IoT Hubs, which are another form of message queuing.