Testing and Tracing Applications

Exclusive offer: get 50% off this eBook here
RabbitMQ Essentials

RabbitMQ Essentials — Save 50%

Hop straight into developing your own messaging applications by learning how to utilize RabbitMQ with this book and ebook

$16.99    $8.50
by David Dossot | April 2014 | Open Source

This article by David Dossot, author of RabbitMQ Essentials, presents two handy tracing tools provided by RabbitMQ, which are very likely to become prominent in your developer's toolbox.

(For more resources related to this topic, see here.)

Tracing RabbitMQ

Tracing the execution of a program is a convenient way to figure out what is really happening under the hood when reasoning about a particular behavior leads to no firm conclusion. Usually, the buck stops at the border where the application interacts with external resources such as the RabbitMQ broker. The good news is that RabbitMQ provides two tools that can be of tremendous help when it comes to tracing the interactions with a broker.

The first of these tools is Tracer, an AMQP-aware network proxy that can be placed between a RabbitMQ client and a broker in order to gain insight into the interactions that are happening between each other. Tracer is available as part of the Java client download available at http://www.rabbitmq.com/download.html.

After installation, Tracer can be started with the following:

runjava.sh com.rabbitmq.tools.Tracer [listenPort] [connectHost] [connectPort].

All the parameters are optional. If left blank, Tracer will start a local proxy listening on port 5673 and connect to a local RabbitMQ on port 5672. Since you're happy with the defaults, you start Tracer with just the following command line:

$ ./runjava.sh com.rabbitmq.tools.Tracer

Now you can run the integration tests you've just created through this proxy. Do you remember that we made the connection information configurable on these tests? The approach is going to pay off now as we will configure them to go through the proxy port instead of directly hitting the RabbitMQ broker. You do this by running the following command line:

$ mvn -Pintegration_tests -Dtest.rmq.addresses=localhost:5673 verify

The output of Tracer is very verbose as it includes the complete details of the AMQP operations. Hereafter only the columns that show the channel ID, interaction direct (-> is client to broker and <- is opposite), and the name of the operation is reproduced, with the interactions related to the subscriber highlighted:

ch#0 <- <connection.start> ch#0 -> <connection.start-ok> ch#0 <- <connection.tune> ch#0 -> <connection.tune-ok> ch#0 -> <connection.open> ch#0 <- <connection.open-ok> ch#1 -> <channel.open> ch#1 <- <channel.open-ok> ch#1 -> <queue.declare> ch#1 <- <queue.declare-ok> ch#1 -> <channel.close> ch#1 <- <channel.close-ok>() ch#1 -> <channel.open> ch#1 <- <channel.open-ok> ch#1 -> <basic.consume> ch#1 <- <basic.consume-ok> ch#2 -> <channel.open> ch#2 <- <channel.open-ok> ch#2 -> <basic.publish> ch#2 -> <channel.close> ch#1 <- <basic.deliver> ch#2 <- <channel.close-ok> ch#1 -> <basic.cancel> ch#1 <- <basic.cancel-ok> ch#1 -> <channel.close> ch#1 <- <channel.close-ok> ch#0 -> <connection.close> ch#0 <- <connection.close-ok>

You can see the operations from the client and the responses from the broker, typically being named after the operation and suffixed with -ok. In essence, the following is the AMQP synopsis of the test code you're running:

  • Establish a connection

  • Open a channel, use it to declare the test queue, and close it

  • Open a channel, use it to consume a queue

  • Open a channel, use it to publish the test message, and close it

  • Receive the message delivery, cancel the consumer, and close its channel

  • Close the connection

Notice how the connection start and tune operations are initiated by the broker as a response to establishing the connection to it. Also, notice that the channel number gets reused after being closed; it may seem that the same channel #1 has been used for creating the test queue and subscribing to it, but that's not the case since this channel has been explicitly closed. Only its identifier has been reused.

Tracer is a very powerful tool to easily gain a deep understanding of the AMQP protocol and the usage your applications make of it. However, it requires you to insert a proxy between a client and the broker it connects to. Fear not if this is an issue; RabbitMQ has more than one trick in its bag of tracing tools.

Drinking at the Firehose

RabbitMQ offers the possibility of spying on all message publications and delivery operations that happens in a particular virtual host of a broker. This feature is called the Firehose tracer. When activated on a virtual host, a copy of all published and all delivered messages is sent to the amq.rabbitmq.trace exchange (which is automatically created in every virtual host).

The routing key used for messages published to the amq.rabbitmq.trace exchange is publish.<exchange_name> for publication events and deliver.<queue_name> for message deliveries. The original message body is carried to the copies sent to this exchange. Extra information about the original publication or delivery event are added in a set of headers, including exchange_name for the name of the exchange where the message was originally published or redelivered if the message has been delivered more than once.

You want to use the Firehose when running the integration tests to see the exchanged messages from the broker's standpoint. Before activating the Firehose on RabbitMQ, you need first to create a client application that will subscribe to the exchange and print out the messages that come to it. For this, you create the following Python script:

#!/usr/bin/env python import amqp connection = amqp.Connection(host='localhost', userid='ccm-dev',
password='coney123', virtual_host='ccm-dev-vhost') channel = connection.channel() EXCHANGE = 'amq.rabbitmq.trace' QUEUE = 'firehose-queue' channel.queue_declare(queue=QUEUE, durable=False,
auto_delete=True, exclusive=True) channel.queue_bind(queue=QUEUE, exchange=EXCHANGE, routing_key='#') def handle_message(message): print message.routing_key, '->', message.properties, message.body print '--------------------------------' channel.basic_consume(callback=handle_message, queue=QUEUE, no_ack=True) print ' [*] Waiting for messages. To exit press CTRL+C' while channel.callbacks: channel.wait() channel.close() connection.close()

After starting this script, you turn the Firehose on by running the following command line:

$ sudo rabbitmqctl -p ccm-dev-vhost trace_on Starting tracing for vhost "ccm-dev-vhost" ... ...done.

Now you can run the integration tests again, this time on the standard port since no proxying is needed with the Firehose:

$ mvn -Pintegration_tests verify

Let's now look at the following output of the Firehose consumer Python script:

publish. -> {'application_headers': {u'node':
u'rabbit@pegasus', u'exchange_name': u'', u'routing_keys':
[u'amq.gen-vTMWL--04lap8s8JPbX5gA'], u'properties': {}}}
93b56787-b4f5-41e1-8c6f-d5f9b64275ca -------------------------------- deliver.amq.gen-vTMWL--04lap8s8JPbX5gA ->
{'application_headers': {u'node': u'rabbit@pegasus',
u'exchange_name': u'', u'redelivered': 0, u'routing_keys':
u'amq.gen-vTMWL--04lap8s8JPbX5gA'], u'properties': {}}}
93b56787-b4f5-41e1-8c6f-d5f9b64275ca --------------------------------

As you can see, the publication to the default exchange (remember, its name is an empty string) and the delivery to the automatically named test queue are clearly visible. All the details that concern them are readily available in the message properties.

Keep in mind that running the Firehose is taxing for the RabbitMQ broker, so when you're done with your tracing session, shut it down with the following:

$ sudo rabbitmqctl -p ccm-dev-vhost trace_off Stopping tracing for vhost "ccm-dev-vhost" ... ...done.

The Firehose will come handy when tracing what's happening between your different applications and your RabbitMQ brokers in depth. Keep in mind that using unique message IDs, as you've learned throughout this article, will help you a lot when the time comes to perform forensics analysis and trace the progression of messages across your complete infrastructure.

Summary

In this article, you have discovered powerful tracing tools to peek deeper under the hood of the AMQP protocol and the RabbitMQ broker.

Resources for Article:


Further resources on this subject:


RabbitMQ Essentials Hop straight into developing your own messaging applications by learning how to utilize RabbitMQ with this book and ebook
Published: April 2014
eBook Price: $16.99
Book Price: $27.99
See more
Select your format and quantity:

About the Author :


David Dossot

David Dossot has worked as a software engineer and an architect for more than 18 years. He has been using RabbitMQ since 2009 in a variety of different contexts. He is the main contributor to the AMQP transport for Mule. His focus is on building distributed and scalable server-side applications for the JVM and the Erlang VM. He is a member of IEEE, the Computer Society, and AOPA, and holds a diploma in Production Systems Engineering from ESSTIN.

He is a co-author for the first and second editions of Mule in Action (Manning Publications Co.). He is a Mule champion and a DZone Most Valuable Blogger. He commits on multiple open source projects and likes to help people on Stack Overflow. He's also a judge for the annual Jolt Awards software competition.

Books From Packt


OpenStack Cloud Computing Cookbook
OpenStack Cloud Computing Cookbook

Getting Started with ownCloud
Getting Started with ownCloud

Cloud Development and Deployment with CloudBees
Cloud Development and Deployment with CloudBees

Getting Started with Citrix® CloudPortal™
Getting Started with Citrix® CloudPortal™

Apache CloudStack Cloud Computing
Apache CloudStack Cloud Computing

Instant RabbitMQ Messaging Application Development How-to
Instant RabbitMQ Messaging Application Development How-to

RabbitMQ Cookbook
RabbitMQ Cookbook

OpenStack Cloud Computing Cookbook - Second Edition
OpenStack Cloud Computing Cookbook - Second Edition


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software