RabbitMQ Essentials - Second Edition

By Lovisa Johansson , David Dossot
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. A Rabbit Springs to Life

About this book

RabbitMQ is an open source message queuing software that acts as a message broker using the Advanced Message Queuing Protocol (AMQP). This book will help you to get to grips with RabbitMQ to build your own applications with a message queue architecture. You’ll learn from the experts from CloudAMQP as they share what they've learned while managing the largest fleet of RabbitMQ clusters in the world.

Following the case study of Complete Car, you’ll discover how you can use RabbitMQ to provide exceptional customer service and user experience, and see how a message queue architecture makes it easy to upgrade the app and add features as the company grows. From implementing simple synchronous operations through to advanced message routing and tracking, you’ll explore how RabbitMQ streamlines scalable operations for fast distribution. This book will help you understand the advantages of message queue architecture, including application scalability, resource efficiency, and user reliability. Finally, you’ll learn best practices for working with RabbitMQ and be able to use this book as a reference guide for your future app development projects.

By the end of this book, you’ll have learned how to use message queuing software to streamline the development of your distributed and scalable applications.

Publication date:
August 2020
Publisher
Packt
Pages
154
ISBN
9781789131666

 
A Rabbit Springs to Life

Messaging or message queuing is a method of communication between applications or components. Thanks to message queues, these applications can remain completely separate as they process their individual tasks. Messages are typically small requests, replies, status updates, or even just information. A message queue provides a temporary place for these messages to stay, allowing applications to send and receive them as necessary.

RabbitMQ is an open source message broker that acts as the intermediary or middleman for independent applications, giving them a common platform to communicate. RabbitMQ mainly uses an Erlang-based implementation of the Advanced Message Queuing Protocol (AMQP), which supports advanced features such as clustering and the complex routing of messages.

This chapter includes information about how to get started with RabbitMQ, and why it would benefit an architecture. This book follows a fictitious taxi agency, Complete Car (CC), to demonstrate how they have implemented RabbitMQ into the architecture. This chapter shows how to install and configure RabbitMQ so that it's easy to get everything up and running.

This chapter will cover the following topics:

  • Explaining message queues
  • Discovering AMQP and RabbitMQ
  • Using RabbitMQ in real life
  • Exploring the benefits of message queuing
  • A RabbitMQ scenario
  • Getting ready for RabbitMQ

Let's get started!

 

Technical requirements

 

Explaining message queues

Smoke signals, couriers, carrier pigeons, and semaphores: if this was a riddle, the word messages would immediately spring to mind. Humanity has always had the need to connect, finding new ways to defy challenges posed by the distance between the different groups of people needing to communicate. Humankind has come a long way with modern technologies, but essentially, the basics remain. Senders, recipients, and messages are at the core of all our communication infrastructures.

Software applications have the same needs; systems need to communicate and send messages between each other. They sometimes need to be sure that the message that's been sent has reached its destination, and sometimes they need to receive an immediate response. In some cases, they may even need to receive more than one response. Based on these different needs, different styles of communication between systems have emerged.

AMQP, RabbitMQ's default protocol, is explained in the next section.

 

Discovering AMQP and RabbitMQ

Message queuing is a one-way communication style that provides asynchronous interaction between systems. As this chapter continues to describe how message queues work, the benefits will become clear. Some background on the request-response message exchange pattern will shed light on how RabbitMQ works.

 

The request-response message exchange pattern

There are many types of message exchange patterns, but the request-response style is the most common. A system, acting as a client, interacts with another remote system, which is acting as a server. The client sends a request for data, and the server responds to the request, as shown in the following diagram:

Fig 1.1: The request-response interaction between the client and the server

The request-response style is used when the client must have an immediate response or wants the service to complete a task without delay, such as being placed on hold when calling a restaurant to reserve a table:

Fig 1.2: Request-response between a client and a restaurant

Whether it takes the form of a remote procedure call, a web service invocation, or consumption of a resource, the model is the same: one system sends a message to another and waits for the remote party to respond. Systems communicate with each other in a point-to-point manner, where events and processes occur simultaneously or have dependencies or events related to time; the interaction between the client and server is synchronous.

One on hand, this request-response style gives developers a simple programming model as everything happens procedurally. On the other hand, the tight coupling between both parties has a deep impact on the architecture of the whole system as it is hard to evolve, hard to scale, and hard to ship in independent releases.

 

Message queuing exchange pattern

Message queuing is a one-way style of interaction where one system asynchronously interacts with another system via messages, generally through a message broker. A requesting system in asynchronous communication mode does not wait for an answer or require return information; it continues processing no matter what. The most common example of such an interaction is an email. The point is, asynchronous communication does not involve waiting for a response in order to continue processing. In fact, there may be no response or it may take some time for a response to be sent. Whatever the case, the system does not rely on a response to continue the process.

Messages flow in one direction, from the publisher to the broker and finally to the consumer:

Fig 1.3: Basic components of a one-way interaction with message queuing

Systems and applications play both the role of message publishers (producers) and message consumers (subscribers). A publisher publishes a message to a broker that they rely on to deliver the data to the intended consumer. If a response is required, it will arrive at some point in time through the same mechanism, but reversed (the consumer and producer roles will be swapped).

 

A loosely coupled architecture

One big advantage of the messaging queuing approach is that systems become loosely coupled with each other. They do not need to know the location of other nodes on the network; a mere name is enough to reach them. Systems can, therefore, be evolved in an independent manner with no impact on each other as the reliability of message delivery is entrusted to a broker.

The following diagram illustrates a loose coupling between the publisher and the consumer:

Fig 1.4: Message queuing enabling a loosely coupled architecture

If one system is down for any reason, the other part of the system can still operate, and messages that are supposed to be sent between them wait in the queue.

The architecture represented via message queuing allows for the following:

  • The publishers or consumers can be updated one by one, without them impacting each other.
  • The performance of each side leaves the other side unaffected.
  • The publishers or consumers are allowed to fail without impacting each other.
  • The number of instances of publishers and consumers to scale and to accommodate their workload in complete independence.
  • Technology mixing between consumer and publishers.

The main downside of this approach is that programmers cannot rely on the mental model of procedural programming where events occur one after another. In messaging, things happen over time. Systems must be programmed to deal with this.

If all this is a little blurry, use the example of a well-known protocol, Simple Mail Transfer Protocol (SMTP). In this protocol, emails are published (sent) to an SMTP server. This initial server then stores and forwards the email to the next SMTP server, and so on until the recipient email server is reached. At this point, the message is queued in an inbox, waiting to be picked up by the consumer (typically, via POP3 or IMAP). With SMTP, the publisher has no idea when the email will be delivered or whether it will eventually be delivered at all. In the case of a delivery failure, the publisher is notified of issues later down the line.

The only sure fact is that the broker has successfully accepted the message that was initially sent. This entire process can be seen in the following diagram:

Fig 1.5: The email infrastructure as an analogy for message queuing

Furthermore, if a response is needed, it will arrive asynchronously using the same delivery mechanism but with the publisher and consumer roles reversed.

With these fundamental notions established, it is the perfect time to delve into the messaging protocol that will be used in this book, which is AMQP.

 

Meet AMQP

AMQP is an open standard protocol that defines how a system can exchange messages. The protocol defines a set of rules that needs to be followed by the systems that are going to communicate with each other. In addition to defining the interaction that happens between a consumer/producer and a broker, it also defines the representation of the messages and commands being exchanged. AMQP is truly interoperable as it specifies the wire format for messages, leaving nothing open to interpretation by a particular vendor or hosting platform. Since it is open source, the AMQP community is prolific and has broker and client implementations in a wide range of languages.

RabbitMQ is built upon the AMQP 0-9-1 specification, but plugins are available that support AMQP 1.0.

The AMQP 0-9-1 specification can be downloaded at http://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf.

The following is a list of the core concepts of AMQP, which will be explained in detail in upcoming chapters:

  • Broker or message broker: A broker is a piece of software that receives messages from one application or service, and delivers them to another application, service, or broker.
  • Virtual host, vhost: A vhost exists within the broker. It's a way to separate applications that are using the same RabbitMQ instance, similar to a logical container inside a broker; for example, separating working environments into development on one vhost and staging on another, keeping them within the same broker instead of setting up multiple brokers. Users, exchanges, queues, and so on are isolated on one specific vhost. A user connected to a particular vhost cannot access any resources (queue, exchange, and so on) from another vhost. Users can have different access privileges to different vhosts.
  • Connection: Physical network (TCP) connection between the application (publisher/consumer) and a broker. When the client disconnects or a system failure occurs, the connection is closed.
  • Channel: A channel is a virtual connection inside a connection. It reuses a connection, forgoing the need to reauthorize and open a new TCP stream. When messages are published or consumed, it is done over a channel. Many channels can be established within a single connection.
  • Exchange: The exchange entity is in charge of applying routing rules for messages, making sure that messages are reaching their final destination. In other words, the exchange ensures that the received message ends up in the correct queues. Which queue the message ends up in depends on the rules defined by the exchange type. A queue needs to be bound to at least one exchange to be able to receive messages. Routing rules include direct (point-to-point), topic (publish-subscribe), fanout (multicast), and header exchanges.
  • Queue: A queue is a sequence of items; in this case, messages. The queue exists within the broker.
  • Binding: A binding is a virtual link between an exchange and a queue within the broker. It enables messages to flow from an exchange to a queue.

The following diagram illustrates an overview of some of the concepts in AMQP:

Fig 1.6: Overview of some of the concepts defined by the AMQP specification

The open source broker shown in detailin this book has been built from the ground up to support AMQP, but many other protocols are also supported by RabbitMQ, such as MQTT, HTTP, and STOMP.

Now, it's time to turn the focus to RabbitMQ.

 

The RabbitMQ broker

RabbitMQ is an Erlang implementation of an AMQP broker. It implements Version 0-9-1 of AMQP with custom extensions, as allowed by the protocol. Erlang has been chosen because of its intrinsic support for building highly reliable and distributed applications. Indeed, Erlang is used to run telecommunication switches in several large telecommunication systems, and a total system's availability of nine nines has been reported (that's only 32 milliseconds of downtime per year). Erlang is able to run on any operating system.

For data persistence, RabbitMQ relies on Mnesia, the in-memory/file-persisted embedded database of Erlang. Mnesia stores information about users, exchanges, queues, bindings, and so on. The queue index stores message positions and information on whether a message has been delivered or not. Messages are stored either in the queue index or in the message store, a key-value store shared among all queues.

For clustering, it mainly relies on Erlang's ingrained clustering abilities. RabbitMQ can easily be extended with the addition of plugins. For example, a web-based administration console can be deployed on RabbitMQ thanks to this mechanism.

Plugins can be used to extend the core broker functionality. There are many plugins available for RabbitMQ, and it's also possible to develop plugins, if needed: https://www.rabbitmq.com/plugins.html.

RabbitMQ can be set up on a single, standalone instance, or as a cluster on multiple servers:

Fig 1.7: Standalone instance, or as a cluster on multiple servers

RabbitMQ brokers can be connected together using different techniques, such as federation and shovels, in order to form messaging topologies with smart message routing across brokers and the capacity to span multiple data centers.

The following screenshot shows federation between RabbitMQ brokers located in different places around the world:

Fig 1.8: The RabbitMQ broker engaging in various topologies
RabbitMQ supports AMQP 1.0 through plugins. 

AMQP 1.0 was published at the end of 2011 after the development and maintenance of AMQP was transferred to OASIS. AMQP has been drastically revised between 0-9-1 and 1.0. This was so drastic that some core concepts, such as the exchange, no longer exist. Thus, AMQP 1.0 is a different protocol than 0-9-1, but there is no truly compelling reason to adopt it. It is not more capable than 0-9-1, and some would also argue that it has lost some of the key aspects that made it attractive in the first place.

So, when or where is RabbitMQ used? The next section describes some common use cases for RabbitMQ.

 

Using RabbitMQ in real life

The most common use case for RabbitMQ is a single producer, single consumer queue. Think of it as a pipe where one application puts messages into one end of the pipe and another application reads the messages that come out the other end. Messages are delivered in first in, first out order. These messages may be commands or contain important data. This sounds easy, but where could this type of architecture be applied? It's time to understand when and why message queuing shines!

 

Message queues between microservices

Message queues are often used in between microservices, but what does that mean?

Microservice architectural style divides the application into small services, with the finished application being the sum of its microservices. The services are not strictly connected to each other. Instead, they use, for example, message queues to keep in touch. One service asynchronously pushes messages to a queue and those messages are delivered to the correct destination when the consumer is ready.

The microservice architecture is often compared and contrasted with the monolith architecture, where the entire system is bundled together into one piece of software. One application is not only responsible for a particular task; it actually performs every step needed to complete a particular function. Monoliths communicate within the system since all the parts are running in the same process. This system is highly coupled since every function is reliant on the others.

In an example of a webshop built on a monolith architecture style, one system handles all of the functions, including inventory, payments, reviews, and ratings and so on, as shown in the following diagram:

Fig 1.9: A webshop built in a monolith architecture style

A webshop built on the microservice architecture, on the other hand, means that each part of the system is an individual activity. One microservice handles reviews and ratings. Then, there's another inventory, and then yet another for payments, and so on, as shown in the following diagram:

Fig 1.10: A microservice architecture style where each part is focused on a single business capability

Each pair of requests and responses communicates independently. This is known as stateless communication. While many microservices are involved, they are not directly dependent on each other.

Another typical use case for RabbitMQ is as a task queue, which we'll cover in the next section.

 

Event and tasks

Events are notifications that tell applications when something has happened. One application can subscribe to events from another application and respond by creating and handling tasks for themselves. A typical use case is when RabbitMQ acts as a task queue that handles slow operations.

Let's take a look at two examples of this:

  • Imagine a social media application such as Instagram. Every time someone publishes a new post, the network (followers) needs to be informed about the new post. This could be a very time-consuming operation. Millions of people could be trying to perform the same task at the same time. The application can, with the use of message queues, enqueue a task onto the queue for each post as it arrives. When the worker receives the request, it retrieves a list of followers for the sender, and updates each of them.
  • As another example, think of an email newsletter campaign tool that is sending out thousands of emails to thousands of users. With a possible scenario where many users trigger bulk messages at the same time. The email newsletter campaign tool needs to be able to handle this volume of messages. All these emails can be added to a push queue with instructions to the worker regarding what to send and to whom. Every single email is handled, one by one, until all the emails have been sent.

The following diagram shows a task queue, where messages are first entering the queue, and then handled. New tasks are then added to another queue:

Fig 1.11: Event and task queue

With that, we've looked at and reviewed two typical use cases. The benefits of RabbitMQ have been apparent in each. We'll make this even more evident by exploring the benefits of message queuing in the next section.

 

Exploring the benefits of message queuing

Communication between various applications plays an important role in distributed systems. There are many examples of when a message queue can be used, so let's highlight some features and benefits of message queuing in microservice architectures:

  • Development and maintenance made easier: Dividing an application across multiple services allows separate responsibilities and gives developers the freedom to write code for a specific service in any chosen language. It will be easier to maintain written code and make changes to the system; when updating a single authentication scheme, only the authentication module must have code added for testing, without it disrupting any other functions.
  • Fault isolation: A fault can be isolated to a single module and will thereby not affect other services. For example, an application with a reporting service temporarily out of function will not affect the authenticate or payment services. As another example, making changes to the reporting service still allows customers to perform essential transactions, even when they aren't able to view reports.
  • Enhanced levels of speed and productivity: Different developers are able to work on different modules at the same time. In addition to speeding up the development cycle, the testing phase is also impacted by the use of microservices and message queues. This is because each service can be tested on its own to determine the readiness of the overall system.
  • Improved scalability: Microservices also allow for effortless scale-out at will. It's possible to add more consumers if the message queue is growing. Adding new components to just one service is easy to do without changing any other service.
  • Easy to understand: Since each module in a microservice architecture represents a single functionality, getting to know the relevant details for a task is easy. For example, hiring a consultant for a single service does not require them to understand the entire system.

Now that is enough knowledge to be dangerous, so it is a good time to dive into the RabbitMQ scenario company that sets the scene for the rest of this book.

 

A RabbitMQ scenario

CC is a new taxi agency with huge potential. Today, the company has just two taxi drivers and two developers, but they want to expand a lot in the upcoming year. CC has already built a website in Ruby and started out with a backend, also written in Ruby, that stores CC trips in a database. CC also has some scripts, written in Python, that generate route reports.

So far, CC's system runs as follows:

  • The company's website and blog runs on Ruby.
  • The Rich Internet Application that stores route data, such as the starting point and the endpoint of the trip, is written in Ruby.
  • There's a back-office that sends route updates to drivers and is written in Ruby.
  • Multiple ad hoc Python scripts are used to extract and message data to generate route reports.
  • Taxi applications are written in Python.

The old architecture is illustrated as follows:

Fig 1.12: CC software landscape

Why is CC looking at adding RabbitMQ to an already busy environment? The main reason is because of a new feature CC wants to offer to their customers – they want to build a taxi application that handles reservations on the go. CC also wants to be able to scale painlessly. The plan is to build an app where users can book a car via a smartphone, receive booking confirmation, and view the car approaching the starting point of the trip.

Since CC already has some services in different languages, and since CC wants to be able to scale easily, they decided to use a ready-made message-oriented middleware such as RabbitMQ for asynchronous communication between the app, the client, and the backend.

As CC's knowledge and usage of RabbitMQ increases, they will discover new opportunities to leverage it in the environment. For now, let's follow CC as it gets started with its very first step into working with RabbitMQ.

 

Getting ready for RabbitMQ

To get started, the following three installation and configuration steps need to be completed:

  • Installing the RabbitMQ broker
  • Installing the management plugin (Web UI)
  • Configuring the vhost and user

Let's start by installing the broker!

 

Installing the broker

CC runs its production servers on Ubuntu Linux. One developer has macOS and Linux, while the other one is all Windows. This heterogeneity is not a concern for RabbitMQ, which can run natively on all these operating systems.

RabbitMQ provides complete online installation guides for all the supported operating systems, and they can be found here: http://www.rabbitmq.com/download.html. This book contains instructions for Debian/Ubuntu, where RabbitMQ is installed from the apt repository. It also contains instructions for Docker further down in this chapter.

 

RabbitMQ installation on Ubuntu

There are relatively few steps required to install RabbitMQ. They are as follows:

  1. Update Ubuntu.
  2. Download and install the repository key.
  3. Ensure the key is in the repository.
  4. Install RabbitMQ from the package repository.

Ensure that Ubuntu is up to date before starting the download process. Make sure that the operating system is using the latest versions of all software since outdated dependencies create security vulnerabilities.

Run the apt update command to download the latest releases of the installed software:

apt upgrade

RabbitMQ requires several software packages. Verify that curl, apt-transport-https, and GnuPG are on the system by running the following command:

sudo apt install curl gnupg -y
sudo apt install apt-transport-https

The -y option accepts any licenses for these dependencies. Ubuntu installs all required sub-packages.

Discover the name of the operating system by running any of the following commands:

  • cat /etc/os-release
  • lsb_release -a
  • hostnamectl

The release name is non-technical. Previous names include focal and bionic. Ubuntu does not include RabbitMQ by default, so it must be added to the repository key before you proceed. Execute the following set of commands in a Terminal:

curl -fsSL https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc 
sudo apt-key add -
sudo tee /etc/apt/sources.list.d/bintray.rabbitmq.list <<EOF
deb https://dl.bintray.com/rabbitmq-erlang/debian [os release name] erlang
deb https://dl.bintray.com/rabbitmq/debian [os release name] main
EOF

These commands download the key and add it to the repository list before adding the appropriate operating system packages for the broker and Erlang.

RabbitMQ is written in Erlang, a functional language that has robust built-in support for creating distributed networks. The developers maintain a list of minimum versions (https://www.rabbitmq.com/which-erlang.html) of the language for the latest supported releases of the broker. At the time of writing, RabbitMQ 3.8 supports Erlang 21.3 through 23.

RabbitMQ can now be installed correctly.

Though not absolutely required for using RabbitMQ, it is encouraged to discover this powerful language and platform. You can learn more about Erlang at http://www.erlang.org/. Alternatively, you can consider Elixir as an optional language for the Erlang virtual machine (VM). You can find out more about this at http://elixir-lang.org.

Run the following commands to install RabbitMQ:

sudo apt install -y rabbitmq-server
sudo apt install librabbitmq-dev

The librabbitmq-dev library includes a client for interacting with the broker. However, the server may be the only requirement.

 

RabbitMQ installation on Docker

Docker containers allow the separation and control of resources without risking corrupting the operating system. Instructions for installing Docker are available from the official website: https://docs.docker.com/get-docker/. With Docker installed, pull the RabbitMQ image:

docker pull rabbitmq

Run the broker with reasonable defaults:

docker run -d --hostname my-rabbit --name my-rabbit -p 5672:5672 -p 15672:15672 -e RABBITMQ_ERLANG_COOKIE='cookie_for_clustering' -e RABBITMQ_DEFAULT_USER=user -e RABBITMQ_DEFAULT_PASS=password  --name some-rabbit rabbitmq:3-management

A Docker container needs to be created so that it's accessible from the localhost with the management console enabled. This will be discovered shortly.

 

Starting RabbitMQ

Installing the RabbitMQ server from the repository also installs a suite of command-line tools used to start the server for the first time. This is done by executing the following command:

rabbitmq-server start 

The server starts in the foreground. To run the broker as a service, use the following commands:

sudo systemctl enable rabbitmq-server
sudo systemctl start rabbitmq-server
sudo systemctl status rabbitmq-server

The systemctl command can also be used to manage services in Ubuntu. The output of the final command should show that the broker is running. Consult the RabbitMQ documentation (https://www.rabbitmq.com/troubleshooting.html) if not.

 

Downloading the example code

 

Verifying that the RabbitMQ broker is running

Now, verify that the RabbitMQ broker is actually working by using the status service command.

Write the following line in the Terminal:

$ sudo service rabbitmq-server status
rabbitmq-server.service - RabbitMQ broker
Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/rabbitmq-server.service.d
└─10-limits.conf, 90-env.conf
Active: active (running) since Mon 2019-04-29 13:28:43 UTC; 1h 43min ago
Process: 27474 ExecStop=/usr/lib/rabbitmq/bin/rabbitmqctl shutdown (code=exited, status=0/SUCCESS)
Main PID: 27583 (beam.smp)
Status: "Initialized"
Tasks: 87 (limit: 1121)
CGroup: /system.slice/rabbitmq-server.service
├─27583 /usr/lib/erlang/erts-10.2.2/bin/beam.smp -W w -A 64 -MBas ageffcbf -MHas ageffcbf -MBlmbcs 512 -MHlmbcs 512 -MMmcs 30 -P 1048576 -t 5000000
├─27698 /usr/lib/erlang/erts-10.2.2/bin/epmd -daemon
├─27854 erl_child_setup 1000000
├─27882 inet_gethost 4
└─27883 inet_gethost 4

Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: ## ##
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: ## ## RabbitMQ 3.7.14. Copyright (C) 2007-2019 Pivotal Software, Inc.
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: ########## Licensed under the MPL. See https://www.rabbitmq.com/
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: ###### ##
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: ########## Logs: /var/log/rabbitmq/[email protected]
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: /var/log/rabbitmq/[email protected]_upgrade.log
Apr 29 13:28:42 test-young-mouse-01 rabbitmq-server[27583]: Starting broker...
Apr 29 13:28:43 test-young-mouse-01 rabbitmq-server[27583]: systemd unit for activation check: "rabbitmq-server.service"
Apr 29 13:28:43 test-young-mouse-01 systemd[1]: Started RabbitMQ broker.
Apr 29 13:28:43 test-young-mouse-01 rabbitmq-server[27583]: completed with 9 plugins.
The default folders where the package has installed files are /etc/rabbitmq for configuration files, /usr/lib/rabbitmq for application files, and /var/lib/rabbitmq for data files (mnesia).

Look at the running processes for RabbitMQ and find both the service wrapper and the Erlang VM (also known as BEAM) that's running, as follows:

$ pgrep -fl rabbitmq
27583 beam.smp

$ ps aux | grep rabbitmq
ubuntu 10260 0.0 0.1 14856 1004 pts/0 S+ 15:13 0:00 grep --color=auto rabbitmq
rabbitmq 27583 0.5 8.5 2186988 83484 ? Ssl 13:28 0:36 /usr/lib/erlang/erts-10.2.2/bin/beam.smp -W w -A 64 -MBas ageffcbf -MHas ageffcbf -MBlmbcs 512 -MHlmbcs 512 -MMmcs 30 -P 1048576 -t 5000000 -stbt db -zdbbl 128000 -K true -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/librabbitmq/lib/rabbitmq_server-3.7.14/ebin -noshell -noinput -s rabbit boot -sname [email protected] -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit lager_log_root "/var/log/rabbitmq" -rabbit lager_default_file "/var/log/rabbitmq/[email protected]" -rabbit lager_upgrade_file "/var/log/rabbitmq/[email protected]_upgrade.log" -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/plugins:/usr/lib/rabbitmq/lib/rabbitmq_server-3.7.14/plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/[email protected]" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/[email protected]" -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672
rabbitmq 27698 0.0 0.1 8532 1528 ? S 13:28 0:00 /usr/lib/erlang/erts-10.2.2/bin/epmd -daemon
rabbitmq 27854 0.0 0.1 4520 1576 ? Ss 13:28 0:00 erl_child_setup 1000000
rabbitmq 27882 0.0 0.1 8264 1076 ? Ss 13:28 0:00 inet_gethost 4
rabbitmq 27883 0.0 0.1 14616 1808 ? S 13:28 0:00 inet_gethost 4

It is possible that, when RabbitMQ runs, a process named epmd is also running. This is the Erlang port mapper daemon, which is in charge of coordinating Erlang nodes in a cluster. It is expected to start even if the clustered RabbitMQ application is not running.

Note that by default, the broker service is configured to auto-start when the Linux host starts.

Skip the hassle of the installation and configuration of RabbitMQ and use a hosted RabbitMQ solution. CloudAMQP is the largest provider of hosted RabbitMQ clusters: www.cloudamqp.com.

 

Installing the management plugin (Web UI)

RabbitMQ does not install a management console by default, but the optional web-based plugin used in this example makes it easy to peek into a running RabbitMQ instance.

The Debian package installs several scripts. One of them is rabbitmq-plugins. Its purpose is to allow us to install and remove plugins. Use it to install the management plugin, as follows:

$ sudo rabbitmq-plugins enable rabbitmq_management 
Enabling plugins on node [email protected]:
rabbitmq_management
The following plugins have been configured:
rabbitmq_consistent_hash_exchange
rabbitmq_event_exchange
rabbitmq_federation
rabbitmq_management
rabbitmq_management_agent
rabbitmq_shovel
rabbitmq_web_dispatch
Applying plugin configuration to [email protected]
The following plugins have been enabled:
rabbitmq_management
rabbitmq_management_agent
rabbitmq_web_dispatch

Yes, it is that easy!

Use a web browser to reach the home page of the management console by navigating to http://<hostname>:15672, as shown in the following screenshot:

Fig 1.13: The login screen of the management console

Stay tuned for the next episode creating and configuring users!

 

Configuring users

One of the scripts that's installed by the Debian package is rabbitmqctl, which is a tool for managing RabbitMQ nodes and used to configure all aspects of the broker. Use it to configure an administration user in the broker, as follows:

$ sudo rabbitmqctl add_user cc-admin taxi123
Adding user "cc-admin" ... $ sudo rabbitmqctl set_user_tags cc-admin administrator
Setting tags for user "cc-admin" to [administrator] ...

By default, RabbitMQ comes with a guest user that's authenticated with the guest password. Change this password to something else, as follows:

$ sudo rabbitmqctl change_password guest guest123

Navigating back to the management console login screen allows us to log in with the username cc-admin and the password taxi123.

The welcome screen provides an overview of the broker's internals, as shown in the following screenshot:

Fig 1.14: The main dashboard of the management console

Note that at this point, the cc-admin user is not able to examine any exchange or queue in any vhost. For now, another user must be created for development purposes so that applications can connect to RabbitMQ.

Create the cc-dev user, as follows:

$ sudo rabbitmqctl add_user cc-dev taxi123
Adding user "cc-dev" ...

As discussed earlier in this chapter, RabbitMQ supports the notion of vhosts, which is where different users can have different access privileges. The CC development environment will have a vhost, also known as vhost. Anything that happens in the vhost happens in isolation from any other environment created in the future (such as a QA environment). It is possible to set per-vhost limits on a number of queues and concurrent client connections in later versions of RabbitMQ (3.7+).

Create a vhost called cc-dev-vhost, as follows:

$ sudo rabbitmqctl add_vhost cc-dev-vhost
Adding vhost "cc-dev-vhost" ...

This creates a user and a vhost for development.

 

Configuring dedicated vhosts

RabbitMQ comes with a default vhost called / that the guest user has full permissions for. Though this is convenient for quick tests, it is recommended that a dedicated vhost is created to keep concerns separated so that it is possible to completely drop a vhost and restart from scratch without unexpected impacts.

As it stands, neither the cc-admin nor cc-dev users have permission to do anything on cc-dev-vhost. You can fix this by giving the vhost full rights, as follows:

$ sudo rabbitmqctl set_permissions -p cc-dev-vhost cc-admin ".*" ".*" ".*"
Setting permissions for user "cc-admin" in vhost "cc-dev-vhost" ... $ sudo rabbitmqctl set_permissions -p cc-dev-vhost cc-dev ".*" ".*" ".*"
Setting permissions for user "cc-dev" in vhost "cc-dev-vhost" ...

To recap what was just done, most of the command is straightforward but the ".*" ".*" ".*" part looks a tad mysterious, so let's analyze it.

It is a triplet of permissions for the considered vhost, which grants configure, write, and read permissions on the designated resources for the considered user and vhost. Resources, which consist of exchanges and queues, are designated by regular expressions that match their names. In this case, any resource that's requested via the .* regular expression is allowed.

The actual commands that are granted depend on the resource type and the granted permissions. For a complete list of the access control policies supported by RabbitMQ, see http://www.rabbitmq.com/access-control.html.

As an alternative to all command lines, turn to the user management features of the management console. Click on the Admin tab of the console and then on the cc-dev user listed in the Users tab to view information similar to what's shown in the following screenshot. The entire user configuration that was set from the command line is visible and can be edited in the management console:

Fig 1.15: User management from the RabbitMQ management console

The details of an individual user can be found by clicking on a given user's name in the management console:

Fig 1.16: Details of an individual user in the management console

The RabbitMQ broker and the management plugin (Web UI) have been installed and the vhost and the users have been configured.

 

Summary

This chapter explored the architectural and design promises of messaging, including how AMQP and RabbitMQ deliver on these promises. In addition, the reason why the taxi agency Complete Car decided to introduce RabbitMQ in its software landscape was discovered. Finally, a RabbitMQ broker was installed and a user and various vhosts were configured for it. Armed with a basic understanding of message queues and RabbitMQ, the next chapter builds on these concepts and explores the architecture behind the Complete Car taxi application.

It's time to hit the ground running and write some code. Turn to the next chapter to start building a RabbitMQ-powered application!

About the Authors

  • Lovisa Johansson

    Lovisa Johansson has been working daily with RabbitMQ for many years. Through CloudAMQP, the company 84codes offers managed RabbitMQ clusters. With 50,000+ running instances, they are considered to be the largest provider of RabbitMQ as a Service in the world. Lovisa is an experienced developer with a Master's degree in Computer Science and Engineering. Through her work, she continues to write the most popular and widespread educational content about RabbitMQ, and occasionally shares this knowledge as a speaker at various conferences.

    Browse publications by this author
  • David Dossot

    David Dossot has worked as a software engineer and architect for more than 18 years. He has been using RabbitMQ since 2009 in a variety of different contexts. He is the main contributor to the AMQP transport for Mule. His focus is on building distributed and scalable server-side applications for the JVM and the Erlang VM. He is a member of IEEE, the Computer Society, and AOPA, and holds a diploma in Production Systems Engineering from ESSTIN. He is a Mule champion and a DZone Most Valuable Blogger. He commits on multiple open source projects and likes to help people on Stack Overflow. He's also a judge for the annual Jolt Awards software competition.

    Browse publications by this author