Scalability is one of the major problems of our time, and messaging is an integral part of the solution. It finally comes down to the message broker software to manage and control messaging between applications, processes, and threads. Message brokers can help to solve scalability issues and architectural issues, such as coupling.
RabbitMQ is one of the most powerful open source message broker software that is widely used in the tech companies such as Mozilla, VMware, Google, AT&T, and so on. RabbitMQ is a highly configurable messaging platform developed and supported by a knowledgeable and committed community.
Before diving into the details and technologies behind the RabbitMQ, let's introduce you to the topics that we will cover in this chapter:
A brief introduction to message brokers and the message queue
An introduction to advanced message queue protocol
Getting started with RabbitMQ
Installing RabbitMQ
Starting RabbitMQ
Summary
Recently, software systems evolved dramatically. Applications have to communicate with other applications, these applications can be internal and external to the application itself. For the same application, we may have different type of clients, such as browsers, mobile clients, and so on. Hence, we absolutely need a communication layer between internal applications and between applications and clients. We need to deliver different messages to different applications or clients. Delivering messages can be a bottleneck if the communication layer isn't scalable. Pursuing scalable systems for communication layer leads us to Message Brokers and Message Queues. Let's now discuss what Message Brokers and Message Queues are.
A Message Broker is an architectural pattern that can receive messages from multiple destinations, determine the correct destination, and route the message along the correct route, as stated in the book Enterprise Integration Patterns by Hohpe and Woolf. Message brokers enable systems to deal with messaging and routing by mediating communication among components. Once applications implement a message broker pattern, it decreases the coupling between application components.
Message Brokers are centralized, in the architectural sense, to control and manage all messages. Therefore, all of the incoming and outgoing messages are sent through Message Brokers, which analyze and deliver the messages to their correct destination. This procedural step can be understood with the following diagram:
Message Broker
Message Brokers address the following concerns in the communication layer:
Transforming messages to alternative formats
Routing messages to destinations
Supporting different types of patterns to send messages
Receiving and responding to events
Performing message aggregation
Persisting the message states
Ensuring the receiving and sending of message
Decoupling the destination software systems
Many tasks of the Message Broker need a Message Queue for exchanging or passing data to the destination. The next section covers Message Queues. We will talk about the mechanism behind Message Brokers in Chapter 3, Architecture and Messaging.
A Message Queue is, briefly, a queue for messaging. Queue is the basic data structure behind the functioning of a Message Queue. Message Queue operations are similar to Queue data structure operations, such as the enqueue and dequeu operations. An enqueue operation leads to adding an element to the back of the queue. A dequeue operation leads to the deletion of an element from the front of the queue.
Message Queues provide concurrent and asynchronous operations to scale applications. In a message queue, messages wait up until a message is retrieved by an application. Let's take a look at the following diagram:
Message Queue
Different types of standards and protocols define the Message Queuing specifications. Some protocols are open to everyone; however, some protocols are closed. Let's come back to our topic. RabbitMQ uses Advanced Message Queuing Protocol (AMQP) that determines the policies of the Message Queues. The next topic will cover detailed information on AMQP. Chapter 3, Architecture and Messaging, covers the detailed explanation of Message Queues.
John O'Hara from J. P. Morgan started AMQP in 2003. He put incredible amount of work into it. Then, J. P. Morgan approached other firms to establish an organization for creating open standards in messaging. According to AMOP's official website (http://www.amqp.org), AMQP is an open standard for passing messages between applications or organizations. So, AMQP just defines the messaging properties, queue properties, how messages are routed between applications and clients, how Message Brokers ensure that the message is received or sent, and other concerns such as reliability and security.
According to the AMQP website (http://www.amqp.org), AMQP has lots of capabilities to accomplish goals:
Security
Reliability
Interoperability
Standard
Open standard
Interoperability and reliability are very important for today's software engineering problems. The power of AMQP comes from its features like interoperability, reliability and so forth. Especially, with interoperability, we can use different types of technologies in sender and receiver. The main problem for most of the Internet giants is scalability. Scalability has direct relationship to reliability. Chapter 3, Architecture and Messaging, covers the details and specifications of AMQP.
As mentioned earlier, RabbitMQ is an open source Message Broker software that tries to solve messaging problems by implementing the AMQP. RabbitMQ is licensed with Mozilla Public License. RabbitMQ became part of GoPivotal in May 2013 and the community has helped in the development of RabbitMQ. Since then, the community has been trying to improve RabbitMQ.
RabbitMQ logo
As we explained, RabbitMQ solves messaging problems; however, you may ask yourself what kind of messaging problems RabbitMQ solves. In summary, RabbitMQ has the following functionalities to solve messaging problems:
Ensures that messages are sent and received
Routes the messages to the correct destinations
Saves the state of the messages
Supports multiple transportation protocols (AMQP, MQTT, STOMP, HTTP)
Highly scalable, reliable and available
Extendible with plugins
Supports clients for almost any of language
Large community support also provides commercial support
Now that we have discussed the concepts behind the RabbitMQ, let's try to install and use RabbitMQ on your computer.
Installation of RabbitMQ is not distinctly different from other software in different operation systems. Unix-based operating systems can build RabbitMQ from source code and Microsoft Windows can run the standard MSI installers. RabbitMQ installation files can be found in the download webpage of the RabbitMQ website, as shown in the following image:
Download Webpage of RabbitMQ
The only prerequisite for the RabbitMQ installation is the Erlang runtime environment because RabbitMQ runs on the Erlang VM. Therefore, we have to install Erlang before installing the RabbitMQ. Erlang can be downloaded from the Erlang download webpage, as shown in the following image, and installation instructions will be covered in the topics that follow:
Download Webpage of Erlang
Now, let's to talk about the installation of both RabbitMQ and Erlang on Windows, Mac OS X, Ubuntu, Fedora, and Amazon Web Services.
RabbitMQ runs on both 32-bit and 64-bit machines from the same package. Erlang is installed either as 32-bit and 64-bit. So, RabbitMQ can be easily installed on the Windows operating system. Let's install these stuff for running RabbitMQ.
Firstly, we should install the Erlang runtime environment on Windows. Erlang has Windows installers for 32-bit and 64-bit as shown in the previous image. We can easily download the related binary file to our computer and install Erlang using it:
Installation of Erlang in Windows
After installing the Erlang runtime environment, we've completed the requirements of RabbitMQ installation. The next step is to download and install the RabbitMQ binary file with related the Windows version:
Installation of Erlang in Windows
We can find the related Windows installer for RabbitMQ with the help of RabbitMQ download webpage as shown in the screenshot showing the download webpage of RabbitMQ. Then, we just need to click and install the RabbitMQ on our Windows computer. Besides installing using the installer, we can install using the Windows binary file that is served within the RabbitMQ download webpage. The following instructions will be enough for installing RabbitMQ without the installer:
Download the binary file for RabbitMQ Windows binary files
Extract the downloaded RabbitMQ zip file to our local folder
It is possible to install the RabbitMQ on your Windows computer in both ways. Note that you may add the RabbitMQ binaries directory to the windows system path in the system/environment variable settings.
As we specified, we only have one requirement to install RabbitMQ on our computers. In Mac OS X, we have package managers and we have the opportunity to compile from the source for both Erlang runtime environment and RabbitMQ.
Firstly, both Erlang and RabbitMQ can be easily installed on Mac OS X using package managers. Although we have lots of package managers on Mac OS X, Homebrew and MacPort are the ones that are mostly used in Mac OS X. So, we'll talk about the installation using Homebrew and MacPorts.
Homebrew has both RabbitMQ and Erlang on its repository. As RabbitMQ has a dependency with Erlang, Homebrew finds its dependent software and installs them together.
Note
Homebrew is just another package manager for Mac OS X. Homebrew is quite easy to install on Mac OS X and has lots of packages. So, you would find your application in its repository. Check it out at http://brew.sh
So, we just need to install RabbitMQ in Homebrew using the brew install rabbitmq command on our terminal as shown in the following image:
Homebrew Installation of RabbitMQ
MacPorts has the similar method of operation with Homebrew. MacPorts also installs the software with its dependencies. Therefore, we just need to install RabbitMQ in MacPorts using the port install rabbitmq-server command on our terminal, as shown in the following image:
MacPorts Installation of RabbitMQ
Another way to install RabbitMQ and Erlang is by compiling the source codes in Mac OS X. Before compiling, we need the following to compile Erlang source code:
The GNU make
The GNU C compiler
Perl 5
After downloading and unzipping the source codes of the Erlang, we just need common commands on the Erlang folder for compiling from source code, as follows:
./configure make make install
Finally, we just need to download and unzip the RabbitMQ binary files.
Ubuntu is just another Linux distribution based on Debian. Similar instructions as the ones we discussed for the installation on the Mac OS X would be applied for Ubuntu.
Ubuntu has a package manager called Advanced Packaging Tool (apt-get) and has a Debian package manager called dpkg. So, we are able to install RabbitMQ and Erlang runtime environment using apt-get. Moreover, similar to Mac OS X, we can compile from source codes of Erlang.
Firstly, as we said in the previous paragraph, we can install RabbitMQ using apt-get and dpkg. Before installing RabbitMQ, we should add the RabbitMQ repository to the APT repository using the following line (add the following line to /etc/apt/sources.list):
deb http://www.rabbitmq.com/debian/ testing main
Now, we are ready to install RabbitMQ and its dependency Erlang runtime environment, as shown in the following image:
sudo apt-get install rabbitmq-server
Ubuntu Installation of RabbitMQ
Also, we have the dpkg dependency management tool for installing RabbitMQ. RabbitMQ has packages for dpkg in its download webpage. We can download it from its website then run the following command:
dpkg –i rabbitmq-server.deb
Secondly, we have another option, which was explained in the Mac OS X topic. That is, compiling from source codes. We just need to compile the downloaded Erlang source code, and we are ready to run the downloaded binary files of RabbitMQ. You can look at the details in the Mac OS X section.
Fedora is yet another Linux distribution based on Red Hat. Installation instructions of Fedora are similar to Ubuntu's installation. Fedora has package managers called rpm and yum.
Firstly, we are able to install RabbitMQ with its dependency Erlang using package managers. Before using yum, we should run the following command to add RabbitMQ repository:
wget -O /etc/yum.repos.d/epel-erlang.repo http://repos.fedorapeople.org/repos/peter/erlang/epel-erlang.repo
Then, we can install RabbitMQ using the following command as shown in the following screenshot:
sudo yum install rabbitmq-server
Fedora Installation of RabbitMQ
Fedora has another package manager, which comes from Red Hat, called rpm. As RabbitMQ publishes package as rpm, we can easily install using the rpm package manager. After downloading the rpm package from RabbitMQ webpage, we can install RabbitMQ with Erlang, using following command:
rpm –ivh rabbitmq-server.rpm
Secondly, we have another option, which was explained in the Mac OS X topic. That is compiling from source code. We just need to compile the downloaded Erlang source code and we are ready to run the downloaded binary files of RabbitMQ. You can look at the details in the Mac OS X section.
Amazon Web Services (AWS) is cloud-computing platform offered from Amazon. AWS has lots of features for developers such as autoscaling. Besides RabbitMQ, AWS also offers their own messaging service called Simple Queue Service: however, RabbitMQ has lots of advantages over Amazon SQS. For instance, RabbitMQ has an extendable plugin system, whereas SQS capabilities are short. RabbitMQ implements a standard approach to message acknowledgement and consumption, whereas SQS has its own standards.
Anyway, we can easily install our RabbitMQ to the AWS EC2 instance and can save images of the RabbitMQ installed operating system. In AWS EC2, we choose one of the operating systems from the list or any other instance that we used earlier.
Tip
Amazon EC2 would be a good choice for your servers. In a scalable architecture, we need clusters of Message Brokers, databases, caches, and so on. EC2 gives you great API, and it's really quite easy to create clusters using EC2. We will talk about the clusters of RabbitMQ using EC2 in Chapter 4, Clustering and High Availability.
As we explained, we are able to install RabbitMQ on Windows, Linux, and Mac OS X, we just need to follow the instructions for AWS EC2 that are explained well in the preceding sections. Let's take a look at the following screenshot:
Amazon EC2
As we can see, the installation part of RabbitMQ is quite easy and starting RabbitMQ is similar to its installation. Some package managers in Linux, Mac OS X, and Windows installer add configuration parameters to operation system's configuration for automatic startup. In such a case, we don't need to run the RabbitMQ command manually; however, if we install RabbitMQ manually, we need to run the RabbitMQ commands manually.
If we use the Windows installer of RabbitMQ, the installer already makes configurations for starting automatically. Therefore, we don't need to run RabbitMQ manually; however, whenever we'd like to control the status of the server, we just need to run following command on the sbin folder of RabbitMQ:
rabbitmqctl status
Status of RabbitMQ in Windows
You may have installed RabbitMQ manually on your Windows and you might wonder how you can run the RabbitMQ server. You should run the following command to start RabbitMQ (you have to run this command with an administrative user). Moreover, you can install the RabbitMQ server as a Windows service:
rabbitmq-server
There's isn't much difference in running RabbitMQ on Windows and other operating systems. If we have RabbitMQ already installed using package managers, such as apt-get, yum, and so on, we don't need to run the RabbitMQ manually because RabbitMQ has already started automatically. So, we'd like to check the status of the RabbitMQ using the following command:
sudo rabbitmqctl status
Status of RabbitMQ in Unix
After controlling the RabbitMQ status, if we get a message that says that RabbitMQ isn't running, then we should run RabbitMQ using the following command on the sbin folder of RabbitMQ installation folder:
rabbitmq-server
Starting of RabbitMQ in Unix
We finally finished our first chapter, which introduces messaging concepts along with brief details about Message Queues and Message Brokers, and the protocol called AMQP that defines the functionalities of a Message Queue. Finally, our chapter introduces the RabbitMQ, providing information about its installation on different types of operating systems and how we can run the RabbitMQ server. Now, we are ready to jump into the details of RabbitMQ, starting from its configuration in the next chapter.
