Amazon Simple Queue Service (SQS)

Before we look at Simple Queue Service (SQS), let's look at what a message queue is. A message queue is a queue of messages exchanged between applications. Messages are data objects that are inserted in the queue by sender applications and received by receiving applications. Receiving applications get the data objects from the queue and process the data received from the queue based on the application requirements. The following diagram describes the message queue in a simple way:

Message queue

Amazon SQS is a highly reliable, scalable, and distributed message queuing service provided by Amazon. It's a hosted solution provided by Amazon, so you do not need to manage the service infrastructure. Amazon SQS stores the messages in transit as they travel between various applications and micro services. Amazon provides a host of web service...

Why use SQS?

There are many reasons to use SQS. Let's start by looking at a simple use case followed by a brief review of a few more use cases.

Consider a scenario where you have a collaborative news site or application that accepts images from users, optimizes these images to display in multiple devices, and stores them for future retrieval. If the application users are spread across the globe, the application may get a huge number of visitors that upload images. If these images are directly offloaded to the processing server, the server may not be able to handle the traffic.

If you use a scaled-up environment to process the images, you end up incurring more costs. Also, a scaled-up environment may miss processing some of the images if the process gets interrupted or crashes.

In these scenarios, a message queue comes in handy. The application can send all the image processing...

How do queues work?

As shown in the following diagram, SQS works on the put-get-delete paradigm:

Working of SQS

The put-get-delete paradigm works in the following way:

1. The Publishing Application pushes a message, M, into the queue.
2. The Consuming Application pulls the message, M, from the queue and processes it.
3. The Consuming Application confirms to the SQS queue that processing on the message is completed and deletes this message from the SQS queue.

Main features of SQS

SQS provides a scalable and reliable messaging platform. It enables you to build operational efficiency in your application without any operational overhead. Here are some of the benefits of using SQS:

• Redundant infrastructure: Amazon SQS provides redundant infrastructure. With its redundant infrastructure, it ensures that a message is delivered at least once in a standard queue and it ensures that a message is delivered exactly once in a First In First Out (FIFO) queue. It provides a concurrent access mechanism and a highly available environment for queue producer and consumer applications.
• Multiple producers and consumers: Multiple components of a distributed application can concurrently send and receive messages at the same time. When a client picks up a message from the queue, SQS locks up that message until the client confirms that it has completed processing...

Queue attributes

SQS uses certain queue attributes that define the behavior of a queue. While creating a queue, you can either create a queue with default attributes or customize these attributes as per your needs.

The following table describes these queue attributes with their acceptable ranges:

Queue attribute	Description	Minimum acceptable range	Maximum acceptable range
Default visibility timeout	The length of time that a message is received from a queue will be invisible to other receiving components.	0 seconds	12 hours
Message retention period	The amount of time that SQS retains a message if it does not get deleted.	1 minute	14 days
Maximum message size	The maximum message size accepted by SQS.	1 KB	256 KB
Delivery delay	The amount of time to delay the first delivery of all messages added to the queue.	0 seconds	15 minutes...

Operations in a queue

There are various operations that we can perform in the queue. The various operations that we will be performing on a queue are as follows:

• Creating a queue
• Sending a message in a queue
• Viewing/deleting a message from a queue
• Purging a queue
• Deleting a queue
• Subscribing a queue to a topic
• Adding user permissions to a queue

Creating a queue

Since you have developed a basic understanding of what SQS is, let's go through the following steps to create and configure our first SQS queue:

1. Log into the AWS Management Console and navigate to https://console.aws.amazon.com/sqs/.
2. Click the Create New Queue button, as shown:

Creating a new queue

3. On the subsequent page, type the name of the queue as shown...

SQS limits

This topic helps us understand the limits with respect to the following:

• Limits related to queues
• Limits related to messages
• Limits related to policies

The following table describes these limits:

Limit	Applies to	Description
Inflight messages per queue	Queues	Standard queue: maximum of 120,000 inflight messages. FIFO queue: maximum of 20,000 inflight messages. SQS returns an OverLimit error if this limit is exceeded.
Queue name	Queues	Queue names can have a maximum of 80 characters. FIFO queue names must end with .fifo. Queue names can have alphanumeric characters, hyphens, and underscores.
Message attributes	Message	A message can have a maximum of 10 metadata attributes.
Message batch	Message	A message batch can have a maximum of 10 messages per batch.
Message content	Message	Message content can include only XML, JSON...

Queue monitoring and logging

Monitoring SQS queues plays a vital role in an application's life cycle. There are many functions that are dependent on monitoring, using CloudWatch and relevant triggers. For example, you monitor the size of a queue and define a trigger to automatically scale up EC2 instances with an Auto Scaling group. Similarly, you can scale the number of instances serving the consumer process if the size of a queue is smaller. Considering the criticality of monitoring with SQS, CloudWatch and SQS are integrated so that you can easily view and analyze various CloudWatch metrics for SQS queues. Queue metrics can be viewed and analyzed using an SQS console, CloudWatch console, programmatically using APIs, or using the CLI.

Amazon automatically gathers CloudWatch metrics for an SQS queue and pushes it to CloudWatch with an interval of five minutes. Amazon gathers...

SQS security

Amazon's SQS is built securely. It requires credentials to initiate any request to SQS queues. Even if you supply credentials while initiating a request to a queue, you may not be able to access it unless you have sufficient permission to access the queues and messages. In this section, let's look at authentication and access control related to SQS queues.

Authentication

AWS allows you to access SQS with any of the following identities:

• Root user
• IAM user
• IAM role
• Federated access
• Cross-account access
• AWS service access
• EC2 applications

Server-Side Encryption (SSE)

...

Summary

• A message queue is a queue of messages exchanged between applications.
• Messages are data objects that are inserted in the queue by sender applications and received by the receiving applications.
• SQS can save an application from getting flooded with messages by storing them into a queue.
• Applications can process the messages without getting interrupted.
• SQS can be used for decoupling application processes.
• SQS provides scalability to an application environment.
• The use of a message queue ensures that a message is delivered at least once, and it is eventually processed by the processing application as long as the process continues reading the queue.
• SQS provides a mechanism to process the data in a predefined order.
• SQS allows you to process the messages asynchronously.
• SQS builds resilience in the system.
• SQS brings elasticity to applications.
• SQS can easily help identify...