Chapter 1. Taking Your First Steps with DynamoDB

In this chapter, we will cover the following topics:

Signing up to the DynamoDB console
Creating the DynamoDB table using the console
Loading data into the table using the console
Querying data using the DynamoDB console
Deleting the DynamoDB table using the console
Analyzing DynamoDB metric on CloudWatch
Downloading and setting up DynamoDB Local
Using DynamoDB Local JavaScript Shell
Setting up AWS Command Line Interface for DynamoDB
Setting up the Eclipse IDE

Introduction

Amazon DynamoDB is a fully managed and cloud-hosted NoSQL database. NoSQL databases allow us to have a schema-less design for the tables. We only need to define the key, and the rest of the attributes can be defined at the time of record insertion itself.

DynamoDB provides a fast and predictable performance with the ability to scale seamlessly. It allows you to store and retrieve any amount of data, serving any level of network traffic without having any operational burden. DynamoDB gives numerous other advantages, such as a consistent and predictable performance, flexible data modelling, and durability. This chapter is all about taking your first steps with DynamoDB, getting your workspace ready, and seeing how to perform DynamoDB operations using the DynamoDB console. Now let's get started with the sign up process.

Signing up to the DynamoDB console

To get started with DynamoDB, we need to first create an account with Amazon Web Services.

Getting ready

AWS provides an interactive console to perform the Sign Up process. Being a pay-as-you-go model, we need to provide valid credit card details while registering. AWS also needs a valid phone, which is verified at the time of registration itself. So, you need to keep both the things handy.

How to do it…

Let's get started with DynamoDB:

To sign in to the AWS Console, go to http://aws.amazon.com/.
Click on the Sign in to the Console button:
To begin with the sign up process, follow the onscreen instructions by providing the valid e-mail address.
Enter all the details asked correctly, and click on the Press Create Account button:
In the next screen, you would need to provide the contact information, and later, capture the billing information by providing the credit card information.
The last step would be to provide your phone number. AWS calls the provided phone number, and you are supposed to enter the on screen PIN number on your phone dial pad. Once you are done, you will see the AWS Management console.
Now, you can select the DynamoDB service under the Database section to go to the DynamoDB console:
That's it! You are all set to use DynamoDB.

How it works…

AWS manages the accounts created through its console, and allows the user to access it over defined authentication methods. For any new account, AWS provides the free tier up to a certain threshold. You can read more about this at http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-free-tier.html.

There's more…

Note that on registration, AWS charges you 1 USD for the verification of your credit card, but the payment gets reversed after some days. You may take a look at the AWS Billing FAQs for more details at http://aws.amazon.com/billing/faqs/.

Creating the DynamoDB table using the console

Now that we have signed up for AWS, let's start by creating our first DynamoDB table using the DynamoDB console.

Getting ready

To perform this recipe, you need to have completed the earlier recipe of Signing up to the DynamoDB AWS console.

How to do it…

Let's create our first DynamoDB table. The steps are as follows:

Sign in to the AWS Console by providing valid account details, and go to the DynamoDB console, as shown in the previous recipe. You will see the following screen:
Click on the Create Table button. This will open a popup window where you will need to provide details, such as the Table Name, Primary Key, their data types, and so on. Here, we need to create a table called a product table, which will save the products-related data for all the e-commerce websites.
Here, we will create a table with the Primary Key Type as the Hash and Range composite keys. We will have the hash key as the product id and range key as the product type. Add the details, as shown in the following screenshot, and click on Continue:
In the next screen, you need to add Global/Local Secondary Indexes. Here, we would like to query DynamoDB items with their name and manufacturer. So, we create an index, and click on the Add Index to Table button, as shown in the following screenshot. Once done, click on Continue to proceed with the next configuration:
Here, we are also selecting Projected Attributes as All Attributes, as we would like DynamoDB to give us all the attributes back when we query it. Alternatively, if you know exactly what attributes you need to be projected into this secondary index, you can select Specify Attributes, in the Projected Attributes dropdown, and add the specific attributes you want to be projected.
Provide Read Capacity Units and Write Capacity Units, depending on the reads and writes load you are expecting. Here, I am providing the value 5 for Read Capacity Units and 5 for Write Capacity Units, as shown in the following screenshot. Click on the Continue button to proceed. A read capacity unit is a strong consistent read per second for items as large as 4 KB, while a write capacity unit is a strong consistent write per second for items as large as 1 KB:
The next screen will be all about setting up the throughput alarms. Here, you can select the checkbox for when you wish to get alarms in case your throughput reaches capacity. This is an optional configuration, so you may wish to skip it. Click on Continue to move to the next step.
The last step is to review the summary. If you are okay with the configurations, then click on Create to actually create the DynamoDB table. If you think that there are some configurations that are missing, then you may wish to go back and change those.
The newly created DynamoDB table may take some time to get active; in the meantime, you will see Status as Creating on the screen.
Lastly, you will see that the table is created and is in the ACTIVE state:

How it works…

In DynamoDB, we don't need to bother about the availability, scalability, and reliability of the table as it is completely owned by Amazon. We just need to create a table and start using it. We can modify the throughput units any time using the same console. This gives us the flexibility to control the reads and writes of the data from DynamoDB. For example, if you think that there is a certain period where you would be expecting more reads/writes, then you can increase the throughput values and vice versa. You can find out how AWS calculates the read and write throughput at

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ProvisionedThroughputIntro.html.

Choosing a right key is very important in order to balance the load on DynamoDB partitions. The hash key should be chosen in such a way that it has unique values in it so that the data can be distributed across the cluster. The range key should be chosen in such a way that items can be grouped together so that if you have any such queries, they will perform faster. This is why we have chosen the product id as the Hash Key attribute, while the product type as the Range Key attribute.

The same is the case with Global Secondary Indexes. The Global secondary indexes allow us to query items on non-primary key attributes. Here, we may want to query items on their names or group them by the manufacturer.

DynamoDB supports various data types for its attributes, which are as follows:

Scalar: Number, String, Binary, Boolean, and Null
Multi-valued: String Set, Number Set, and Binary Set
Document: List and Map

You can choose which suites you best.

There's more…

Here, you might have noticed that instead of putting the manufacturer's name as a proper name to the range key in the Global Secondary Index, I chose to use its short form as mnfr. The reason for this is that when it comes to key-value pair types of databases, the entries stored in the database are always both keys and values together. So, every time you add an item or an attribute, it will be saved as key=value in the database. So, it's the best practice to use short yet meaningful names for keys and attributes. We will discuss such best practices in Chapter 7, DynamoDB Best Practices.

Loading data into the table using the console

We were able to create a table in DynamoDB in the previous recipe; now, we will load the data into that table using the console.

Getting ready

To perform this recipe, you need to have completed the earlier recipe of Creating the DynamoDB table using the console.

How to do it…

Let's load the data into the table:

Select the table in which you wish to load the data into the console, and click on the Explore Table button.
As we have not added any items to the table yet, it will show No Items Found on your screen, as shown in the following screenshot:
Click on the Create Item button to add a new item to the table. By default, it opens the Put Item screen in the Tree mode, where you can add values and the keys. You can also add more attributes to the item based on the requirements, as shown in the following screenshot:
Click on the Save button to add an item to the table.
Repeat the earlier mentioned steps to add more items to the table.

How it works…

This simply adds the items against the DynamoDB table that we created. DynamoDB supports various data types, such as String, Number, Binary, String Set, Number Set, Map, List, Boolean, and Null. You can choose the correct data type that is suitable for the operations you would need to perform.

You can refer to the documentation for more details at http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DataModel.html#DataModel.DataTypes.

Querying data using the DynamoDB console

In the earlier recipe, we learned how to add new items to the DynamoDB table. In this recipe, we will learn how to query the data that is added using the DynamoDB console.

Getting ready

To perform this recipe, you need to have completed the earlier recipe to add items to the DynamoDB table.

How to do it…

Let's perform different types of queries on AWS DynamoDB console:

To perform the various types of queries, we will first need to go to the AWS DynamoDB console, select the table against which we would like to execute our queries, and then click on the Explore Table button. This will execute a scan operation on the selected table, which will look like this:
Now, to perform a query operation, select the Query radio button. In the dropdown below it, you will see a choice to perform the query either on the table indexes or secondary indexes, which were created earlier. The first query we will perform will be to get the item by its id. Here, if you want to fetch an item/s whose id is 1, put 1 in the textbox against the Hash Key, click on the Query button, and you will see the results immediately:
We can also query data by providing a combination of Hash Key and Range Key to narrow down our search. In the earlier scenario, we got two items: one with the type phone, and another with the type book. Now if we want to fetch the item whose id is 1 and the type is phone, then we can query that accordingly, and we will get the results as required:
We can also query data on the Global Secondary Index, which we created at the time of table creation:
Similarly, we can use the scan operations in order to fetch the data from the DynamoDB table. We can either do a complete scan or a filtered scan. To perform a complete table scan, we have to select the Scan radio button, and click on the Start New Scan button. As we have not put any filters on the scan, this operation will fetch all the items present in the given DynamoDB table:
We can also add filters to the scan operation by clicking on the Add Filter button, and then by providing the filtering criteria. For example, if we want to fetch all the items of the phone type, then we can specify the criteria, and execute the scan operation.
Scan filters allow comparison operations, such as equal to, less than, less than or equal to, greater than, greater than or equal to, null, not null, contains, not contain, begins with, and so on. You can use it as per your need. Scan filtering can also be performed on the global secondary index. For example, if we want to fetch all the items whose manufacture name starts with A, then we can add this filter and get the results:

How it works…

Query and Scan are powerful operations that help us to retrieve data. However, both have their own working style and their own pros and cons.

A Query operation finds items in the table using the primary key attributes. You can specify the hash key and optional range key in order to get the desired items. By default, a Query operation returns all the attributes of a given item, but there is an option available to limit the attributes using ProjectionExpressions. While providing the key conditions, the Query operation expects an equality condition on the Hash Key attribute, but there are a set of conditions allowed to specify the Range Key conditions.

A query request can retrieve a maximum of 1 MB of data at a time. So, you need to plan the Query operation accordingly. You can optionally use FilterExpressions to narrow down this data. The Hash Key index is an unordered index, which means that we can only specify an exact key to fetch the items, whereas the Range Key index is an ordered index, which allows us to query that index using various ConditionalOperations, such as less than, greater than, begins with, between, and so on. If no matches are found, Query returns an empty result set.

A Scan operation examines each and every item in the given DynamoDB table, compares it with the condition, and gets the results back. By default, this also returns all the attributes of the items back to the user. Of course, we can use ProjectionExpressions to limit the set of attributes returned by the scan request.

A single scan request can fetch a maximum of 1 MB of data. To make the best use of it, we can also use FilterExpressions to narrow down the result set.

There are ways to handle this limitation of 1 MB of data per request, but we cannot do much using the DynamoDB console, so we will talk about all these details in Chapter 2, Operating with DynamoDB Tables.

Deleting the DynamoDB table using the console

In the earlier recipe, we discussed using the Query and Scan operations on the DynamoDB table using the console. Now, we will see how to delete a table using the same.

Getting ready

To perform this recipe, you need to create a table. Please create a dummy table, say dummyProduct, in DynamoDB using the recipe Creating the DynamoDB table using the console.

How to do it…

Note that by performing the following set of operations, the table and its data will get deleted forever, and there is no way to get it back, so I would suggest that you perform this recipe only if you know that the table is not needed anymore:

Here, I have two tables in my DynamoDB console now, out of which I will delete a table called dummyProduct:
Select a table to be deleted, and click on the Delete Table button:
This will open up a pop-up window, which will ask you for your confirmation. Check the checkbox against the Delete this table text, and click on the Delete button:
This will start the deleting process, which will run for a while and erase the complete data.

How it works…

The delete operation completely erases the data in the given table and drops that table from DynamoDB. If you delete the DynamoDB table, there is no way to get that data back, so perform this operation very carefully. I would suggest that you avoid this operation if you don't know the criticality of this table.

Analyzing DynamoDB metric on CloudWatch

It is very important to keep an eye on DynamoDB CloudWatch metrics in order to see the DynamoDB performance. CloudWatch provides real-time monitoring and alerting mechanisms to services provided by DynamoDB.

Getting ready

To get a better understanding of DynamoDB metrics, considering the fact that we don't have much of our data in our product table, go to the Query and Scan operations console and perform various Query and Scan operations back to back in order to get some metrics. Simply clicking on the Query and Scan buttons with some sample conditions will do the job for you.

Of course, in real time, in production systems, you would not need to do this as we will automatically have sufficient reads, writes, queries, and scans in progress.

How to do it…

Let's understand DynamoDB metrics on Cloudwatch:

Go to the DynamoDB console, and click on the table whose metrics you would like to analyze. You will notice that the bottom frame will start showing some details about the table. It will have various tabs, such as Details, which will have the table details, Indexes, which will have details about the secondary indexes that we created in our table, Monitoring, which will have the details about the CloudWatch metrics that we would like to see in detail, and the last tab, Alarm Setup, which is used to set up various alarms in case you wish to get any:
If you scroll down a bit, you will see various graphs, such as Read Capacity, Throttled Read Requests, Write Capacity, Throttled Write Requests, Read-Write Capacity for Global Secondary Indexes, that we created. It will also show you graphs related to Read/Write Latency and Query/Scan latency for a given time frame:
You can also click on individual graphs to see the enlarged view. I have made a significant number of scans in the last hour; if I want to see how DynamoDB performed for a particular time frame, I need to click on the Scan Latency graph, which will open the enlarged view of the graph, as shown in the following screenshot:
The preceding graph tells us the average time taken by DynamoDB in milliseconds to perform the various scan operations.
You can also change the Statistic drop-down value to select the various metrics. For example, if I want to see the exact time taken by each scan request over the last hour, it will show you the results immediately. I can also click on the data points on the graph to see the exact time. Here is an example of it:
Here, the graph tells us that the very first scan request took approximately 140 milliseconds to fetch the results, whereas the latency dropped in subsequent requests, and eventually, improved the performance.

How it works…

Almost every operation in Cloud is measured in terms of bytes read/written. CloudWatch keeps track of the requests that we make to DynamoDB and stores it in the internal data store. While generating the graphs, we will use these data points, which will help us analyze the performance.

There's more…

You can also analyze other CloudWatch metrics to better understand the DynamoDB behavior. These graphs will also help you decide upon the throughput values set for the tables and secondary indexes. If you see the consumed read/write capacity reaching the provisioned read/write capacity, then you can increase the provisioned throughput and vice versa.

Always look for consistent metrics; you might miss out on certain spikes because a lot of graphs give you an average of values over the last 5 minutes of time. For more detailed metrics, you can go to the CloudWatch console and look out for DynamoDB metrics at https://console.aws.amazon.com/cloudwatch/https://console.aws.amazon.com/cloudwatch/.

Downloading and setting up DynamoDB Local

A call to any cloud resource may cost money to you, even if you are just doing development and not talking about any hosting in the production cluster. During development, we may need to try many things, and all those trials would cost us. The development best practices demand us to follow a test-driven development. For continuous integration and builds, it's good to run unit tests and integration tests to make sure that the build is intact. If we keep running unit tests and integration tests on the actual DynamoDB, we may end up paying a lot. To cater to this issue, we have something called as DynamoDB Local.

DynamoDB Local is a small client-side database and server that mimics the actual DynamoDB. It enables you to develop and test your code in the local environment, without modifying anything on the actual DynamoDB. It is compatible with the actual DynamoDB API, so there is no need to worry about duplicating your efforts.

Getting ready

DynamoDB Local is a Java Archive (JAR) file, which will run on the Windows, Mac, or Linux machines. To execute these APIs, you should have Java Runtime Engine (JRE) 6.0+ installed on your machine. You can refer to following docs to install JRE on your machine:

For Windows 64-bit machine: http://www.oracle.com/technetwork/java/javase/install-windows-64-142952.html.

For Windows 32-bit machine: http://www.oracle.com/technetwork/java/javase/install-windows-141940.html.

For Mac: http://docs.oracle.com/javase/7/docs/webnotes/install/mac/mac-jre.html.

How to do it…

You can download DynamoDB Local from the following locations:

tar.gz format: http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz
.zip format: http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.zip

Now, let's perform the following steps to install DynamoDB Local:

Once done, just unzip the .jar file, and save it in a folder.
Now use Command Prompt, go to the folder where you have unzipped the .jar file, and execute the following command:
```
java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar
```
You will see the jetty server getting started at http://localhost:8000:
Now you can use this as an endpoint for your development. For example, if we want to use it in the Java SDK, we can use this as follows:
```
client = new AmazonDynamoDBClient(credentials);
client.setEndpoint("http://localhost:8000");
```
We will see more of its uses in the following chapters.

How it works…

DynamoDB Local is a simple .jar file that runs on your local machine and mimics the actual DynamoDB. You can do your development using DynamoDB, test your code, and simply redirect to the actual DynamoDB whenever it is ready for production deployment. Even though DynamoDB Local mimics most of the DynamoDB features, it does not support some important ones, which are as follows:

It does not consider the provisioned throughput settings while making any calls.
It does not throttle the read or write activity. The CreateTable, UpdateTable, and DeleteTable operations occur immediately. The table state is always ACTIVE.
It does not keep track of the consumed capacity units. So, it always returns null instead of the actual capacity units.
Read operations in DynamoDB are eventually consistent, but due to the speed of DynamoDB Local, it appears to be strongly consistent.

There's more…

The DynamoDB Local .jar gives you various other options in addition in order to manage DynamoDB Local smoothly. Here are some more options:

The -help function will list all the options available with DynamoDB Local JAR, as shown in the following command:

java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar – help

The output is shown in the following command:

Using DynamoDB Local JavaScript Shell

With the latest version of DynamoDB Local, you can also get an interactive DynamoDB Local JavaScript shell. In this recipe, we are going to see how to use it.

Getting ready

To perform this recipe, you need to have completed the earlier recipe of Downloading and setting up DynamoDB Local.

How to do it…

Enabling DynamoDB Local does not require any extra effort; we need to simply start DynamoDB Local, and go to http://localhost:8000/shell/ in the browser of your choice - you will see the interactive shell:

This interesting hands-on tutorial will help you understand DynamoDB. At the end of this tutorial, you will be able to create certain tables, try out various item levels and operations, and also create a demo application. Just use the navigation options as directed and this will be done.

How it works…

This DynamoDB Local Shell application runs using a regular application server and code in Java/JavaScript. This gets more interesting as you will actually get to do many things using DynamoDB JavaScript commands.

Setting up AWS Command Line Interface for DynamoDB

The AWS Command Line Interface allows us to operate various AWS resources from the command line itself. In this recipe, we are going to see how to use these and perform operations.

Getting ready

You can download the installer for Windows, Mac, or Linux from http://aws.amazon.com/cli/.

How to do it…

The AWS CLI for DynamoDB supports various commands, such as batch-get-item, batch-write-item, create-table, delete-item, delete-table, describe-table, get-item, list-tables, put-item, query, scan, update-item, update-table, wait, and so on.

To get started, we need to first configure the AWS CLI:

Start Command Prompt, and type the following command:
```
aws configure
```
This will ask you for the Access Key, Secret Key, and Region details. If you have not downloaded the AWS keys yet, you can download them from https://console.aws.amazon.com/iam/home?#security_credential.
Here, you can download the Access Keys and save it in a secure place. Once the keys are submitted, you can start accessing DynamoDB from the command line.
We need to list the tables that we have already created, and then we can execute this command:
```
aws dynamodb list-tables
```
The output of the following command is shown below:
We can also run the Query operations from the command line as well. Suppose that we want to Query a product table for id =5, then we have to write this condition in the JSON document and save it on our machine. The JSON file would like this:
```
{"id": {"AttributeValueList": [{"S":"5"}],"ComparisonOperator": "EQ" }}
```
Assume that we saved it in a file named conditions.json; then, to execute the query, we have to run the following command:
```
aws dynamodb query --table-name product --key-conditions file://conditions.json
```
The output would be something like this:
We can also specify conditions for the Range Key in order to narrow down our results, which is similar to what we did for the Hash Key.

How it works…

The AWS CLI, which is internally called DynamoDB APIs, is used to retrieve results. For authentication, it uses the Access Key and Secret Key that you provide at the time of configuration.

There's more…

You can also try out various other options from the AWS CLI for DynamoDB, which are available at http://docs.aws.amazon.com/cli/latest/reference/dynamodb/index.html#cli-aws-dynamodb.

Setting up the Eclipse IDE

Eclipse is an IDE, which is used mostly for Java development. In order to avoid switching screens between Eclipse and your browser, you can simply install the AWS plugin for Eclipse. In this recipe, we are going to see how to set up this plugin, and how to use it to perform various DynamoDB operations.

Getting ready

The first thing you need to do is install Eclipse on your laptop/desktop. The latest version from Eclipse can be downloaded from https://eclipse.org/downloads/. You need to select the Eclipse IDE for Java developers. Simply extract the ZIP file, and you are ready to go.

How to do it…

To install the AWS Explore plugin in Eclipse, we need to perform the following steps:

Open Eclipse, and go to Help | Install New Software.
In the Work with box, type http://aws.amazon.com/eclipse, and press Enter.
Select AWS Toolkit for Eclipse, and install the plugin:
Once done, it will prompt you to restart Eclipse. On restart, it will give you a popup to enter your AWS Access Key and Secret Key. We have already seen how to get the AWS Access Key and Secret Key for your account. Enter this information, and you are ready to use the AWS Explorer.
Now, you can open the AWS Explorer. By selecting Amazon DynamoDB, you will be able to see the table that we created earlier:
Now, we can perform the Scan operation by specifying the conditions. For example, to get all the items of the type book, we can do something like this:
Now you can perform the various queries using this AWS Explorer.

How it works…

The AWS Explorer for Eclipse uses the Java SDK internally to make calls to DynamoDB and shows the results to us. It authenticates each call using the Access Key and Secret Key that we provided at the time of configuration.