Amazon SimpleDB Developer Guide

By Prabhakar Chaganti , Rich Helms
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Getting to Know SimpleDB

About this book

SimpleDB is a highly scalable, simple-to-use, and inexpensive database in the cloud from Amazon Web Services. But in order to use SimpleDB, you really have to change your mindset. This isn't a traditional relational database; in fact it's not relational at all. For developers who have experience working with relational databases, this may lead to misconceptions as to how SimpleDB works.

This practical book aims to address your preconceptions on how SimpleDB will work for you. You will be quickly led through the differences between relational databases and SimpleDB, and the implications of using SimpleDB. Throughout this book, there is an emphasis on demonstrating key concepts with practical examples for Java, PHP, and Python developers.
 
You will be introduced to this massively scalable schema-less key-value data store: what it is, how it works, and why it is such a game-changer. You will then explore the basic functionality offered by SimpleDB including querying, code samples, and a lot more. This book will help you deploy services outside the Amazon cloud and access them from any web host.

You will see how SimpleDB gives you the freedom to focus on application development. As you work through this book you will be able to optimize the performance of your applications using parallel operations, caching with memcache, asynchronous operations, and more.

Publication date:
June 2010
Publisher
Packt
Pages
252
ISBN
9781847197344

 

Chapter 1. Getting to Know SimpleDB

Most developers would describe a modern database as relational with stored procedures and cross-table functions such as join. So why would you use a database that has none of these capabilities? The answer is scalability.

This morning, CNN ran a story on your new web application. Yesterday you had 10 concurrent users, and now your site is viral with 50,000 users signing on. Which database will handle 50,000 concurrent users without a complex expensive cluster? The answer is SimpleDB.

Why SimpleDB?

  • Scalability

  • Pay only for your use

  • Access from any web-based system

  • No fixed schema

Challenges?

  • New metaphor—write seldom, read many

  • Eventual consistency

SimpleDB is one of the core Amazon Web Services, which include Amazon Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2). Amazon SimpleDB stores your structured data as key-value pairs in the Amazon Web Services (AWS) cloud and lets you run real-time queries against this data. You can scale it easily in response to increased load from your successful applications without the need for a costly cluster database server complex.

SimpleDB, as illustrated in the following diagram, is designed to be used either as an independent data storage component in your applications or in conjunction with some of the other services from Amazon's stable of Cloud Services, such as Amazon S3 and Amazon EC2.

The biggest challenge in SimpleDB is learning to think in its unique metaphor. Like speaking a new language, you need to stop translating and start thinking in that language. Rather than thinking of SimpleDB as a database, approach it as a spreadsheet with some XML characteristics.

SimpleDB functionality can be accessed from almost any programming language (such as Python, Ruby, Java, PHP, Erlang, and Perl) using super simple HTTP-based requests. You can get started anytime you like, and you pay for it based on how much you use it. It is very different from a relational database, and takes a completely different approach toward storing and querying data. It follows the convention of eventual consistency. Think of it as a single master database for updates and a large collection of read database slaves. Any changes made to your data will need to be propagated across all the different copies. This can sometimes take a few seconds depending upon the system load at that time and network latency, which means that a consumer of your domain and data may not see the changes immediately. The changes will eventually be propagated throughout SimpleDB, but this is an important consideration you need to think about when designing your application.

 

Experimenting with SimpleDB


As SimpleDB is so different, it helps to have a tool for manipulating and exploring the database. When developing with a MySQL database, phpMyAdmin allows the developer to work directly on the database. SimpleDB has a similar free Firefox plugin called sdbtool (http://code.google.com/p/sdbtool/). Another Firefox plugin used in the more advanced examples is S3Fox (http://www.s3fox.net/) for administering the Amazon S3 storage. In this book, we cover several basic sample applications. We have also provided code to show each SimpleDB application using three languages: Java, PHP, and Python.

As access to SimpleDB can be from any site on the Web, the PHP samples can be downloaded and run directly from your site. To run any sample, an Amazon account is required. These samples let you explore most of the SimpleDB API, as well as some of the S3 API capabilities.

You can both download and try the PHP samples from http://www.webmasterinresidence.ca/simpledb/.

 

How does SimpleDB work?


The best way to wrap your head around the way SimpleDB works is to picture a spreadsheet that contains your structured data. For instance, a contact database that stores information on your customers can be represented in SimpleDB as follows:

As SimpleDB is a different database metaphor, new terms have been introduced. The use of this new terminology by Amazon stresses that the traditional assumptions may not be valid.

Domain

The entire customers table will be represented as the domain Customers. Domains group similar data for your application, and you can have up to 100 domains per AWS account. If required, you can increase this limit further by filling out a form on the SimpleDB website. The data stored in these domains is retrieved by making queries against the specific domain. There is no concept of joins as in the relational database world; therefore, queries run within a specific domain and not across domains.

Item

Each customer is represented by a unique Customer ID. Items are similar to rows in a database table. Each item identifies a single object and contains data for that individual item as a number of key-value attributes. Each item is identified by a unique key or identifier, or in traditional terminology, the primary key. SimpleDB does not support the concept of auto-incrementing keys, and most people use a generated key such as the unix timestamp combined with the user identifier or something similar as the unique identifier for an item. You can have up to one billion items in each domain.

Attributes

Each customer item will have distinguishing characteristics that are represented by an attribute. A customer will have a name, a phone number, an address, and other such attributes, which are similar to the columns in a table in a database. SimpleDB even enables you to have different attributes for each item in a domain. This kind of schema independence lets you mix and match items within a domain to satisfy the needs of your application easily, while at the same time enables you to take advantage of the benefits of the automatic indexing provided by SimpleDB. If your company suddenly decides to start marketing using Twitter, you can simply add a new attribute to your customer domain for the customers who have a Twitter ID! In traditional database terminology, there is no need to add a new column to the table.

Values

Each customer attribute will be associated with a value, which is the same as a cell in a spreadsheet or the value of a column in a database. A relational database or a spreadsheet supports only a single value for each cell or column, while SimpleDB allows you to have multiple values for a single attribute. This lets you do things such as store multiple e-mail addresses for a customer while taking advantage of automatic indexing, without the need for you to manually create new and separate columns for each e-mail address, and then index each new column. In a relational DB, a separate table with a join would be used to store the multiple values. Unlike a delimited list in a character field, the multiple values are indexed, enabling quick searching.

It is a simple way of modeling your data, but at the same time, it is different from a relational database model that is familiar to most users. The following table compares SimpleDB components with a spreadsheet and a relational database:

Relational Database

Spreadsheet

SimpleDB

Table

Worksheet

Domain

Row

Row

Item

Column

Cell

Attribute

Value

Value

Value(s)

 

How do I interact with SimpleDB?


You interact with SimpleDB by making authenticated HTTP requests along with the desired parameters. There are several libraries available in different programming languages that encapsulate this entire process and make it even easier to interact with SimpleDB by removing some of the tedium of manually constructing the HTTP requests. The next chapter explores these libraries and the advantages provided by them.

There are three main types of actions that you will need to do when you are working with SimpleDB—create, modify, and retrieve information about your domains by using the following operations:

  • CreateDomain: Create a new domain that contains your dataset.

  • DeleteDomain: Delete an existing domain.

  • ListDomains: List all the domains.

  • DomainMetadata: Retrieve information that gives you a general picture of the domain and the items that are stored within it, such as:

    • The date and time the metadata was last updated

    • The number of all items in the domain

    • The number of attribute name/value pairs in the domain

    • The number of unique attribute names in the domain

    • The total size of all item names in the domain, in bytes

    • The total size of all attribute values, in bytes

    • The total size of all unique attribute names, in bytes

You can create or modify the data stored within your domains by using the following operations:

  • PutAttributes: Create or update an item and its attributes. Items will automatically be indexed by SimpleDB as they are added.

  • BatchPutAttributes: Create or update multiple attributes (up to 25) in a single call for improved overall throughput of bulk write operations.

  • DeleteAttributes: Delete an item, an attribute, or an attribute value.

You can retrieve items that match your criteria from the dataset stored in your domains using the following operations:

  • GetAttributes: Retrieve an item and all or a subset of its attributes and values matching your criteria.

  • Select: Retrieve an item and all or a subset of its attributes and values matching your criteria, using the SELECT syntax that is popular in the SQL world.

The following diagram illustrates the different components of SimpleDB and the operations that can be used for interacting with them:

 

How is SimpleDB priced?


Amazon provides a free tier for SimpleDB along with pricing for usage above the free tier limit. The charges are based on the machine utilization of each SimpleDB request along with the amount of machine capacity that is utilized for completing the specified request normalized to the hourly capacity of a 1.7 GHz Xeon processor.

Free tier

As of the publication date of this book, there are no charges on the first 25 machine hours, 1 GB of data transfer, and 1 GB of storage that you consume every month. This is a significant amount of usage being provided for free for this limited time period by Amazon, and there are many kinds of applications that can operate entirely within this free tier. Pricing details are available at http://aws.amazon.com/simpledb/.

While a credit card is required to sign up, usage can be checked at any time with the Amazon Account Activity web page. Amazon estimates about 2,000,000 GET or SELECT API calls per month without any charge.

The pricing details might make it a bit daunting to figure out what your costs may be initially, but the free tier provided by Amazon goes a long way toward getting you more comfortable using the service and also putting SimpleDB through its paces without significant cost. There is also a nice calculator provided on the AWS site that is very helpful for computing the monthly usage costs for SimpleDB and the other Amazon web services. You can find the Amazon web services cost calculator at http://calculator.s3.amazonaws.com/calc5.html.

 

Why should I use SimpleDB?


You now have an overview of the service, and you are reasonably familiar with what SimpleDB can do. It is a great piece of technology that enables you to create scalable applications that are capable of using massive amounts of data, and you can put this power and simplicity to use in your own applications.

Make your applications simpler to architect

You can leverage SimpleDB in your applications to quickly add, edit, and retrieve data using a simple set of API calls. The well-thought-out API and simplicity of usage will make your applications easier to design, architect, and maintain in the long run, while removing the burdens of data modeling, index maintenance, and performance tuning.

Build flexibility into your applications

You no longer have to either know or pre-define every single piece of data that you will possibly "need to store" for your application. You expand your data set as you go and add the new attributes only when they are absolutely needed. SimpleDB enables you to do this easily, and even the indexing for these newly-added attributes is automatically handled behind the scenes without any need for your intervention.

Create high-performance web applications

High-performance web applications need the ability to store and retrieve data in a fast and efficient way. Amazon SimpleDB provides your applications with this ability while removing a lot of the administrative and maintenance complexities, leaving you free to focus on what's important to you—your application.

Take advantage of lower costs

You pay only for the SimpleDB resources that you actually consume, and you no longer need to lay out significant expenditures up front for database software licenses or even hardware. The capacity planning and handling of any spikes in load and traffic are automatically handled by Amazon, freeing valuable resources that can be deployed in other areas. SimpleDB pricing passes on to you the cost savings achieved by Amazon's economies of scale.

Scale your applications on demand

And last but most importantly, you can easily handle traffic and load spikes on your applications, as SimpleDB will be doing all of the heavy lifting and scaling for you. You can even handle the massive and tsunami-like increases in traffic that can result from being mentioned on the front page of Yahoo or Digg, or becoming a trendy topic on Twitter.

Architect for the cloud

SimpleDB is designed to integrate easily and work well with the other cloud services from Amazon such as Amazon EC2 and Amazon S3. This enables you to take full advantage of these other services and offload data processing and file storage needs to the cloud, while still using SimpleDB for your structured data storage needs. Web-scale computing for your application needs along with cost-effectiveness is easier thanks to these cloud services.

 

Summary


In this chapter, we explored SimpleDB and the advantages of utilizing it to build web-scale applications. In the next chapter, we will start interacting with SimpleDB, and getting familiar with creating and modifying datasets utilizing one of the widely available SimpleDB software libraries.

About the Authors

  • Prabhakar Chaganti

    Prabhakar Chaganti is the founder and CTO of Ylastic, a start-up that is building a single unified interface to architect, manage, and monitor a user's entire AWS Cloud computing environment: EC2, S3, RDS, AutoScaling, ELB, Cloudwatch, SQS, and SimpleDB. He is the author of Xen Virtualization and GWT Java AJAX Programming, and is also the winner of the community choice award for the most innovative virtual appliance in the VMware Global Virtual Appliance Challenge. He hangs out on Twitter as @pchaganti.

    Read about his tips on time management...

    Browse publications by this author
  • Rich Helms

    "It's never been done" is a call to action for Rich Helms. He has built a career on breaking new ground in the computer field. He developed CARES (Computer Assisted Recovery Enhancement System) for the Metropolitan Toronto Police in Canada. CARES was the first computer system in the world for aging missing children. CARES has been internationally recognized as pioneering work in child aging. Rich has also created several generations of e-Learning platforms including Learn it Script and most recently Educate Press.

    Rich is a seasoned software developer with over 30 years of experience. He spent 22 years in various positions at IBM including Chief Image Technology Architect. His credentials range from deep technical work (five patents in hardware and software) to running multinational R&D.

    Browse publications by this author
Book Title
Unlock this book and the full library for only $5/m
Access now