Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Amazon SimpleDB Developer Guide
Amazon SimpleDB Developer Guide

Amazon SimpleDB Developer Guide: Scale your application's database on the cloud using Amazon SimpleDB

$15.99 per month
Book Jun 2010 252 pages 1st Edition
eBook
$25.99 $17.99
Print
$43.99
Subscription
$15.99 Monthly
eBook
$25.99 $17.99
Print
$43.99
Subscription
$15.99 Monthly

What do you get with a Packt Subscription?

Free for first 7 days. $15.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing

Product Details


Publication date : Jun 1, 2010
Length 252 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781847197344
Vendor :
Amazon
Category :
Languages :
Table of content icon View table of contents Preview book icon Preview Book

Amazon SimpleDB Developer Guide

Chapter 1. Getting to Know SimpleDB

Most developers would describe a modern database as relational with stored procedures and cross-table functions such as join. So why would you use a database that has none of these capabilities? The answer is scalability.

This morning, CNN ran a story on your new web application. Yesterday you had 10 concurrent users, and now your site is viral with 50,000 users signing on. Which database will handle 50,000 concurrent users without a complex expensive cluster? The answer is SimpleDB.

Why SimpleDB?

  • Scalability

  • Pay only for your use

  • Access from any web-based system

  • No fixed schema

Challenges?

  • New metaphor—write seldom, read many

  • Eventual consistency

SimpleDB is one of the core Amazon Web Services, which include Amazon Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2). Amazon SimpleDB stores your structured data as key-value pairs in the Amazon Web Services (AWS) cloud and lets you run real-time queries against this data. You can scale it easily in response to increased load from your successful applications without the need for a costly cluster database server complex.

SimpleDB, as illustrated in the following diagram, is designed to be used either as an independent data storage component in your applications or in conjunction with some of the other services from Amazon's stable of Cloud Services, such as Amazon S3 and Amazon EC2.

The biggest challenge in SimpleDB is learning to think in its unique metaphor. Like speaking a new language, you need to stop translating and start thinking in that language. Rather than thinking of SimpleDB as a database, approach it as a spreadsheet with some XML characteristics.

SimpleDB functionality can be accessed from almost any programming language (such as Python, Ruby, Java, PHP, Erlang, and Perl) using super simple HTTP-based requests. You can get started anytime you like, and you pay for it based on how much you use it. It is very different from a relational database, and takes a completely different approach toward storing and querying data. It follows the convention of eventual consistency. Think of it as a single master database for updates and a large collection of read database slaves. Any changes made to your data will need to be propagated across all the different copies. This can sometimes take a few seconds depending upon the system load at that time and network latency, which means that a consumer of your domain and data may not see the changes immediately. The changes will eventually be propagated throughout SimpleDB, but this is an important consideration you need to think about when designing your application.

Experimenting with SimpleDB


As SimpleDB is so different, it helps to have a tool for manipulating and exploring the database. When developing with a MySQL database, phpMyAdmin allows the developer to work directly on the database. SimpleDB has a similar free Firefox plugin called sdbtool (http://code.google.com/p/sdbtool/). Another Firefox plugin used in the more advanced examples is S3Fox (http://www.s3fox.net/) for administering the Amazon S3 storage. In this book, we cover several basic sample applications. We have also provided code to show each SimpleDB application using three languages: Java, PHP, and Python.

As access to SimpleDB can be from any site on the Web, the PHP samples can be downloaded and run directly from your site. To run any sample, an Amazon account is required. These samples let you explore most of the SimpleDB API, as well as some of the S3 API capabilities.

You can both download and try the PHP samples from http://www.webmasterinresidence.ca/simpledb/.

How does SimpleDB work?


The best way to wrap your head around the way SimpleDB works is to picture a spreadsheet that contains your structured data. For instance, a contact database that stores information on your customers can be represented in SimpleDB as follows:

As SimpleDB is a different database metaphor, new terms have been introduced. The use of this new terminology by Amazon stresses that the traditional assumptions may not be valid.

Domain

The entire customers table will be represented as the domain Customers. Domains group similar data for your application, and you can have up to 100 domains per AWS account. If required, you can increase this limit further by filling out a form on the SimpleDB website. The data stored in these domains is retrieved by making queries against the specific domain. There is no concept of joins as in the relational database world; therefore, queries run within a specific domain and not across domains.

Item

Each customer is represented by a unique Customer ID. Items are similar to rows in a database table. Each item identifies a single object and contains data for that individual item as a number of key-value attributes. Each item is identified by a unique key or identifier, or in traditional terminology, the primary key. SimpleDB does not support the concept of auto-incrementing keys, and most people use a generated key such as the unix timestamp combined with the user identifier or something similar as the unique identifier for an item. You can have up to one billion items in each domain.

Attributes

Each customer item will have distinguishing characteristics that are represented by an attribute. A customer will have a name, a phone number, an address, and other such attributes, which are similar to the columns in a table in a database. SimpleDB even enables you to have different attributes for each item in a domain. This kind of schema independence lets you mix and match items within a domain to satisfy the needs of your application easily, while at the same time enables you to take advantage of the benefits of the automatic indexing provided by SimpleDB. If your company suddenly decides to start marketing using Twitter, you can simply add a new attribute to your customer domain for the customers who have a Twitter ID! In traditional database terminology, there is no need to add a new column to the table.

Values

Each customer attribute will be associated with a value, which is the same as a cell in a spreadsheet or the value of a column in a database. A relational database or a spreadsheet supports only a single value for each cell or column, while SimpleDB allows you to have multiple values for a single attribute. This lets you do things such as store multiple e-mail addresses for a customer while taking advantage of automatic indexing, without the need for you to manually create new and separate columns for each e-mail address, and then index each new column. In a relational DB, a separate table with a join would be used to store the multiple values. Unlike a delimited list in a character field, the multiple values are indexed, enabling quick searching.

It is a simple way of modeling your data, but at the same time, it is different from a relational database model that is familiar to most users. The following table compares SimpleDB components with a spreadsheet and a relational database:

Relational Database

Spreadsheet

SimpleDB

Table

Worksheet

Domain

Row

Row

Item

Column

Cell

Attribute

Value

Value

Value(s)

How do I interact with SimpleDB?


You interact with SimpleDB by making authenticated HTTP requests along with the desired parameters. There are several libraries available in different programming languages that encapsulate this entire process and make it even easier to interact with SimpleDB by removing some of the tedium of manually constructing the HTTP requests. The next chapter explores these libraries and the advantages provided by them.

There are three main types of actions that you will need to do when you are working with SimpleDB—create, modify, and retrieve information about your domains by using the following operations:

  • CreateDomain: Create a new domain that contains your dataset.

  • DeleteDomain: Delete an existing domain.

  • ListDomains: List all the domains.

  • DomainMetadata: Retrieve information that gives you a general picture of the domain and the items that are stored within it, such as:

    • The date and time the metadata was last updated

    • The number of all items in the domain

    • The number of attribute name/value pairs in the domain

    • The number of unique attribute names in the domain

    • The total size of all item names in the domain, in bytes

    • The total size of all attribute values, in bytes

    • The total size of all unique attribute names, in bytes

You can create or modify the data stored within your domains by using the following operations:

  • PutAttributes: Create or update an item and its attributes. Items will automatically be indexed by SimpleDB as they are added.

  • BatchPutAttributes: Create or update multiple attributes (up to 25) in a single call for improved overall throughput of bulk write operations.

  • DeleteAttributes: Delete an item, an attribute, or an attribute value.

You can retrieve items that match your criteria from the dataset stored in your domains using the following operations:

  • GetAttributes: Retrieve an item and all or a subset of its attributes and values matching your criteria.

  • Select: Retrieve an item and all or a subset of its attributes and values matching your criteria, using the SELECT syntax that is popular in the SQL world.

The following diagram illustrates the different components of SimpleDB and the operations that can be used for interacting with them:

How is SimpleDB priced?


Amazon provides a free tier for SimpleDB along with pricing for usage above the free tier limit. The charges are based on the machine utilization of each SimpleDB request along with the amount of machine capacity that is utilized for completing the specified request normalized to the hourly capacity of a 1.7 GHz Xeon processor.

Free tier

As of the publication date of this book, there are no charges on the first 25 machine hours, 1 GB of data transfer, and 1 GB of storage that you consume every month. This is a significant amount of usage being provided for free for this limited time period by Amazon, and there are many kinds of applications that can operate entirely within this free tier. Pricing details are available at http://aws.amazon.com/simpledb/.

While a credit card is required to sign up, usage can be checked at any time with the Amazon Account Activity web page. Amazon estimates about 2,000,000 GET or SELECT API calls per month without any charge.

The pricing details might make it a bit daunting to figure out what your costs may be initially, but the free tier provided by Amazon goes a long way toward getting you more comfortable using the service and also putting SimpleDB through its paces without significant cost. There is also a nice calculator provided on the AWS site that is very helpful for computing the monthly usage costs for SimpleDB and the other Amazon web services. You can find the Amazon web services cost calculator at http://calculator.s3.amazonaws.com/calc5.html.

Why should I use SimpleDB?


You now have an overview of the service, and you are reasonably familiar with what SimpleDB can do. It is a great piece of technology that enables you to create scalable applications that are capable of using massive amounts of data, and you can put this power and simplicity to use in your own applications.

Make your applications simpler to architect

You can leverage SimpleDB in your applications to quickly add, edit, and retrieve data using a simple set of API calls. The well-thought-out API and simplicity of usage will make your applications easier to design, architect, and maintain in the long run, while removing the burdens of data modeling, index maintenance, and performance tuning.

Build flexibility into your applications

You no longer have to either know or pre-define every single piece of data that you will possibly "need to store" for your application. You expand your data set as you go and add the new attributes only when they are absolutely needed. SimpleDB enables you to do this easily, and even the indexing for these newly-added attributes is automatically handled behind the scenes without any need for your intervention.

Create high-performance web applications

High-performance web applications need the ability to store and retrieve data in a fast and efficient way. Amazon SimpleDB provides your applications with this ability while removing a lot of the administrative and maintenance complexities, leaving you free to focus on what's important to you—your application.

Take advantage of lower costs

You pay only for the SimpleDB resources that you actually consume, and you no longer need to lay out significant expenditures up front for database software licenses or even hardware. The capacity planning and handling of any spikes in load and traffic are automatically handled by Amazon, freeing valuable resources that can be deployed in other areas. SimpleDB pricing passes on to you the cost savings achieved by Amazon's economies of scale.

Scale your applications on demand

And last but most importantly, you can easily handle traffic and load spikes on your applications, as SimpleDB will be doing all of the heavy lifting and scaling for you. You can even handle the massive and tsunami-like increases in traffic that can result from being mentioned on the front page of Yahoo or Digg, or becoming a trendy topic on Twitter.

Architect for the cloud

SimpleDB is designed to integrate easily and work well with the other cloud services from Amazon such as Amazon EC2 and Amazon S3. This enables you to take full advantage of these other services and offload data processing and file storage needs to the cloud, while still using SimpleDB for your structured data storage needs. Web-scale computing for your application needs along with cost-effectiveness is easier thanks to these cloud services.

Summary


In this chapter, we explored SimpleDB and the advantages of utilizing it to build web-scale applications. In the next chapter, we will start interacting with SimpleDB, and getting familiar with creating and modifying datasets utilizing one of the widely available SimpleDB software libraries.

Left arrow icon Right arrow icon

Key benefits

  • Offload the time, effort, and capital associated with architecting and operating a simple, flexible, and scalable web database
  • A complete guide that covers everything from installation to advanced features aimed at optimizing your application
  • Examine SimpleDB and the relational database model and review the Simple DB data model
  • Packed with examples in Java, PHP, and Python and screenshots to illustrate key concepts allowing you to focus application development

Description

SimpleDB is a highly scalable, simple-to-use, and inexpensive database in the cloud from Amazon Web Services. But in order to use SimpleDB, you really have to change your mindset. This isn't a traditional relational database; in fact it's not relational at all. For developers who have experience working with relational databases, this may lead to misconceptions as to how SimpleDB works.This practical book aims to address your preconceptions on how SimpleDB will work for you. You will be quickly led through the differences between relational databases and SimpleDB, and the implications of using SimpleDB. Throughout this book, there is an emphasis on demonstrating key concepts with practical examples for Java, PHP, and Python developers.You will be introduced to this massively scalable schema-less key-value data store: what it is, how it works, and why it is such a game-changer. You will then explore the basic functionality offered by SimpleDB including querying, code samples, and a lot more. This book will help you deploy services outside the Amazon cloud and access them from any web host.You will see how SimpleDB gives you the freedom to focus on application development. As you work through this book you will be able to optimize the performance of your applications using parallel operations, caching with memcache, asynchronous operations, and more.

What you will learn

Explore several SimpleDB operations with the easy-to-use Boto library on Java, PHP, and Python consoles Encode and decode various data types into string data using Lexicographical comparison with precise instructions Query, sort, and count data using SELECT syntax Store large binary objects in Amazon S3 while using SimpleDB as the metadata store Tune SimpleDB queries using the box usage value Avoid excessive SimpleDB requests by using a cache Optimize your application s performance using parallel operations

What do you get with a Packt Subscription?

Free for first 7 days. $15.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing

Product Details


Publication date : Jun 1, 2010
Length 252 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781847197344
Vendor :
Amazon
Category :
Languages :

Table of Contents

16 Chapters
Amazon SimpleDB Developer Guide Chevron down icon Chevron up icon
Credits Chevron down icon Chevron up icon
Foreword Chevron down icon Chevron up icon
About the Authors Chevron down icon Chevron up icon
About the Reviewers Chevron down icon Chevron up icon
Preface Chevron down icon Chevron up icon
Getting to Know SimpleDB Chevron down icon Chevron up icon
Getting Started with SimpleDB Chevron down icon Chevron up icon
SimpleDB versus RDBMS Chevron down icon Chevron up icon
The SimpleDB Data Model Chevron down icon Chevron up icon
Data Types Chevron down icon Chevron up icon
Querying Chevron down icon Chevron up icon
Storing Data on S3 Chevron down icon Chevron up icon
Tuning and Usage Costs Chevron down icon Chevron up icon
Caching Chevron down icon Chevron up icon
Parallel Processing Chevron down icon Chevron up icon

Customer reviews

Filter icon Filter
Top Reviews
Rating distribution
Empty star icon Empty star icon Empty star icon Empty star icon Empty star icon 0
(0 Ratings)
5 star 0%
4 star 0%
3 star 0%
2 star 0%
1 star 0%

Filter reviews by


No reviews found
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is included in a Packt subscription? Chevron down icon Chevron up icon

A subscription provides you with full access to view all Packt and licnesed content online, this includes exclusive access to Early Access titles. Depending on the tier chosen you can also earn credits and discounts to use for owning content

How can I cancel my subscription? Chevron down icon Chevron up icon

To cancel your subscription with us simply go to the account page - found in the top right of the page or at https://subscription.packtpub.com/my-account/subscription - From here you will see the ‘cancel subscription’ button in the grey box with your subscription information in.

What are credits? Chevron down icon Chevron up icon

Credits can be earned from reading 40 section of any title within the payment cycle - a month starting from the day of subscription payment. You also earn a Credit every month if you subscribe to our annual or 18 month plans. Credits can be used to buy books DRM free, the same way that you would pay for a book. Your credits can be found in the subscription homepage - subscription.packtpub.com - clicking on ‘the my’ library dropdown and selecting ‘credits’.

What happens if an Early Access Course is cancelled? Chevron down icon Chevron up icon

Projects are rarely cancelled, but sometimes it's unavoidable. If an Early Access course is cancelled or excessively delayed, you can exchange your purchase for another course. For further details, please contact us here.

Where can I send feedback about an Early Access title? Chevron down icon Chevron up icon

If you have any feedback about the product you're reading, or Early Access in general, then please fill out a contact form here and we'll make sure the feedback gets to the right team. 

Can I download the code files for Early Access titles? Chevron down icon Chevron up icon

We try to ensure that all books in Early Access have code available to use, download, and fork on GitHub. This helps us be more agile in the development of the book, and helps keep the often changing code base of new versions and new technologies as up to date as possible. Unfortunately, however, there will be rare cases when it is not possible for us to have downloadable code samples available until publication.

When we publish the book, the code files will also be available to download from the Packt website.

How accurate is the publication date? Chevron down icon Chevron up icon

The publication date is as accurate as we can be at any point in the project. Unfortunately, delays can happen. Often those delays are out of our control, such as changes to the technology code base or delays in the tech release. We do our best to give you an accurate estimate of the publication date at any given time, and as more chapters are delivered, the more accurate the delivery date will become.

How will I know when new chapters are ready? Chevron down icon Chevron up icon

We'll let you know every time there has been an update to a course that you've bought in Early Access. You'll get an email to let you know there has been a new chapter, or a change to a previous chapter. The new chapters are automatically added to your account, so you can also check back there any time you're ready and download or read them online.

I am a Packt subscriber, do I get Early Access? Chevron down icon Chevron up icon

Yes, all Early Access content is fully available through your subscription. You will need to have a paid for or active trial subscription in order to access all titles.

How is Early Access delivered? Chevron down icon Chevron up icon

Early Access is currently only available as a PDF or through our online reader. As we make changes or add new chapters, the files in your Packt account will be updated so you can download them again or view them online immediately.

How do I buy Early Access content? Chevron down icon Chevron up icon

Early Access is a way of us getting our content to you quicker, but the method of buying the Early Access course is still the same. Just find the course you want to buy, go through the check-out steps, and you’ll get a confirmation email from us with information and a link to the relevant Early Access courses.

What is Early Access? Chevron down icon Chevron up icon

Keeping up to date with the latest technology is difficult; new versions, new frameworks, new techniques. This feature gives you a head-start to our content, as it's being created. With Early Access you'll receive each chapter as it's written, and get regular updates throughout the product's development, as well as the final course as soon as it's ready.We created Early Access as a means of giving you the information you need, as soon as it's available. As we go through the process of developing a course, 99% of it can be ready but we can't publish until that last 1% falls in to place. Early Access helps to unlock the potential of our content early, to help you start your learning when you need it most. You not only get access to every chapter as it's delivered, edited, and updated, but you'll also get the finalized, DRM-free product to download in any format you want when it's published. As a member of Packt, you'll also be eligible for our exclusive offers, including a free course every day, and discounts on new and popular titles.