Amazon Cloud Storage Services

We explored Amazon EC2 and compute services in the previous chapter and provisioned EC2 instances in the Amazon cloud, including Windows and Linux instances. In this chapter, we will discuss Amazon cloud storage, including the block cloud storage that can be attached to an EC2 instance, the network filesystem cloud storage that can be shared by many EC2 instances, and the object cloud storage storing objects in the cloud. We will cover the following topics in this chapter:

• Amazon Elastic Block Store (EBS): Provides block-level storage volumes to EC2 instances. We will show how to create and attach storage volumes to EC2 instances and use them as primary storage.
• Amazon Elastic File System (EFS): Provides scalable and fully managed filesystem storage to be shared by EC2 instances and on-premises resources.
• Amazon Simple Storage Service (S3): Provides object storage that can store and retrieve any amount of data from anywhere on the web.
...

Understanding EBS

When we run out of storage on our home PC, we usually buy a new disk drive, shut down the computer, install the disk drive, reboot the computer to recognize the new disk drive, and then log into the computer guest OS to format the new disk drive and start using it.

This type of disk storage is called block storage, which is a technology that controls storage on the computer, using a block as the storing unit. It takes the data to be stored, divides it into blocks of equal sizes, and stores the data blocks on the underlying physical storage. With block storage, a block is the unit for data storing and retrieving, and only changes to the blocks are written to the disk. For example, when you change a sentence in a Microsoft Word doc and save the doc, only the blocks that store the sentence were updated to the physical disk drive, in contrast to object storage, which uses an object (such as a fingerprint file) as a storing unit - any partial changes to the object will...

Understanding EFS

An Amazon EBS volume can be created and attached to an EC2 instance, but usually, it is not meant for many instances to share. Amazon EFS provides simple, scalable, elastic file system storage that can be shared among many EC2 instances on the AWS Cloud and servers on-premises. With only a couple of clicks, we can create and share Amazon EFS to EC2 instances easily. Here are the detailed steps:

1. Create an Amazon EFS filesystem

Log into the AWS console, go to the EFS service, and create a filesystem, as shown in Figure 2.8:

Figure 2.8 – Create the Amazon EFS filesystem (1)

I. Click "Create file system":

In the pop-up window, provide a name for the EFS filesystem and choose a storage class: a One Zone EFS filesystem stores data at the AZ level, and a standard filesystem stores data redundantly across multiple AZs in the same region. Note that there is no need to specify the size of the EFS filesystem since...

Understanding S3

As we have discussed, Amazon EBS is block storage that can be attached to an EC2 instance. Amazon EFS is an elastic filesystem storage that can be shared among EC2 instances. Now, we will examine Amazon’s object storage: S3.

S3 is object-based storage and is a public end point accessible globally via the web and other means. In S3, objects or files are stored in a bucket (folder). S3 is a universal namespace storage, which means the names must be unique globally. While there is unlimited storage for S3 customers, each object or file is limited to 0 TB to 5 TB in size.

Amazon S3 offers a range of object-level storage classes that are designed for different use cases:

• S3 Standard, with 4x9 (99.99%) availability and durability.
• S3 Standard-Infrequent (Standard-IA), with 3x9 (99.9%) availability and 11x9 durability.
• S3 Reduced Redundancy Storage with 3x9 availability and 4x9 durability.
• S3 Intelligent-Tiering, which places the objects...

Understanding Snowball and Snowmobile

To help customers transfer a huge amount of data (close to petabytes) to the cloud, Amazon offers data transportation services including AWS Snowcone, AWS Snowball, and AWS Snowmobile.

AWS Snowcone is a small, rugged, but secure and versatile device that can help organizations to extend their AWS infrastructure to the edge and enable them to do fast data transfers. With Snowcone, you can collect, process, and transfer data to AWS, either offline by shipping the device or online with AWS DataSync.

AWS Snowball is an appliance-based data transfer service that uses secure, portable storage devices that are delivered to your doorstep, for you to load your data and then send it back to AWS after the data transfer is complete. The data is encrypted during the transferring process, leveraging Amazon S3 for secure storage and Amazon Key Management Service (KMS) for encrypting the data on the device. An AWS Snowball appliance provides up to 80 TB...

Accessing S3 from EC2 instances

Now that we have launched an EC2 instance in Chapter 1 and created S3 buckets in Chapter 2, naturally, we will ask the question: do my EC2 instances have access to my S3 buckets?

To answer this question, we need to look at it from two perspectives:

• S3 is a public endpoint, so the EC2 instance needs to have a public IP address.

However, that’s not enough – when you log into the EC2 instance, and run the command (as shown in the following figure):

aws s3 ls

You will find that there are no S3 buckets listed:

Figure 2.14 – No S3 bucket is found from EC2

• To have an EC2 instance access S3 buckets, we need to assign an IAM role to the EC2 instance. Recall that we briefly covered this in the EC2 section in Chapter 1:

An IAM role can be assigned with permissions to access other AWS resources, such as reading an Amazon Simple Storage Service (S3) bucket. By attaching the IAM...

Summary

Congratulations! We have completed Chapter 2 of our AWS self-learn journey: Amazon Cloud Storage Services. In this chapter, we introduced the Amazon EBS and EFS concepts, provisioned block storage and network filesystems for EC2 instances step by step, and then discussed the Amazon S3 services and Amazon data transfer services: Snowcone, Snowball, and Snowmobile.

As you can see, AWS provides these storage solutions to meet different business needs. Amazon EBS provides block storage volumes for EC2 instances, and some use cases include running a database that needs high-performance block storage, hosting a website that requires persistent storage, or an application that requires low-latency access to data. Amazon EFS provides a shared filesystem that can be accessed from multiple EC2 instances simultaneously. Some use cases for EFS include running web applications that require shared file storage, or a big data application that requires shared storage. Amazon S3 provides...

Practice questions

Questions 1-4 are based on Figure 2.22. You need to design an AWS storage system to move the on-prem storage to the cloud:

Figure 2.22 – An AWS system storage

1. How will you choose production storage in Region 1#?
   1. EBS
   2. S3
   3. Glacier
   4. EFS
2. How will you design cloud storage in Region 2#?
   1. Amazon Glacier with expedited retrievals
   2. Amazon S3 Standard-Infrequent Access
   3. Amazon EFS
   4. Amazon S3 Standard
3. What’s the best way to transfer files from on-prem to Region 1#?
   1. Amazon transfer with multi-part uploading
   2. AWS Snowball
   3. AWS Snowmobile
   4. AWS Transfer using direct-connect networks
4. What’s the best way to back up from Region 1# to Region 2#?
   1. Amazon S3 Cross-Region Replication
   2. AWS Snowball or Snowmobile
   3. Copy the video records from the cloud to a third-party data center
   4. Migration using direct-connect networks between regions
5. For Amazon S3, what is the largest object that can be uploaded in a single PUT?
   1. 5 gigabytes
   2. 5 terabytes
   3. 500 megabytes...

Answers to the practice questions

1. B.
2. A.
3. B.
4. A.
5. A.
6. D.
7. B.
8. Create an EC2 role that has read and write access to the S3 bucket, and assign the role to the EC2 instance.
9. Create a VPC endpoint for S3. Create an EC2 role that has read and write access to the S3 bucket, and assign the role to the EC2 instance.
10. No.

Further reading

For further insights into what you’ve learned in this chapter, refer to the following links:

• https://aws.amazon.com/ebs/
• https://aws.amazon.com/efs/
• https://aws.amazon.com/s3/
• https://docs.aws.amazon.com/ebs/index.html
• https://docs.aws.amazon.com/efs/index.html
• https://docs.aws.amazon.com/s3/index.html