Storage in AWS – Choosing the Right Tool for the Job

Storage is a critical and foundational service for any cloud provider. If this service is not implemented in a durable, available, efficient, low-latency manner, it doesn’t matter how many other excellent services are offered.

File, block, and object storage are at the core of many applications. In Chapter 7, Selecting the Right Database Service, we will learn about other storage services focused on databases. However in this chapter, we will focus on basic file and object storage.

In this chapter, we will first look at Amazon EBS, EFS, and S3. We will then look at the difference between block storage and object storage. We will also look at versioning in Amazon S3 and explore Amazon S3 best practices.

In this chapter, we will cover the following topics:

• Understanding local storage with Amazon Elastic Block Store (EBS)
• Investigating file storage with Amazon Elastic File System (EFS)
...

Understanding Amazon Elastic Block Store

Block storage is a foundational storage technology that has been around since the early days of computing. The hard drive in your laptop, the memory in your mobile phone, and all other forms of data storage, from USB thumb drives to storage arrays that organizations place in their data centers, are all based on block storage.

Persistent block storage that can be used with Amazon EC2 instances is provided by Amazon Elastic Block Store (EBS). When using EC2, you have the option to use local instance storage or EBS for block storage:

• Instance storage is great for high-performance (over 80K IOPS and over 1,750 MB/s throughput) and low-latency (under 1 ms) applications. However, instance storage is ephemeral, which means when you stop, hibernate, or terminate an EC2 instance, every block of storage in the instance store is reset. Therefore, do not rely on instance storage for valuable, long-term data.
• EBS volumes provide excellent...

Join our book community on Discord

https://packt.link/AWS4SAs

AWS has come a long way since Amazon started in 2006, when it offered just two basic services. In this chapter, we will highlight the scale of AWS's global infrastructure. Enterprises today have become exponentially more agile by leveraging the power of the cloud. In this chapter, you will learn about the AWS Global Infrastructure along with AWS networking foundations.

Networking is the first step for any organization to setup their landing zone and entire IT workload built on top of it. You can say that networking is backbone of IT application and infrastructure workload. AWS provided various networking services to build you IT landscape in cloud and in this chapter your will dive deep into AWS networking services.

Every business is now running at global scale and it’s important for organization to target global populations for their product. With traditional on-premise IT workload, it become challenging to...

Learning about the AWS Global Infrastructure

The infrastructure offered by AWS is highly secure and reliable. It offers over 200 services. Most of them are available in all AWS Regions around the world spread across 245 countries. Regardless of the type of technology application you are planning to build and deploy, AWS is sure to provide a service that will facilitate its deployment.

AWS has millions of customers and tens of thousands of consulting and technology partners around the world. Businesses large and small across all industries rely on AWS to handle their workloads. To give you an idea of the breadth of the scale that AWS provides, here are some statistics. AWS provides the followings as their global infrastructure:

• 26 launched Regions and 8 Announced Regions
• 84 Availability Zones
• 110+ Direct Connect locations
• 310+ Points of Presence
• 17 Local Zones and 32 Announced Local Zones
• 24 Wavelength Zones

IMPORTANT NOTE

These numbers are accurate as of the writing of this book....

Summary

Networking is backbone of any IT workload whether it is cloud or on-premise network. To start you cloud journey in AWS you must have good knowledge of AWS networking. In this chapter, you started with learning AWS global infrastructure and understand details of AWS regions, Availability Zones and Edge location.

When you are starting with AWS the first things you create your Virtual Private Cloud (APC) within AWS. You learned about AWS VPC with various components such as security group, NACL, Route table, Internet gateway and NAT gateway etc. You learned how to segregate and secure your IT resources by putting them into private and public subnet.

With ease of creating VPC in AWS organizations tends to create multiple VPCs whether it is intentional to keep each team has their own VPC or unintentional when dev team create multiple test workloads. Often these VPCs need to communicate with each other for example Finance department need to get information from accounting. You learned...

Versioning in Amazon S3

Amazon S3 can optionally store different versions of the same object. Have you ever been working on a document for hours and suddenly made a mistake where you deleted all of the content in the document, or have you made a big mistake and wanted to go back to a previous version? Many editors, such as Microsoft Word, offer the ability to undo changes and recover from some of these mistakes. However, once you save, close, and open the document again, you may not be able to undo any changes.

What if you have a document where multiple people make revisions, and you want to keep track of who made what changes?

Amazon S3 offers versioning capabilities that can assist with these use cases. So, what is versioning? Simply put, versioning is the ability to keep incremental copies. For example, if you store an important proposal document in S3, the first version of the document may have the initial architecture and statement of work, and the subsequent version...

Choosing the right cloud storage type

So far, you have learned about three different kinds of cloud storage in this chapter. First, Amazon EBS stores data in blocks; you can also use this as SAN in the cloud. Second, Amazon EFS is cloud file storage that is a kind of NAS in the cloud. Finally, Amazon S3 stores data as objects. So now that we covered all these storage type services, the obvious question is which one is better to use. The following table should help you to decide what service is best for your use case:

Figure 5.3: Choosing the service based on your use case

EBS volume is always attached to a single EC2 instance so when you need high-performance, persistent storage, always use EBS volume. If you need shared file storage between multiple EC2 instances, then you want to use EFS. S3 is your choice to store any amount of data in any format that you want to use for big data analytics, backups, and even large volume content for your application.

As you are...

Exploring Amazon S3 best practices

Amazon S3 is one of the simplest services in AWS, and at the same time, it is one of the most powerful and scalable services. We can easily scale our Amazon S3 applications to process thousands of requests per second while uploading and retrieving files. This scalability can be achieved “out of the box” without needing to provision any resources or servers.

Some customers in AWS are already leveraging Amazon S3 to host petabyte-scale data lakes and other applications storing billions of objects and performing billions of requests. These applications can upload and retrieve multiple terabytes of data per second with little optimization.

Other customers with low latency requirements have used Amazon S3 and other Amazon file storage services to achieve consistent low latency for small objects. Being able to retrieve this kind of object in 100 to 200 milliseconds is not uncommon.

For bigger objects, it is possible to achieve...

Building hybrid storage with AWS Storage Gateway

While working on cloud migration, some applications will not be so simple to move to the cloud. Those apps might need to stay on-premises for performance reasons or compliance reasons, or they may be too complex to move into the cloud quickly. Some apps may need to remain on-premises indefinitely such as mainframe applications or legacy applications that need to meet licensing requirements. To address these use cases, you need to explore hybrid cloud storage solutions that provide ready access for on-premises apps to data stored in AWS.

AWS Storage Gateway

AWS Storage Gateway acts as a bridge to provide access to almost unlimited cloud storage by connecting applications running on-premises to Amazon storage. As shown in the diagram below, Storage Gateway allows customers to connect to and use key cloud storage services such as Amazon S3, Amazon S3 Glacier, Amazon FSx for Windows File Server, and Amazon EBS. Additionally, Storage...

AWS Backup

AWS Backup is a service that centralizes backup management and enables a straightforward and economical means of backing up application data across multiple AWS services to help customers comply with their business continuity and backup requirements. It automates backup scheduling and retention management, and it provides a centralized way for configuring and auditing the resources that require backup. Additionally, it keeps an eye on backup activity and alerts you in case of any issues. AWS Backup integrates with CloudTrail and AWS Organizations for governance and management, giving customers many options to help meet their recovery, restoration, and compliance needs.

AWS Backup enables centralized configuration and management of backups for various AWS resources including Amazon EC2 instances, Amazon EBS volumes, Amazon Relational Database Service databases, Amazon DynamoDB tables, Amazon EFS file systems, and other resources. You will learn about all of the database...

Summary

In this chapter, you learned about Storage Area Network (SAN) in the cloud with Amazon EBS. You learned about various EBS options and how to choose the right EBS volume per your workload. You further learned about Network Attached Storage (NAS) in the cloud with Amazon EFS and file system-specific workloads with Amazon FSx.

With the ever-increasing amount of data, you need scalable storage to store petabytes of data, and AWS provides Amazon S3 to fulfill that need. You learned about various tiers of Amazon S3, including S3 Standard, Intelligent Tiering, Infrequent Access-IA, One Zone-IA, and S3 Glacier. You further learned about S3 versioning to save a copy of your file and build multi-destination replication.

Later in the chapter, you learned about Amazon S3 best practices and optimized your S3 storage for performance, cost, and security. Finally, you learned about building a hybrid cloud with AWS Storage Gateway and a cloud-native backup option with AWS Backup.

...