Understanding the difference between block storage and object storage

Amazon EBS stores data in blocks in the previous sections, whereas Amazon S3 stores data as objects. So now that we covered both of these services, the obvious question is which one is better to use. And the obvious answer is—it depends.

Amazon S3 is good for and often used to do the following:

• Host static websites and web pages
• Host web images and videos
• Store petabyte-scale amounts of data to perform data analytics on it
• Assisting in mobile applications

Amazon EBS is well suited for the following:

• Support business continuity
• Host big data applications that require high control of the environment using Hadoop, Spark, and similar frameworks
• Enable software testing
• Deployment of databases that need to be managed by the user and not AWS

The following table should also help you to decide what service is best for your use case:

Figure 5.2 – Choosing the service based on your use case

Hopefully, this...

Versioning in Amazon S3

Amazon S3 can optionally store different versions of the same object. Have you ever been working on a document for hours and suddenly made a mistake where you deleted all of the content in the document, or have you made a big mistake and wanted to go back to a previous version? Many editors, such as Microsoft Word, offer the ability to undo changes and recover from some of these mistakes. Once you save, close, and open the document again, you may not be able to undo any changes.

What if you have a document where multiple people make revisions, and you want to keep track of who made what changes?

Amazon S3 offers versioning capabilities that can assist with these use cases. So, what is versioning? Simply put, versioning is the ability to keep incremental copies. For example, if you store a Microsoft Word document in S3, the first version of the document may have ten pages, and the second version has 20 pages. It is easy to view and recover the previous version,...

Exploring Amazon S3 best practices

Amazon S3 is one of the simplest services in AWS, and at the same time, it is one of the most powerful and scalable services. We can easily scale our Amazon S3 applications to process thousands of requests per second while uploading and retrieving files. This scalability can be achieved "out of the box" without needing to provision any resources or servers.

Some customers in AWS are already leveraging Amazon S3 to host petabyte-scale data lakes and other applications storing billions of objects and performing billions of requests. These applications, with little optimization, can upload and retrieve multiple terabytes of data per second.

Other customers with low latency requirements have used Amazon S3 and other Amazon file storage services to achieve consistent low latency for small objects. Being able to retrieve this kind of object in 100 to 200 milliseconds is not uncommon.

For bigger objects, it is possible to achieve similar low latency...

Build hybrid storage with AWS Storage Gateway

While working on cloud migration, some applications will not be so simple to move to the cloud. Those apps may need to remain on-prem for performance reasons, compliance reasons, or they may be too complex to move into the cloud quickly. Some apps may need to remain on-prem indefinitely, so enterprises will want to explore hybrid cloud storage solutions that provide ready access for on-premises apps to data stored in AWS.

AWS Storage gateway acts as a bridge to provide access to virtually unlimited cloud storage by connecting applications running on-premises to Amazon storage. Storage Gateway allows customers to connect to and use key cloud storage services such as Amazon S3, Amazon S3 Glacier, Amazon FSx for Windows File Server, and Amazon Elastic Block Store (EBS). Additionally, Storage Gateway integrates with AWS services such as AWS Key Management Service (KMS), AWS Identity and Access Management (IAM), AWS CloudTrail, and AWS CloudWatch...

AWS Backup

AWS Backup is a centralized backup service that provides a simple and cost-effective way to back up application data across many AWS services to help customers meet their business continuity and backup compliance requirements. AWS Backup automates backup scheduling and retention management, provides a centralized way to configure and audit which resources are backed up, and monitors backup activity to alert you. AWS Backup integrates with CloudTrail and AWS Organizations for governance and management, giving our customers many options to help meet their recovery, restoration, and compliance needs.

AWS Backup is a fully managed, policy-based service that lets you centralize and automate the backup and restore of your applications spanning across 12 AWS services: Amazon Elastic Compute Cloud (Amazon EC2) instances, Amazon Elastic Block Store (EBS) volumes, Amazon Relational Database Service (RDS) databases (including Amazon Aurora clusters), Amazon DynamoDB tables, Amazon Neptune...

Summary

In this chapter, you reviewed some of the different options provided for storing files. Files make up one of the most valuable assets an enterprise has. These files could store a secret recipe, an important algorithm, client information, and other information that could prove catastrophic if it ended up in the wrong hands. For this reason, it is highly critical that, wherever these files are stored, only the people that should have access to them can. Similarly, it is of the utmost importance that these folks access this information whenever they need it, so these files should always be available. Some files are so critical that, if they were lost, it could prove fatal to the enterprise.

AWS offers a wide array of services to support the storage of files. Regardless of the AWS service, you select for file storage, they all offer features that provide a high degree of security, availability, and durability. Depending on the criticality of your data, you can architect your solution...

High-Performance Computing

High-Performance Computing (HPC) is a field of computing that involves using specialized hardware and software to solve complex, compute-intensive problems. HPC systems are typically used for scientific and engineering applications that require a high level of computational power, such as weather forecasting, molecular modeling, and oil and gas exploration. HPC systems require a high level of performance and scalability, but they can help organizations achieve results faster and more efficiently, making them an essential tool for many scientific and engineering applications.

HPC workloads are characterized by a combination of multiple technologies, such as storage, compute, networking, Artificial Intelligence (AI), machine learning, scheduling and orchestration, and streaming visualization, combined with specialized third-party applications.

HPC workloads are classified into categories that help identify the AWS services and solutions that can best...

Hybrid compute

While you want to benefit from the advantages of using the cloud, not all your applications can be migrated to AWS due to latency or the need for local data processing. Latency-sensitive applications such as patient care flow require less than 10 ms responses, and any delays can affect critical processes, so you want your compute to be near your equipment. Similarly, there are instances when you can’t afford downtime due to intermittent networking and want local data processing, for example, manufacturing execution systems, high-frequency trading, or medical diagnostics.

If you can’t move to a Region because of data residency, local processing, or latency requirements, you have to build and maintain the on-premises infrastructure at your facility. In that case, you must maintain an IT infrastructure, which involves a complex procurement and provisioning process from multiple vendors with a months-long lead time. In addition, you will have the overhead...

Summary

In this chapter, you learned about compute services available in AWS, which will help you choose the right to compute per your workload requirement. You learned about why terms changed from servers to compute recently due to the broad set of options provided by the cloud.

The most popular compute service is EC2, which is the foundation for the rest of the services provided by AWS. For example, a service such as Amazon SageMaker or Amazon DynamoDB under the hood relies on core services such as EC2. You learned about various EC2 families, pricing models, and advantage.

There are so many EC2 options available, which may be confusing when it comes to optimizing your cost model. You learned about AWS compute optimization, which can help you choose the right compute option and optimize cost. You also learned about AMI, which helps you choose the operating system for your workload and spin up EC2 per your needs. Further, you learned about EC2 best practices.

For distributed...