Implementing Cloud Design Patterns for AWS - Second Edition

5 (2 reviews total)
By Sean Keery , Clive Harber , Marcus Young
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Introduction to Amazon Web Services

About this book

Whether you're just getting your feet wet in cloud infrastructure or already creating complex systems, this book will guide you through using the patterns to fit your system needs.

Starting with patterns that cover basic processes such as source control and infrastructure-as-code, the book goes on to introduce cloud security practices. You'll then cover patterns of availability and scalability and get acquainted with the ephemeral nature of cloud environments. You'll also explore advanced DevOps patterns in operations and maintenance, before focusing on virtualization patterns such as containerization and serverless computing. In the final leg of your journey, this book will delve into data persistence and visualization patterns. You'll get to grips with architectures for processing static and dynamic data, as well as practices for managing streaming data.

By the end of this book, you will be able to design applications that are tolerant of underlying hardware failures, resilient against an unexpected influx of data, and easy to manage and replicate.

Publication date:
April 2019
Publisher
Packt
Pages
274
ISBN
9781789136203

 

Chapter 1. Introduction to Amazon Web Services

Welcome to the first chapter of this book, Implementing Cloud Design Patterns for AWS, Second Edition. After a brief history of the development of the cloud and the part that AWS plays in it, we'll look into the Amazon philosophy. An exploration of some of its most well-known customers will give us a platform to dive into the reasons why it's a good choice for your product development effort. We'll survey the service offerings from AWS and create a trial account, which you can use to develop your products.

This chapter will briefly cover the following topics:

  • Introduction to AWS
  • Utility computing
  • Categories of service offerings
  • Benefits of moving to the cloud

 

 

Introduction to AWS


Amazon—aren't they an online bookstore? This is what someone in the late 1990s might have asked. Nowadays, with nearly 40% of the e-commerce market in the US and over 30% in India, everyone knows who they are. But not so well-known outside of technology circles is Amazon Web Services (AWS). Amazon describes this part of the business as follows:

AWS is a secure cloud services platform (https://aws.amazon.com/what-is-cloud-computing/), offering compute power, database storage, content delivery, and other functionality to help businesses scale and grow. Millions of customers (https://aws.amazon.com/solutions/case-studies/) are currently leveraging AWS cloud products (https://aws.amazon.com/products/) and solutions (https://aws.amazon.com/solutions/) to build sophisticated applications with increased flexibility, scalability, and reliability (https://aws.amazon.com/what-is-aws/).

Amazon is currently the leader in the cloud infrastructure business with 33% market share—about the same as Google Cloud, Azure, and IBM combined. They're using their huge lead, and the profits from it, to invest in new services to further differentiate themselves. Since the publishing of the first edition of this book, they've added hundreds of new offerings. Millions of customers are taking advantage of these services. The initial target market of startups and small businesses has spawned web-scale enterprises such as Netflix, Pinterest, and Airbnb. The Fortune 500 and governments around the world are also on board.

Note

On an annual basis, all of Amazon's operating income derived from AWS in 2017 (https://www.zdnet.com/article/all-of-amazons-2017-operating-income-comes-from-aws/).

History

How we got here provides valuable lessons for building your product and scaling it globally. The retail business is peaky. Amazon needed capacity to meet its highest demand period during Thanksgiving and Black Friday. Come January, it had a whole lot of hardware sitting around, underutilized. Having worked for years providing e-commerce services to partners, they were able to take their internal offerings and make them available to the public. Utility computing was born.

 

This pattern has been repeated many times. Amazon is constantly adding services to tackle new and unique problems that they have encountered within their own business. Those internal services then get previewed and customer feedback is collected. When they meet a certain service level, they're released for use by anyone with a credit card. You need to use this same strategy for your products.

The future

Product-driven development will continue to evolve within the Amazon ecosystem. The benefits that are identified internally will trickle down to all of the AWS cloud users. Newer services will continue to simplify the consumption of complex offerings. Economics of scale will push prices lower while moving resources closer to your customers.

Product

Whether you're developing new features for Amazon or your local football club, or hoping to create the next Netflix, you need to be focused on your product. The patterns in this book have helped people develop and scale their products. Throughout this book, we'll build Minimum Viable Products (MVP) (http://www.startuplessonslearned.com/2009/08/minimum-viable-product-guide.html) on AWS. 

 Amazon will continue to use this pattern to develop new services, and you should too.

Data

The third part of this book looks at persistence. The item we are persisting is data. AWS offers a number of services to help us manage the volume, velocity, and variety of data. Other products will help identify the veracity and value in the flood. In Chapter 11, Data Processing - Handling Your Data Transformation, we'll use another set of tools to predict how our customers are going to react to new features. Continuous experimentation and measurement allow our product to become more successful.

 

Code

AWS also provides a number of open source projects as services (https://www.slideshare.net/AmazonWebServices/aws-reinvent-2016-open-source-at-awscontributions-support-and-engagementarc213). This allows them to leverage large active communities for product development and bug hunting. In the case of the AWS Software Development Kit for Python, Boto (https://github.com/boto/boto3), Amazon has taken over the project. We're not going to go into all of the benefits of open source here, but remember that many of the services included in our patterns are available as source code to everyone. 

Coding is going to be an important part of your product, the systems that run it, and the people that use it. We'll be writing code to allow the reuse and modification of our cloud patterns. To get into the spirit of things, we'll be using a cloud-based development tool that was originally written by the open source Mozilla foundation. A group of the developers decided to create a freemium model to provide support for people using their Integrated Development Environment (IDE), Cloud9. Amazon bought the company—another great pattern for applying to your product.

Utility computing

The cloud is an extension of the practice of co-locating hardware at a site away from your company. Historically, you would put your stuff closer to a fast connection in order to get your pages to load more quickly. Amazon stretched the market by allowing you to run your product on their hardware. They further innovated by creating a utility model that lets you pay for only what you need.

Anything as a Service

Obviously, IDE as a service was not the first service offering of AWS, but it fit well with their philosophy of Anything as a Service (often abbreviated as XaaS). By going beyond their initial offering of co-located virtual machines, Amazon has redefined existing infrastructure services. Providing configurable services and software implementations of hardware found in data centers has allowed them to develop complementary products such as load balancing, content delivery networks, and security plus, failover and replication solutions. Extending their internal tools or purchasing synergistic products such as Cloud9 has provided the full stack developer an opportunity to use a well thought out, low cost, pay for use, integrated suite.

 

AWS provides three primary categories of service offerings:

  • Infrastructure as a Service (IaaS)
  • Platform as a Service (PaaS)
  • Software as a Service (SaaS)
Infrastructure as a Service

IaaS can be described as a service that provides virtual abstractions for hardware, servers, and networking components. The service provider owns all of the equipment and is responsible for its housing, running, and maintenance. In this case, AWS provides Application Programming Interfaces (APIs), SDKs, and a web console for creating and modifying virtual machines; their network components; routers; gateways; subnets; load balancers; and much more. Where a user with a physical data center would incur charges for the hardware, power, and connectivity, this is removed by IaaS with a payment model that's per-hour (or per-use). Specific IaaS components will be the building blocks for our product. A discussion of the key pieces will take place in Chapter 2, Core Services - Building Blocks for Your Product.

Platform as a Service

While AWS doesn't provide a full-blown PaaS offering such as Cloud Foundry (https://www.cloudfoundry.org/) at the moment, it has a number of products that can simplify the product development experience. Its broad service catalog can also be used to compose a DIY platform. PaaS is described as the delivery of a computing platform, typically an operating system, programming language execution environment, database, or web server. With AWS, a user can easily turn a code base into a running environment without having to worry about any of the pieces underneath, such as setting up and maintaining the database, web server, or code runtime versions. It also allows your product to be scaled without having to do anything other than set policies through the configuration. We'll survey the services for on-demand expansion in Chapter 3, Availability Patterns - Understanding Your Needs. 

Software as a Service

The best interpretation for the SaaS model is on-demand software. The user need only configure the software to use and interact with it. The draw to SaaS is that there's no need to learn how to deploy the software to get it working in a larger stack, and generally the charges are per-usage-hour. In the case of AWS, much of the maintenance burden is also removed. 

The AWS suite is both impressive and unique in that it doesn't fall under any one of the cloud service models we described previously. Until AWS made its name, the need to virtualize an entire environment or stack was usually not an easy task and required integration of different providers offerings, each solving a specific part of the deployment puzzle. The cost of using many different providers to create a virtual stack might not be cheaper than the initial hardware cost for moving equipment into a data center. Besides the cost of the providers themselves, having multiple providers also created the problem of scaling in one area and notifying another of the changes. While making applications more resilient and scalable, this Frankenstein method usually did not simplify the problem as a whole.

Cloud continuum

When you build your first MVP for venture capitalists or an internal review board, we recommend you do it fast and focus on the user experience. In most cases, the monolith you create isn't going to be able to achieve web scale. The Cloud-Native Maturity Model is based on the twelve factor design principals. Its focus is on how to run your applications on the cloud: https://pivotal.io/replatforming

AWS supplies all of the services that are needed to launch cloud-native applications. We'll start with a cloud-friendly product, then in Chapter 8, Application Virtualization - Using Cloud Native Patterns for Your Workloads, we'll go into great depth. First, let's look at some of the trends driving cloud adoption.

Microservices

In most cases, the monolith you create is not going to be able to achieve web scale. As you break off pieces to refactor, you'll develop smaller services. This allows for the decoupling of components upgrade schedules, which speeds up product development. By adopting the continuous delivery pattern espoused by Jez Humble, you can deploy new features and measure their success quickly. We'll take a peek at these architectures in Chapter 5, Continuous Deployment - Introducing New Features with Minimal Risk. Isolation of your microservices will be the subject of Chapter 4, Security - Ensuring the Integrity of Your Systems. 

By providing easy to use, low cost SaaS options, AWS lets you focus on building software. High value feature development replaces low level toil. In Chapter 6, Ephemeral Environments - Sandboxes for Experiments, we'll present patterns that help foster collaboration between developers and operators. AWS composable services empower all team members to participate in the product development they know best.

Crazy new services

Amazon has introduced thousands of services since the introduction of the Elastic Cloud Compute (EC2), their first publicly available multi-tenant virtualization offering.

Note

Hardware virtualization or platform virtualization refers to the creation of a virtual machine (https://en.wikipedia.org/wiki/Virtual_machine) that acts like a real computer with an operating system. 

Most of them are enhancements to their MVP. Many of the services are tucked away in the management console. All of them are constantly monitored to ensure that customers are happy and to identify opportunities for new features. The container and serverless patterns in Chapter 8, Application Virtualization - Using Cloud Native Patterns for Your Workloads, promote self-contained service development for your product. Some, such as the Internet of Things (IoT) services, are out of scope for this book.

 

Free trials

If you don't have an AWS account, you can sign up for a free one at https://aws.amazon.com:

 

You'll need a valid email, phone number, and credit card (in case you go over your allotted capacity): 

 

The exercises in this book will use many of the free AWS services. We'll note when there are pay-per-use services that may be a better solution for certain classes of problems.

 

Huge catalog

AWS learned from the Amazon e-commerce team the value of partnerships. Just like you can shop the Crocs store on the parent side, you can rent an in-memory data grid powered by Apache Geode for a few hours through the AWS marketplace. The AWS marketplace provides an avenue where users can purchase or trial third-party images that provide configurable services such as databases and web applications. As a product developer, you can easily add your offerings to the exchange:

In the spirit of competition, a lot of the software in the marketplace offers competition to the AWS native services. Operations engineers may find it easier to move workloads to the cloud by reusing components that they're familiar with, such as F5 load balancers, instead of the AWS Elastic Load Balancers (Chapter 2, Core Services - Building Blocks for Your Product). Other products, such as the trend Micro Deep Security Agent, fill in gaps where AWS doesn't currently have a solution. Partnerships with Microsoft, VMware, and Tableau let you easily migrate internal workloads or extend your data center to the cloud.

 

Software defined, API driven

The adoption of a DevOps culture for your product team provides you with similar cooperation benefits. Getting operations engineers involved in product development helps eliminate hand-offs. Developers being on call for production issues acquaints them with troubleshooting practices and boosts empathy for users. By working together, the product team can establish a clear contact for use. Well-defined APIs are an excellent way to ensure that the contract can be met. AWS publishes extensive documentation for its interfaces, as well as defining service-level objectives for its products.

Benefits of moving to the cloud

For a new product, you should start development in the cloud to take advantage of Amazon's well-thought-out offerings. For existing companies, the AWS suite can handle almost everything a modern data center can while significantly reducing your operating costs. The shift suits small teams but, for mid-sized teams, the effort that's saved begins to outweigh the cost. In larger organizations, AWS can also help you get rid of some of the process bottlenecks that come with a data center. Requesting, provisioning, repairs, and scheduling downtime are all streamlined. There are many different answers to why moving to the cloud might be beneficial. We'll start at mid-sized because this is the size that usually includes the two teams that benefit the most. Let's take a look at our personas:

  • The software engineer
  • The operations engineer
  • The product owner
Software engineer

For a software developer, the ability to share code and delivery tools across teams is critical. AWS gives you the ability to enforce consistency across environments. Different versions of core languages or libraries can be avoided by having uniform templates. Chapter 5, Continuous Deployment - Introducing New Features with Minimal Risk, emphasizes how these good continuous delivery practices in the cloud can lower business risk. Chapter 3, Availability Patterns - Understanding Your Needs, goes deeper on making sure your product is available, while Chapter 4, Security - Ensuring the Integrity of Your Systems, covers how a secure groundwork can be inherited across products and services. We'll lay out how developers can easily build cost-effective products, which can grow as demand intensifies. Data management patterns in Chapter 10, Databases - Identifying which Type Fits Your Needs, and Chapter 11, Data Processing - Handling Your Data Transformation, will help software engineers understand the trade-offs in architecting their MVPs.

 

The ability to spin up entire environments for testing and simulation in AWS is amazing. Providing developers with their own sandboxes improves their ability to innovate and deliver new features at a high velocity. In the cloud-native world, a single environment can be encapsulated on a single EC2 instance. We'll explore how application virtualization can further improve developer productivity in Chapter 8, Application Virtualization - Using Cloud Native Patterns for Your Workloads.

Operations engineer

Throwaway environments are also be beneficial to the operations teams. In Chapter 6, Ephemeral Environments - Sandboxes for Experiments, we'll show you how to take centralized configuration, which describes AWS resources, map them into a dependency graph, and create an entire stack. This flexibility would be nearly impossible in traditional hardware settings and provides an on-demand mentality—not just for the base application, but also for the entire infrastructure, leading to a more agile core. 

The operations team's responsibility differs greatly from company to company, but it's safe to assume that the team is heavily involved with monitoring the applications and systems for issues and possible optimizations. AWS provides enough infrastructure for monitoring and acting on metrics, and an entire book could be dedicated to the topic. For an operations team, the benefit of moving to Amazon might justify itself only to alleviate all of the work involved in duplicating this functionality elsewhere, allowing the team to focus on creating deeper and more meaningful system health checks. There will be more coverage of operations in Chapter 7, Operation and Maintenance - Keeping Things Running at Peak Performance, while observability will be addressed in Chapter 12, Observability - Understanding How Your Products Are Behaving.

Product owner

As a product owner, the low cost of entry into very complex technology is a huge win. Chapter 3, Availability Patterns - Understanding Your Needs, will analyze the cost benefits of using AWS to provide global availability. The ability to move from hourly pricing to longer term subscriptions will give you a good baseline for product operating expenses. Avoiding inferior and counterproductive solutions will be the topic of Chapter 9, Antipatterns - Avoiding Counterproductive Solutions, and Chapter 13, Anti-Patterns - Bypassing Inferior Options. The shared responsibility model AWS provides will be explored in Chapter 4, Security - Ensuring the Integrity of Your Systems. By inheriting industry standard security controls, you can concentrate on user-centric product aspects.

 

Letting AWS manage the services at scale lets you avoid investing in dedicated support staff. Amazon has also adopted the freemium pattern for their own support. Start small and expand if you need more help:

 

Summary


Throughout this brief introduction to AWS, we learned not only about the background and industry shift that started the movement into utility computing, but also where AWS fits into it. We touched on the kinds of problems AWS can solve and where its services are competitive. There are countless unique processes to be solved with this ever-growing environment. Picking up consistent patterns throughout this book will help you strengthen products of many forms and protect against risk. In the following chapters, we'll go over some of the core services that are used to create basic design patterns. These patterns will be used to deliver more complex architectures using DevOps and data best practices in the DevOps Patterns and Persistence Patterns sections.

 

 

Further reading


A crash course in virtualization is available here: https://www.packtpub.com/application-development/introduction-virtualization-one-hour-crash-course-video.

An in-depth overview of the different cloud service models can be found in the following video: https://www.packtpub.com/virtualization-and-cloud/learn-cloud-computing-scratch-video.

About the Authors

  • Sean Keery

    Sean Keery began hacking obscure video game systems at the age of 13. Sean then developed interpersonal skills while teaching snowboarding. Nowadays, Cloud Foundry, choreography, containers and plenty of .io. Cluster deployments, IaaS independence, and his studies for a master's in data science keep Sean occupied. The daily commute is filled with podcasts and chipmunk bunny hops. Some family time, spicy food, a good book, and wrecking the latest toys keep Sean busy at home.

    Browse publications by this author
  • Clive Harber

    Clive Harber has been programming computers since he was 13, when Commodore Basic was all the rage and the internet, as an idea, was something that was only just starting to appear in films such as WarGames. Fast-forward a decade, he gained a master's degree in chemical engineering from University of Wales, Swansea, which he used briefly in a stint when he worked in a semi-conductor foundry making microchips. Not being totally satisfied with this, he decided to change fields and learn how to write stuff for the internet. Now, he runs his own consultancy, Distorted Thinking, providing services to organizations that find themselves in digital transition, whether that's software changes or infrastructure to cloud service migration.

    Browse publications by this author
  • Marcus Young

    Marcus Young obtained a degree in computer science and mathematics before getting involved in system administration and DevOps. He currently works in software automation using open source tools and technologies. His hobbies include playing ice hockey and brewing homebrew beer. He also enjoys hardware projects based on microcontrollers and single-board computers.

    Browse publications by this author

Latest Reviews

(2 reviews total)
Smooth purchase experience. Good deal as well.
Good AWS design resource!

Recommended For You

Book Title
Unlock this full book FREE 10 day trial
Start Free Trial