Cloud computing introduced a brand-new way of managing the infrastructure.

As the demand for the AWS cloud grew, the usual routine and operational tasks became troublesome. The AWS cloud allowed any type of business to rapidly grow and solve all the business needs regarding compute power; however, the need to maintain a certain stack of resources was hard.

DevOps culture brought a set of methodologies and ways of working, and one of those is called infrastructure as code. This process is about treating your infrastructure—network, virtual machines, storages, databases, and so on—as a computer program.

AWS CloudFormation was developed to solve this kind of problem.

You will already have some working knowledge of CloudFormation, but before we dive deep into learning advanced template development and how to provision at scale, use CloudFormation with CI/CD pipelines, and extend its features, let's quickly refresh our memory and look again at what CloudFormation is and how we use it.

In this chapter, we will learn the following:

• The internals of AWS CloudFormation
• Creating and updating a CloudFormation stack
• Managing permissions for CloudFormation
• Detecting unmanaged changes in our stack

The code used in this chapter can be found in the book's GitHub repository at 
https://github.com/PacktPublishing/Mastering-AWS-CloudFormation/tree/master/Chapter1.

Check out the following video to see the Code in Action:

https://bit.ly/2WbU5Lh

AWS services consist of three parts:

• API
• Backend
• Storage

We interact with AWS by making calls to its API services. If we want to create an EC2 instance, then we need to perform a call, ec2:RunInstances.

When we develop our template and create a stack, we invoke the cloudformation:CreateStack API method. AWS CloudFormation will receive the command along with the template, validate it, and start creating resources, making API calls to various AWS services, depending on what we have declared for it.

If the creation of any resource fails, then CloudFormation will roll back the changes and delete the resources that were created before the failure. But if there are no mistakes during the creation process, we will see our resources provisioned across the account.

If we want to make changes to our stack, then all we need to do is update the template file and invoke the cloudformation:UpdateStack API method. CloudFormation will then update only those resources that have been changed. If the update process fails, then CloudFormation will roll the changes back and return the stack to the previous, healthy, state.

Now that we have this covered, let's start creating our stack.

I'm sure you've done this before.

We begin by developing our template first. This is going to be a simple S3 bucket. I'm going to use YAML template formatting, but you may use JSON formatting if you wish:

MyBucket.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: This is my first bucket
Resources:
  MyBucket:
    Type: AWS::S3::Bucket

Now we just need to create the stack with awscli:

$ aws cloudformation create-stack \
                     --stack-name mybucket\
                     --template-body file://MyBucket.yaml

After a while, we will see our bucket created if we go to the AWS console or run aws s3 ls.

Now let's add some public access to our bucket:

MyBucket.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: This is my first bucket
Resources:
  MyBucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead

Let's run the update operation:

$ aws cloudformation update-stack \ 
                     --stack-name mybucket \
                     --template-body file://MyBucket.yaml

To clean up your workspace, simply delete your stack using the following command:

$ aws cloudformation delete-stack --stack-name mybucket

Let's now look at the CloudFormation IAM permissions.

We already know that CloudFormation performs API calls when we create or update the stack. Now the question is, does CloudFormation have the same powers as a root user?

When you work with production-grade AWS accounts, you need to control access to your environment for both humans (yourself and your coworkers) and machines (build systems, AWS resources, and so on). That is why controlling access for CloudFormation is important.

By default, when the user runs stack creation, they invoke the API method cloudformation:CreateStack. CloudFormation will use that user's access to invoke other API methods during the stack creation.

This means that if our user has an IAM policy with an allowed action ec2:*, but attempts to create an RDS instance with CloudFormation, the stack will fail to create with an error, User is unauthorized to perform this action.

Let's try this. We will create an IAM role with ec2:*, assume that role, and try to create the same bucket stack:

Important note

We already have an IAM user Admin in our AWS account and we will add that user as a principal.

MyIamRole.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: "This is a dummy role"
Resources:
  IamRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: AllowAssumeRole
            Effect: Allow
            Principal:
              AWS:
                - !Join
                  - ""
                  - - "arn:aws:iam::"
                    - !Ref "AWS::AccountId"
                    - ":user/Admin"
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
        - "arn:aws:iam::aws:policy/AWSCloudformationFullAccess"
Outputs:
  IamRole:
    Value: !GetAtt IamRole.Arn

If we create this stack, assume that role, and try to create the previous mybucket stack, it will fail to create with an error. Let's take a look:

$ aws cloudformation create-stack \
                     --stack-name iamrole \
                     --capabilities CAPABILITY_IAM \
                     --template-body file://IamRole.yaml
$ IAM_ROLE_ARN=$(aws cloudformation describe-stacks \
                                    --stack-name iamrole \
--query "Stacks[0].Outputs[?OutputKey=='IamRole'].OutputValue" \
--output text)
$ aws sts assume-role --role-arn $IAM_ROLE_ARN \
                      --role-session-name tmp
# Here goes the output of the command. I will store the access credentials in the env vars
$ export AWS_ACCESS_KEY_ID=… 
$ export AWS_SECRET_ACCESS_KEY=…
$ export AWS_SESSION_TOKEN=…
$ aws cloudformation create-stack \
                     --stack-name mybucket \
                     --template-body file://MyBucket.yaml

We will see the following error on the AWS console:

Figure 1.1 – CloudFormation console – stack events

On the other hand, we cannot provide everyone with an AdminAccess policy, so we need to find a way to use CloudFormation with the necessary permissions while only letting CloudFormation use those permissions.

CloudFormation supports service roles. Service roles are the IAM roles that are used by different AWS services (such as EC2, ECS, Lambda, and so on). CloudFormation service roles are used by CloudFormation during stacks and StackSets operations—creation, update, and deletion:

1. Let's create a specific role for CloudFormation:

   CfnIamRole.yaml

   AWSTemplateFormatVersion: "2010-09-09"
   Description: "This is a CFN role"
   Resources:
     IamRole:
       Type: AWS::IAM::Role
       Properties:
         AssumeRolePolicyDocument:
           Version: 2012-10-17
           Statement:
             - Sid: AllowAssumeRole
               Effect: Allow
               Principal:
                 Service: "cloudformation.amazonaws.com"
               Action: "sts:AssumeRole"
         ManagedPolicyArns:
           - "arn:aws:iam::aws:policy/AdministratorAccess"
   Outputs:
     IamRole:
       Value: !GetAtt IamRole.Arn

2. We create this stack for the service role and obtain the CloudFormation role ARN:

   $ aws cloudformation create-stack \
                        --stack-name cfniamrole \
                        --capabilities CAPABILITY_IAM \
                        --template-body file://CfnIamRole.yaml
   $ IAM_ROLE_ARN=$(aws cloudformation describe-stacks \
                                       --stack-name cfniamrole \
   --query "Stacks[0].Outputs[?OutputKey=='IamRole'].OutputValue" \
   --output text)

3. Now we run the creation of the stack, which will use our role, specifying the Role ARN:

   $ aws cloudformation create-stack \
                        --stack-name mybucket \
                        --template-body file://MyBucket.yaml \
                        --role-arn $IAM_ROLE_ARN

4. After a while, we can verify that our stack has been created, and we see our bucket!

   $ aws s3 ls
   # Output
   2019-10-16 14:14:24 mybucket-mybucket-jqjpr6wmz19q

   Before we continue, don't forget to clean your account:

   $ for i in mybucket iamrole cfniamrole; do aws cloudformation delete-stack --stack-name $i ; done

   Important note

   Note that in the preceding example, we provide the CloudFormation role with an AdminPolicy (the one that is provided by AWS by default).

In production-grade systems, we want to allow CloudFormation only those privileges that are required for the stack.

There are two permission schemas that are being applied for CloudFormation roles:

• We have a certain list of services that we can use (for example, EC2, RDS, VPC, DynamoDB, S3, and so on).
• Each template/stack combination will use only those services it needs—for example, if we declare Lambda functions with Simple Notification Service (SNS), then we should create the role with policies only for Lambda and SNS.

CloudFormation as a service often refers to the term state. The state is basically inventory information that contains a pair of values: the logical resource name and the physical resource ID.

CloudFormation uses its state to understand which resources to create or update. If we create a stack with a resource with a logical name foo, change the property of this resource (foo) in a template, and run an update, then CloudFormation will change the corresponding physical resource in the account.

CloudFormation has a set of limitations. For example, it will not update the stack if we do not introduce changes to it. If we perform manual changes to the resource, then CloudFormation will change them only when we make changes to the template.

Developers had to rethink their way of managing the infrastructure once they started using CloudFormation, but we will get to that in the later chapters. For now, we would like to show you a feature that doesn't solve problems of manual intervention, but at least notifies us when they happen. This feature is called drift detection.

For this example, we will use the same template (Dummy IAM Role) as we did in the previous section:

$ aws cloudformation create-stack \
                     --stack-name iamrole \
                     --template-body file://IamRole.yaml \
                     --capabilities CAPABILITY_IAM

After a while, we see our stack created:

Figure 1.2 – CloudFormation console

Note the link on the right called Drifts. If we follow that link, we will see the Drifts menu and under that Drift status: NOT_CHECKED. At the time of writing, we will have to run drift detection manually, so we need to run drift detection on our stack. After a while, we will see that everything is all right:

1. I'm going to run Detect stack drifts and verify that my stack is compliant:
   

   Figure 1.3 – CloudFormation console – drifts

2. Now what we will do is add an extra policy to our role and rerun drift detection:

   $ ROLENAME=$(aws cloudformation describe-stack-resources --stack-name iamrole --query "StackResources[0].PhysicalResourceId" --output text)
   $ aws iam attach-role-policy --role-name $ROLENAME --policy-arn "arn:aws:iam::aws:policy/AdministratorAccess"

3. We can now detect drift again:
   

   Figure 1.4 – CloudFormation console – drift detected

4. If we check our IamRole resource and click on View drift details, we will see what exactly has been changed and differs from CloudFormation's state:

Figure 1.5 – CloudFormation console – actual modification

Now we have two options: either roll back the change to the resource manually or add any dummy property to the template and run update-stack.

We've learned about CloudFormation drifts, how to run its drift detection, and the actions that must be taken afterward. But don't worry—we will revisit drifts again in the following chapters.

In this refresher chapter, we refreshed our memory as to what CloudFormation is, how we create and update stacks, why service roles are important, and how to implement them. We also remembered what drifts in CloudFormation are, when they occur, and how to detect them.

While this is an introductory chapter, we covered the fundamental building blocks of CloudFormation. In the following chapters, we will use service roles and drift detection again, but first, we need to deep dive into the internals of the CloudFormation template, which we are going to do in the next chapter.

1. Which API method is invoked when we create a CloudFormation stack?
2. What is a CloudFormation service role?
3. Which IAM policies are used if we do not specify the CloudFormation service role?
4. How is the information about stack resources stored in CloudFormation?
5. What happens if we delete the resource created by CloudFormation and try to create the same stack?
6. What happens if we delete the resource created by CloudFormation and try to update the same stack?
7. Why can't CloudFormation recreate the deleted resource?

• CloudFormation service roles: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-servicerole.html
• Drift detection in CloudFormation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-drift.html