To properly introduce Google Cloud Build and the value it provides to its users, it’s important to review the value that automation brings to IT organizations for common workflows such as cloud infrastructure provisioning and software delivery.

Automating these tasks may be helpful in increasing developer productivity for organizations; doing so with a managed service enables this productivity at a lower cost of operation, allowing individuals and teams to focus on the business task at hand, rather than managing all the infrastructure that runs the automation. There has been an increase in automation needs for processing AI/ML types of workloads (https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning), which are beyond the typical developer workflows. We will be focusing on the developer automation workflow (that is, continuous integration) aspects of Cloud Build in this book.

In this chapter, we will review Google Cloud Build through this lens, specifically discussing the following:

• The value of automation
• Before there was the cloud
• Reducing toil with managed services
• Cloud-native automation with Google Cloud Build

• Data center and infrastructure concepts
• Public cloud concepts
• Software build concepts

The compilation of applications and services comes in all shapes and sizes. It may seem straightforward that code just becomes a packaged artifact, but for many scenarios, builds can have a number of complex steps with many dependencies. The steps involved in creating and testing an artifact may be manual, automated, or a combination of both to form a build pipeline. The following figure demonstrates an example build pipeline with a set of activities critical to the building of an application:

Figure 1.1 – Example build pipeline

Running these builds manually could potentially lead to careless mistakes with an outcome that may not be consistent or repeatable. When code is being built, it is very important to document even the smallest changes that were made to the build that resulted in something working as opposed to not working. This is where the use of a source code management (SCM) system, such as Git (https://git-scm.com), becomes critical in our overall pipeline. If a change was actually the result of a build step changed locally, not being able to repeat this can result in frustration and productivity loss.

This is especially relevant from the perspective of handing off work to a colleague. Having to understand and tweak a set of manual steps in a build would not be a good use of that colleague’s time, when they could instead be focused on the code at hand. The time of each member of an organization is valuable and it’s best to allow that individual to focus on being productive. This could be during a production outage, where time is best spent trying to fix the root cause of the outage rather than analyzing how to actually build the code. Depending on the impact of the outage, every second could have a monetary impact on the organization. In cases of a simple development handoff to a production outage, automation of a build would be very beneficial to the situation.

Imagine if a developer could solely focus on code development, rather than analyzing manual steps or difficult-to-execute builds. An organization might have automation in place, but it must be seamless for the developer in order to maximize productivity. This can be in the form of the following:

• Coding standards
• Boilerplate code
• Blueprints on how to use the pipeline

The preceding reference points could also apply to both what is considered the automation of the inner loop and the outer loop of software development. The inner loop of development typically consists of local development by a developer. Once code is completed in the inner loop, a merge request is created for addition to an integration branch. Once merged in an integration branch, this is where the typical build pipeline starts: the outer loop. Providing a starting point in the form of standards itself may not be automation; however, it could be baked into the configuration files. It may just be a starting point, a foundation that can also provide a level of flexibility for the developer to apply specific preferences.

Figure 1.2 – Example inner and outer loops

The ecosystem of tools and integration that has been built around Git has helped drive the importance of version controlling not only source code but also configurations that define a build pipeline. GitOps (https://www.weave.works/blog/the-history-of-gitops) primarily focuses on infrastructure as code (IaC), ensuring that a runtime environment represents configurations declaratively stored in Git. The common use of Git tooling across developer and operation teams reduces the amount of friction for onboarding, which makes GitOps also critical for end-to-end automation.

Automation helped reduce end-to-end deployment times for this organization: https://cloud.google.com/customers/reeport/.

Once automation is streamlined, the team that owns the pipeline is able to aggregate metrics in order to determine areas for improvement at scale. This becomes a more achievable task, compared to when builds are executed manually by each developer. As mentioned earlier, pipelines in an organization could also include manual steps. These metrics could identify patterns where manual steps could possibly be automated. Reducing manual steps would increase the efficiency of a pipeline while also reducing the potential human errors that can occur. There may be situations where manual steps aren’t automatable, but identification is key so that it can be considered in the future or to allow for teams to focus on other steps that can be improved.

This can reduce developers’ frustration and improve overall productivity across teams, which can benefit the organization in the following ways:

• Delivering features faster
• Reducing the amount of time it takes to resolve issues
• Allowing teams to focus on other business-critical activities
• Feedback for continuous improvement of the pipeline

The value of automation can help an organization in many aspects. While metrics can be manually gathered, they can be most effective when aggregated in an automated pipeline. Decisions can be made to determine the most effective changes to a build pipeline. The metrics gathered from frameworks in place, such as GitOps, can also help feed into improving the end-to-end pipeline, not just the automation of source code compilation. Continuous improvement becomes more achievable when an organization can use metrics for data-driven decisions.

There are a variety of tools on the market, ranging from open source to closed source and self-managed to hosted offerings, supporting build pipelines. Availability of the pipeline solution is critical in ensuring that code is built in a timely manner; otherwise, it may impact the productivity of multiple teams. Organizations may have separate teams that are responsible for maintaining the solution that executes the build pipeline.

Making sure there are enough resources

For self-managed solutions, the maintenance includes the underlying infrastructure, OS, tools, and libraries that make up the pipeline infrastructure. Scale is also a factor for build pipelines; depending on the complexity, organizations may have multiple concurrent builds occurring at the same time. Build pipelines need at least compute, memory, and disk to execute, typically referred to as workers within the build platform. A build pipeline may consist of multiple jobs, steps, or tasks to complete a particular pipeline to be executed. The workers are assigned tasks to complete from the build platform. Workers need to be made available so that they can be assigned tasks and such tasks are executed. Similar to capacity planning and sizing needs for applications, enough compute, memory, storage, or any other resource for workers must be planned out.

There must be enough hardware to handle builds at the peak. Peak is an important topic because in a data center scenario, hardware resources can be somewhat finite because it takes time to acquire and set up the hardware. Technologies such as virtualization have given us the ability to overprovision compute resources, but at some point, physical hardware becomes the bottleneck for growth if our build needs become more demanding. While an organization needs to size for peak, that also means that builds are not always running constantly at peak to make full use of the allocated resources. Virtualization, as mentioned previously, may help us with other workloads consuming compute during off-peak time, but this may require significant coordination efforts throughout the organization. We may be left with underutilized and wasted resources.

Figure 1.3 – Under-utilized resources when allocating for peak utilization

Who needs to manage all of this?

A team typically maintains and manages the build infrastructure within an organization. This team may be dedicated to ensuring the environment is available, resources are kept up to date, and new capabilities are added to support organizational needs. Requirements can come from all directions, such as developers, operators, platform administrators, and infrastructure administrators. Different build and pipeline tools on the market do help to facilitate some of this by offering plugins and extensions to extend capabilities. For instance, Jenkins has community contributions of 1,800+ plugins (https://plugins.jenkins.io/) at the time of writing this book. While that is quite an amount, this can also mean teams have to ensure plugins are updated and keep up to date with the plugins’ life cycles. For instance, if the plugin is no longer being maintained, what are the alternatives? If multiple plugins perform similar functions, which one should be chosen? A rich community is beneficial as popular plugins bubble up and may have better support.

While productivity is impacted as mentioned, not having enough capacity or improperly sizing the build infrastructure could lead to slower builds. Builds come in all shapes; they can run in seconds for some, while to others, they can take hours. For builds that take hours or a long time, this would mean the developer and many other downstream teams are waiting. Just because a build is submitted successfully, it does not mean it completes successfully too; it could possibly fail at any point of the build, leading to lost time.

The team that is responsible for managing the build infrastructure may also be likely responsible for maintaining a service-level agreement (SLA) to the users of the system. The design of the solution may also have been designed by another team. As noted earlier, if builds are not running, there may be a cost associated because it impacts the productivity of developers, delays in product releases, or delays in pushing out critical patches to the system. This needs to be taken into account when self-managing a solution. While this was the norm for much of the industry before there was the cloud, in an on-premises enterprise, vendors developed tools and platforms to ease the burden of infrastructure management. Managed service providers (MSPs) also provided tooling layers to help organizations manage compute resources, but organizations still had to take into account resources that were being spun up or down.

Security is a critical factor to be considered when organizations need to manage their own software components on top of infrastructure or the entire stack. It’s not just the vulnerability of the code itself being built, but the underlying build system needs to be securely maintained as well. In the last few years, a significant vulnerability was exposed across all industries (https://orangematter.solarwinds.com/2021/05/07/an-investigative-update-of-the-cyberattack/).

Eventually, when public cloud resources were available, much of the similar patterns discussed could be used – in this case, infrastructure as a service (IaaS) offerings in a cloud provider for handling the compute infrastructure. These eased the burden of having to deal with compute resources, but again, like MSPs, the notion of workers had to be determined and managed.

Organizations have had to deal with the build software pipeline platform regardless of whether the infrastructure was managed on-premises in their data center, in a co-location, or by an IaaS provider. It is critical to ensure that the platform is available and has sufficient capacity for workers to complete associated tasks. In many organizations, this consisted of dedicated teams that managed the infrastructure or teams that wore multiple hats to ensure the build platform was operational.

In the previous section, we discussed the efforts involved in maintaining a platform for building applications and services. Many of the activities described in making sure the environment is always up and running could involve some toil. For example, Google’s SRE handbook (https://sre.google/sre-book/eliminating-toil/) goes further into the elements of IT tasks that could be considered toil.

If we are able to avoid toil and know that a provider manages the underlying build infrastructure, we are able to focus on what is more important, the application that helps drive our business. This is one of the goals of managed services, letting the provider handle the underlying details, providing a consistent syntax that becomes the common language between teams, providing compute resources as needed, and not billing when the service is not being utilized.

It is one less component of a build pipeline to consider as the provider is maintaining the underlying infrastructure and they are able to provide the team with scale when needed at any given time. The MSP would be responsible for making sure that there are enough workers in order to execute all the jobs in the build pipeline. However, managed services could also be seen as a form of lock-in to a particular vendor or cloud provider. In most cases, a managed service typically has the best integration to services provided by the offering provider. This is where adding additional capabilities are much more streamlined, but not limited, to the following:

• Triggering mechanisms
• Secrets management
• Securing communication and data transfer between integrated services
• Observability

The integrations are there to help save time and, in reference to the original theme of this book, allow an organization to focus on the application at hand. Though important topics are noted in the preceding section, the importance of a managed service to allow flexibility and a way to integrate third-party-specific capabilities is also important when choosing a managed service.

As noted earlier, if an organization chooses to manage their own build solution, they may be responsible for the availability of the platform. In the case of a managed service, the provider is responsible for the availability and may establish an SLA with the customer using its services. The customer would have to make the determination of whether the communicated SLA is acceptable to the business.

Managed services offered by providers reduce the amount of toil to keep the build platform up and running. They allow teams at an organization to focus on critical business functions or revenue-generating activities. In the case of on-premises, not having to wait for hardware procurement or setup allows for maximum business flexibility. The provider would be responsible for making sure the platform is up to date and allowing for fast-paced groups within the organization to experiment with newer capabilities.

This brings us to Cloud Build, a Google Cloud offering that is a serverless platform. This means that teams are not responsible for maintaining, scaling workers, or assuring the availability of the platform. Cloud Build is a managed service as well because Google Cloud is responsible for ensuring that the service is available to the customer. This allows an organization to focus purely on their application code and build pipeline configuration. Google Cloud is responsible for scaling the platform as needed and the billing is priced (https://cloud.google.com/build/pricing) per build minute.

Cloud Build takes advantage of the inherent security of Google Cloud, where the API is only accessible by individuals and automation systems that are granted access. This will be a common theme throughout the book because Cloud Build is a serverless platform. We are able to take advantage of what Google Cloud has to offer in areas such as security, compute resources, integrations, and API- and language-specific client SDKs to access the platform.

As noted in previous sections, workers handle the steps (or jobs) in a build pipeline and Cloud Build is responsible for making sure there are workers available to complete the pipeline from end to end. Cloud Build offers workers in different flavors, noted as a default pool and a private pool. In a default pool, the workers are hosted by Google and not visible to the end user. While the build is running, it would have access to the public internet, but not privately hosted resources. Depending on a build that you need, this may be all that is necessary. However, if during a build you need access to resources on a private network, such as dependencies, localized datasets, higher concurrency limits, more machine type configurations, or integration testing with a private environment, the option would be a private pool. Private pools allow for connectivity to private networks using Google Cloud Platform (GCP) Virtual Private Cloud (VPC) peering capabilities. The following is a table comparing capabilities of default and private pools:

Figure 1.4 – Feature comparison between default and private pools

The data in the figure is up to date as of 2022-06-16 ( https://cloud.google.com/build/docs/private-pools/private-pools-overview).

The steps in a Cloud Build configuration typically remain the same regardless of the type of pool selected: default or private.

In Cloud Build, a single worker is responsible for handling all the steps in a pipeline. The worker and machine type (compute, memory, and storage) can be configured, providing flexibility for each build process, depending on the criticality. Exposing the build process to a machine with more resources may allow for faster builds, where machine resources have been identified as a bottleneck for slower builds.

This is a significant advantage of a serverless platform in Cloud Build, where the options are defined as configuration options. It is the responsibility of Google Cloud to provide the resources based on what is specified in the valid configuration. The flexibility of the cloud is available to the team, by specifying different machine types for the workers based on the pipeline requirements.

If you recall the example of Jenkins (in the Who needs to manage all of this? section), it provides plugins to add integrations or functionality to complete a pipeline. In the case of Cloud Build, each individual step consists of a container image known as a cloud builder. This provides a significant amount of flexibility as it allows for using Google Cloud-provided builders, community-developed builders, or custom builders, which can be container images created in-house. These custom builders can be any container image that executes a script or binary, which can then be incorporated as a build step.

Organizations may place different restrictions on the residency of code; for example, it may not reside on a developer’s workstation, it may only reside on organizationally maintained SCM systems. Cloud Build can integrate with privately hosted/non-GCP solutions, such as GitHub Enterprise. By using private pools, the worker of the build can interact with resources within a private network. This is very important as it allows Cloud Build to support many different scenarios that an organization may have.

Earlier in this chapter, we discussed the value of automation, which includes the automation of build steps, but also how a build itself can be triggered. Triggering a build can be based on a cron schedule using Cloud Scheduler (https://cloud.google.com/build/docs/automating-builds/create-scheduled-triggers), async messages through Cloud Pub/Sub (https://cloud.google.com/build/docs/automating-builds/create-pubsub-triggers), and webhooks, but is usually driven by a commit of code to an SCM system. Webhooks open the door to many other options; for instance, a build can also be invoked with an API or CLI call through an existing tool. This makes the integration and automation with existing components possible.

GCP service integrations

Integration with existing components is critical, but another advantage of Cloud Build is its native integration with Google Cloud services:

• Artifact Registry – store built artifacts (that is, language-specific, container images): https://cloud.google.com/artifact-registry/docs/configure-cloud-build.
• Secret Manager – natively manage sensitive information: https://cloud.google.com/build/docs/securing-builds/use-secrets.
• VPC Service Controls – mechanism to protect against data exfiltration, for instance, where a build can be triggered from or policies that require only private pools be used: https://cloud.google.com/build/docs/private-pools/using-vpc-service-controls.
• Binary Authorization (preview) – sign image attestations at build time with a provided attestor: https://cloud.google.com/build/docs/securing-builds/use-provenance-and-binary-authorization.

If you recall earlier, it was noted that a Cloud Build worker is responsible for executing the steps designated in the configured pipeline. The identity of this worker is established by a GCP resource known as a service account. Cloud Build has a default service account that it uses, or a user-specified service account may also be specified.

This book focuses on the build aspect of Cloud Build and a built resource does not always need to be deployed when using Cloud Build. However, Cloud Build also provides integrations with instructions and images to deploy built container images to a GCP runtime platform, such as the following:

• Google Kubernetes Engine
• Anthos-enabled clusters
• Cloud Run
• App Engine
• Cloud Functions
• Firebase

Organizations are able to use Cloud Build for both the code build and IaC automation capabilities to provide more autonomy to their teams. One example is noted here: https://cloud.google.com/customers/loveholidays.

As a serverless platform, visibility into what is happening with the build is important. Audit logs and build logs are stored by default in GCP’s Cloud Logging service. Audit logs cover API calls to the Cloud Build service, such as creation and cancelation. The actual build logs are stored by default in both Cloud Logging and a Google-created Cloud Storage bucket. The logs from Cloud Build are configurable to destinations of your choice within GCP (that is, only Google Cloud Storage, only Cloud Logging, or log buckets in other projects).

Cloud Build, being a serverless platform, also offers an SLA. As with GCP services, Cloud Build has a monthly uptime percentage of 99.95% (at the time of this writing this book – https://cloud.google.com/build/sla). This is noted as the service-level objective (SLO). Please refer to the previous link for additional details related to the SLA. For services that are provided by a cloud provider, this is an important measurement factoring in the reliability of the service. As noted in an earlier section, if a team were responsible for managing build infrastructure, they would also need to offer a reliability metric. The built solution would typically have to be designed, implemented, and tested to meet the offered metric. In GCP and its applicable services where SLAs are offered, GCP has this responsibility as a service provider.

Organizations have come a long way from having to manage their own infrastructure and their own pipeline software. Service providers provide a way for the infrastructure to be managed in their data center to offload the hardware, the OS, and sometimes even the management of the build pipeline platform itself. The next iteration is serverless, which is what we’ve been talking about, where it’s just an API that is exposed for clients to use. Define your desired configuration and execute the API in your desired mechanism and language.

Cloud Build is a serverless build pipeline platform that allows organizations to focus on the code at hand. Google Cloud takes care of ensuring the platform is available when needed and that there is sufficient compute for the workers. Each step, represented as a container image, provides the utmost flexibility in handling pipeline needs.

Thus far, we’ve covered how build pipeline automation can make our lives easier while also potentially preventing error-prone mistakes. Metadata and metrics can help us make data-driven decisions on making our pipelines more efficient. The build platforms need to be managed by someone, whether it’s an IT team, an outsourced team, or a service managed by a cloud provider. If someone else manages it, it may allow us to focus more on our business needs rather than pipeline platform needs. Cloud Build is a serverless platform providing us with worker pool elasticity and flexibility using container images as build steps.

Stay tuned for further chapters as we dig into how to get started and deeper details on the features of Cloud Build and run through examples of a few scenarios.

In the next chapter, we will focus on getting our Cloud Build environment set up.