Setting Up Self-Hosted Runners

In the vast ecosystem of CI/CD, GitHub Actions stands as one of the most integrated and versatile platforms for automation. While GitHub Actions offers runners that execute your workflows in GitHub-hosted environments, there are scenarios where you might want more control over an environment, need specific hardware, want to utilize private network resources, improve build time, or need to cut costs. Enter self-hosted runners. As the name implies, self-hosted runners are automation environments you host yourself. This allows you to fine-tune, customize, and control the exact setting in which your GitHub Actions workflows run. This flexibility can be crucial for certain types of projects and environments.

In this chapter, we’ll embark on a journey to explore the ins and outs of setting up self-hosted runners for GitHub Actions. We’ll begin by setting up an instance on a local machine, giving you a front-row seat to the nitty-gritty of the...

Technical requirements

To follow along with the hands-on material in this chapter, you will need to follow the steps in the previous chapter or access the resources from it, referring back to it if anything is unclear to you. We will expand on the previously used Azure cloud subscription, so make sure that’s still available.

We’re also going to install a local runner on your machine, so make sure that your local PC meets the hardware requirements based on our OS and architecture. You can do so by verifying the latest requirements at https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#supported-architectures-and-operating-systems.

We’ll also be installing Minikube locally; please follow the instructions at https://minikube.sigs.k8s.io/docs/start/ for a step-by-step guide on how to set this up on your given machine type. Validate the setup by testing a local application using the Ingress add-on...

Exploring self-hosted runners

At its core, a runner in the GitHub Actions context is a computational environment where a workflow runs. GitHub provides hosted runners, which are pre-configured virtual environments managed by GitHub.

These environments are ephemeral, meaning they are provisioned when needed and discarded after use. But GitHub Actions also provides an option for self-hosted runners, which are, in essence, machines or virtual environments where you can run workflows.

Self-hosted runners can be any machine – a server in your local data center, a virtual machine in the cloud, or even a Raspberry Pi sitting on your desk. They provide a bridge between the automation capabilities of GitHub Actions and your environment, allowing workflows to interact more closely with custom or proprietary systems.

There are several reasons why you might opt for a self-hosted runner:

• Customization: With self-hosted runners, you have full control over the environment...

Exploring ARC

The rise of Kubernetes has influenced various CI/CD tools to integrate with its powerful orchestration capabilities. The GitHub ARC is an evolution in this direction, enabling users to deploy GitHub Actions runners on Kubernetes clusters.

This offers better scalability and harnesses Kubernetes’ inherent resiliency and management features. Some of the benefits of taking this route are as follows:

• Efficiency: Rather than over-provisioning to handle peak loads, runners can be dynamically scaled, ensuring resources are used efficiently.
• Resiliency: ARC takes advantage of Kubernetes’ self-healing features. If a runner crashes, Kubernetes ensures another is spawned to maintain the desired count.
• Uniformity: You ensure consistency across deployments by defining runner specifications as Kubernetes manifests. It’s clearer to manage, version, and replicate.
• Cost savings: Dynamic scaling means you only use resources when you need them...

Running ARC locally

In the Technical requirements section, we guided you through the Minikube setup. While it’s crucial to have Minikube up and running, our subsequent mission is to embed ARC into the Kubernetes cluster to address action run demands. However, before we embark on this journey, we must acquaint ourselves with the elements we’re gearing up to implement, ensuring clarity about terminology and core concepts.

We’re setting up a local Kubernetes environment to run tasks on temporary, or ephemeral, infrastructure. Ephemeral essentially means it exists briefly and not indefinitely. This strategy helps us use resources efficiently, ensuring that compute, memory, and storage aren’t tied up by largely idle GitHub Actions runners. There are times when a more lasting setup is necessary, such as when some Docker images need more time to start or when app licensing terms dictate. However, it’s essential to consider the specifics of each situation...

Using the cloud for your runs

Modern software development demands efficiency and reliability, making the choice of infrastructure critical. Azure Kubernetes Service (AKS) is one of the leaders in managed Kubernetes services. By using a cloud-hosted platform such as AKS, teams can effortlessly scale resources to match their growing workload demands, ensuring that resource allocation is efficient and cost-effective. Beyond scalability, the reliability of AKS’s built-in redundancy and high availability ensures that deployment cycles continue unhindered, even if part of the infrastructure faces challenges.

In this section, we will install ARC on our AKS instance, which we’ll create from Bicep. We’ll also bolster our security and move to a GitHub app instead of a PAT.

Setting up Kubernetes using Bicep

This section will involve deploying SSH keys for VMs, using them to power our Kubernetes cluster, which we’ll deploy using Bicep. To get us through this...

Advanced techniques with ARC

This section is dedicated to those who wish to transition from just implementing ARC to mastering it. We will navigate through several advanced concepts and techniques that will equip you to optimize, monitor, and troubleshoot ARC, ensuring that it aligns perfectly with your specific use cases. From understanding the nuances of scaling and monitoring metrics, adapting ARC to function within the confines of a corporate proxy, leveraging runner labels for better workflow orchestration, to even customizing your runner environment with your own image – we’ll cover it all. We’ll also touch upon best practices for scaling down and proactive monitoring, ensuring that you’re not just running your workflows but also optimizing them for performance, resilience, and cost.

So, whether you’re looking to gain deeper insights into your runners, adapt ARC to specialized environments, or simply wanting to optimize for cost and efficiency...

Housekeeping

Before concluding today’s chapter, it’s crucial to shut down the Azure-based resources we’ve set up. This can be accomplished by turning off the Kubernetes cluster in the Azure portal. Remember to reactivate it when you’re ready to begin the next chapter. It’s advisable not to leave it running on your trial account, as this could surpass the free quota available in your subscription.

Let’s now wrap this chapter up.

Summary

This chapter delved deep into self-hosted runners and how to host them. We began by understanding self-hosted runners’ significance and versatility in deployment options. This led us to the intricacies of creating runner groups, shedding light on their establishment at different GitHub levels – at the repository, organization, or even the broader enterprise level. Taking a hands-on approach, we walked through setting up a runner right on our local PC, providing a tangible connection to the concepts discussed. As we navigated the landscape of GitHub Actions, ARC emerged as a pivotal tool. To ensure a comprehensive grasp, we intertwined our exploration of ARC with foundational insights into Kubernetes, drawing parallels with a toy factory to make the intricate workings more relatable.

Building on this foundation, we ventured into the realm of Minikube, a tool that simplified our Kubernetes interactions. Helm further enriched our journey, which we leveraged to...