Containerization on GPU-Enabled Platforms

In this new chapter, we will continue our exploration with GPUs while specifically focusing on user accessibility. You will learn about different environments to choose from when setting up a GPU-based programmable platform. These environments will be compared and discussed to help you decide on the most suitable one pertaining to usability and different situations or conditions. Following this, system-wide and virtual environments will be explained. Their advantages and disadvantages will also be explored.

Virtualenv, which is similar to Conda, will be discussed as an example of a closed environment separate from the base system. We will also look at a scenario where both system-wide and Virtualenv packages can co-exist and work together when accessed from a virtual environment.

Exploring further, containers such as Docker and Kubernetes...

Programmable environments

So far in our last three chapters, we have learned about using Conda to implement different Python-based programming modules for GPU computing. We also learned about system-wide installations for this. As you might already know, Conda can be used system-wide, but it is not primarily meant to be used so in practice. Rather, it is focused toward creating and managing virtual local environments invisible to the remaining outer part of the OS. In this section, we explore both system-wide and virtual ways of accessibility through the exploration of different programmable environments.

Programmable environments – system-wide and virtual

A system-wide programmable environment is one that has packages...

System-wide (open) environments

It is obvious that a system-wide environment, in general, is the base accessible system of an OS. Whatever packages are installed, compiled, or built are affected on the main system path. In a Linux system, there are two locations in general where an application installer can set its path and configuration for visibility within the OS.

$HOME directory

In this location (/home/directory/), you can find one particular file called .bashrc that stores paths specific to the user. In addition, there is also a directory named .config. Many applications install system-wide, store configuration information in the form of a .conf file in this directory and paths in the .bashrc file. These applications...

Virtual (closed) environments

In a virtual environment, the base accessible system is contained within itself. Whichever packages are installed, compiled, or built are affected in the local virtual directory, leaving the outer OS configuration untouched. In a Linux system with a virtual package manager, there are also two locations in general where an application installer can set its path for visibility within the OS. Let's see how it differs for virtual environments.

$HOME directory

Virtual package managers such as Conda also use this location (/home/directory/) to enable path and configuration access. The path to the Conda executable is stored in the .bashrc file. Many applications, installed locally, store configuration...

Virtualization

Now that we know about the basic differences between open and closed environments, along with their advantages and disadvantages, let's proceed further into the virtualization concept. This is essential before we move on to our primary discussion—containerization, which is the main theme of this chapter.

As you might be well aware now, virtualization is a way to run applications and operating systems in an isolated location, allocated on a physical hard disk and RAM. Physical hard disk space and RAM can be use to allocate resources and create multiple virtual environments. The physical space allocation is referred to as the host, whereas the virtual space allocations are referred to as guests.

In the earlier chapters, we discussed installation and configuration steps for different Python modules with Conda. All of those steps were in fact, ways to virtualize...

Local containers

Containerization is a concept that takes the idea of virtualization many steps further. It is the process of providing readily deployable applications and their dependencies, preinstalled and preconfigured within individual containers at operating system level. These containers are all isolated user space instances:

By Natlibfi-arlehiko - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=69328769

With containerization, you no longer require even setting up a virtualized environment, as all of those measures are already taken care of. You can quickly get started with a production-ready interface. Hence we can conclude that containerization boosts productivity by a huge margin.

When we set up containers in a locally accessible system, it is referred to as local containerization. In this section, we are going to continue our discussion on local...

Cloud containers

Unlike local containers, cloud containers are available on the server side that can be remotely accessed from anywhere. Physical maintenance costs are heavily reduced in such scenarios.

One great example of cloud based containerization is Google Colaboratory (Colab) that allows execution of Linux Terminal commands, in addition to Python-based development and testing on Jupyter notebooks. For a brief overview on Jupyter Notebook and Jupyter Lab, please refer to Chapter 5, Setting Up Your Environment for GPU Programming.

The GitHub page for Colab's backend container can be found at https://github.com/googlecolab/backend-container.

Our primary focus in this section is Google Colab on the cloud, specifically because, since April 2019, it offers free access to an NVIDIA Tesla T4 Tensor Core GPU for AI inference! It is actually a card that belongs to the Turing...

Summary

In this chapter, programmable environments were discussed with a perspective on system-wide and virtual environments. Scenarios where a particular option is preferred were discussed. The directory structures of both system-wide and virtual environments were explored, in addition to their advantages and disadvantages. The containerization concept was introduced as an evolution from virtualization. Finally, local and cloud containers were explored in detail with a hands-on approach.

You are now familiar with the different environments to choose from in order to set up and use a development platform with GPUs. With Virtualenv and VirtualBox, you can now set up your own isolated development environments. The benefits of using Virtualenv will help prepare you to customize future preferences when setting up a closed programmable environment. Depending upon your usability requirements...