Working with CUDA and PyCUDA

With this chapter, we begin our hands-on Python-based experience with GPUs via a computing approach. From this perspective, it is important to understand that a computing approach primarily focuses on computational problem-solving as a series of steps: problem outline, problem solution, programming the underlying solution to the problem, and, finally, testing its effectiveness. We will follow this approach for all our computing problems in this book that we try to solve through GPU-accelerated programming.

C programming enthusiasts will be encouraged to invoke NVIDIA GPUs within their program code with CUDA-C, while Python programming enthusiasts will be motivated to use PyCUDA to invoke NVIDIA GPUs within their program code. We first start by understanding how a CUDA-C program works. The fundamental concepts behind a CUDA program and its components...

Technical requirements

Check out the following video to see the Code in Action:

http://bit.ly/2WAeVTy

Understanding how CUDA-C/C++ works via a simple example

By now, you must be aware of the computational advantages of CUDA C/C++ as per our earlier discussions. C/C++ coupled with CUDA allows you to modify parts of your source code to accelerate your computational results. The primary steps necessary for implementing CUDA code will be explored through a GPU program.

Please manually type in the code used in this book on your IDE from this point onward. Directly copying and pasting from the PDF will ruin the indentations in the code and make it unready to deploy.

First, let's look into the following conventional C++ program that multiplies two array elements using double precision. We'll run the kernel on 500 million elements on the CPU. All the elements of the p and q arrays are set to 24 and 12 respectively.

The following is the C++ program we've just described ...

Installing PyCUDA for Python within an existing CUDA environment

Since we have already learned about CUDA's installation and implementation, it will now be easier for us to get started with our PyCUDA installation procedure for Python. You also do not need to install Python as it is already available (both 2.x and 3.x) with a freshly installed version of the Ubuntu 18.04 Linux operating system.

As we have also learned about Anaconda and its setup, we can also make use of Python 2.x or 3.x, which is readily available with an existing Anaconda configuration. Setting up PyCUDA will enable implementing CUDA kernels within your existing Python setup of choice and then computing with it on your NVIDIA GPU.

There are primarily two methods of installation.

Anaconda-based installation...

Configuring PyCUDA on your Python IDE

In this book, we use PyCharm as our preferred IDE because of its unique design focusing on the Python programming language alone. The next steps are specifically used in two scenarios, namely, using an already-existing Conda environment and a system-wide setting. If you prefer a different IDE, you can still use these steps as a reference because the procedure is very similar. To configure PyCUDA with PyCharm, we will revisit the two methods of installing packages as discussed in the previous chapter.

Conda-based virtual environment

Now let's create a virtual environment with conda as a new PyCharm pure Python project. Perform the following steps to create and configure the environment...

How computing in PyCUDA works on Python

In our very first chapter, we emphasized that computing is application-specific and is a technique to calculate any measurable entity across multi-disciplinary fields. These calculations are meant to solve actual computational problems in a real-world scenario, which is why our focus is on computational problem solving, catering to the prime objective of computing.

Computing: The answer to every computational problem lies in its computed solution.

Accelerated computing: A computationally intensive problem requires an accelerated solution.

Let's understand how PyCUDA can help us solve a myriad of computational problems based on GPU computations via Python, with or without CUDA-C/C++ code.

Following our first C++ versus CUDA example, we will now look into a very simple Python versus PyCUDA example with a similar approach, hands-on with...

Comparing PyCUDA to CUDA – an introductory perspective on reduction

Let's compare PyCUDA to CUDA in terms of simplicity in parallelization before we write our first PyCUDA program on PyCharm.

In the following table, we can explore the scope of PyCUDA with respect to CUDA so as to understand scenarios when PyCUDA could be advantageous to CUDA:

CUDA	PyCUDA
Based on C/C++ programming language	Based on the Python programming language
Uses C/C++ combined with specialized code to accelerate computations	Uses Python for GPUs to interface CUDA and accelerate computations
Reduction is a key feature in CUDA that is extremely important to maximize parallelization and efficiently harness threads.	Reduction in PyCUDA is much simpler to use than CUDA, considering the significance of reduction.

...

Writing your first PyCUDA programs to compute a general-purpose solution

Let's continue our exciting journey forward. Now is the right time to get our hands dirty and write our first PyCUDA program that will compute a general-purpose solution. Follow the next steps to get started:

1. First, open a new file in PyCharm:

2. Now use the following code to print Hello World from NVIDIA GPU! from the GPU device itself! This is a regular and basic format in PyCUDA:

# Auto initialization for CUDA (can be done manually as well)
import pycuda.autoinit

# Importing SourceModule from the PyCUDA Compiler module
from pycuda.compiler import SourceModule

# Printing from the GPU device itself!
mod = SourceModule("""
__global__ void hello_from_nvidia_gpu()
{
printf("Hello World from NVIDIA GPU!");
}
""")

#Referencing the function for the GPU kernel
hello = mod.get_function...

Useful exercise on computational problem solving

Mathematical equations are only useful when you can apply them to solve a social problem or, in other words, serve society through computing. We do this through the convergence of multiple interdisciplinary fields.

Before you try to computationally solve a problem, always consider breaking it down into a series of categorized steps:

1. Problem outline: Finding the nature of the problem and choosing the most effective way of solving it and displaying the result
2. Problem solution: Choosing the most effective mathematical formula to solve the problem
3. Program code: Programming the underlying solution to the problem
4. Solution testing: Applying the previous methodologies to solve a computational problem with Python code to provide an effectively programmed solution

For example, tobacco is linked to many diseases, including cancer. To understand...

Summary

In this chapter, the general syntax of CUDA code was explained with a comparative example. Syntax-wise, the concept of threads and blocks was introduced. The steps to install PyCUDA with or without Anaconda were illustrated within an existing CUDA environment. How to set up PyCUDA was explained step by step. Then, we learned how computing works in Python, and the significance of computational problem-solving was highlighted. With a comparison of PyCUDA and CUDA, the concept of parallel reduction was introduced.

At this stage, you should now be able to test your own CUDA program. As you have already learned how to install CUDA previously in this book, you can also install and configure PyCUDA within an existing CUDA environment. You should now have some understanding of the concept of computational problem solving as an essential and primary approach to computing. You can...

Further reading

You can use the following links and articles to learn more about the topics discussed in this chapter:

• PlinkGPU: A Framework for GPU Acceleration of Whole Genome Data Analysis, J Poznanovic (2010), Master of Science, School of Informatics, University of Edinburgh, http://www.inf.ed.ac.uk/publications/thesis/online/IM100799.pdf
• CUDA-based GPU implementation of the Pearson correlation: https://github.com/TravisCG/CudaCorr.
• gpuCor: Calculate various correlation coefficients with a GPU. This can be found at https://rdrr.io/cran/gputools/man/gpuCor.html.
• GPU-PCC: A GPU Based Technique to Compute Pairwise Pearson's Correlation Coefficients for Big fMRI Data, T Eslami, F Saeed (2018), (online) 7(2), p.11. Available at: https://scholarworks.wmich.edu/cgi/viewcontent.cgi?article=1008&context=pcds_reports.
• Smoking-Related Knowledge, Attitudes, Behaviors, Smoking...