Join our community on Discord

https://packt.link/deep-engineering-cpp

Welcome to the world of graphics processing unit (GPU) programming!Before we talk about programming GPUs, we must understand what parallel programming is and how it can benefit our applications. As with everything in life, it has its challenges. In this chapter, we'll explore both the benefits and drawbacks of parallel programming, laying the groundwork for our deep dive into GPU programming. So in this first chapter, we'll be discussing a variety of topics without developing any code. In doing so, we'll establish the foundations on which to build throughout our journey.Apart from being useful, the information provided in this chapter is fundamental to understanding what happens inside a GPU, as we'll discuss shortly. By the end of this chapter, you'll understand why parallelism is important and when it makes sense to use it in your applications.In this chapter, we...

Technical requirements

For this chapter, the only technical requirement that we have is the goodwill to keep reading!

What is parallelism in software?

Parallel programming is a way of making a computer do many things at once. But wait – isn't this what already happens daily? Yes and no. Most common processors today are capable of executing more than one task at the same time – and we mean at the same time. However, this is only the first requirement for parallel software. The second is to make at least some of the processor cores work on the same problem in a coordinated way. Let's consider an example.Imagine that you're taking on a big task, such as sorting a huge pile of books. Instead of doing it alone, you ask a group of friends to help. Each friend takes a small part of the pile and sorts it. You all work at the same time, and the job gets done much faster. This is similar to how parallel programming works: it breaks a big problem into smaller pieces and solves them at the same time using multiple cores.Of course, this example was chosen because it has a...

Why is parallelism important?

There are many situations in which the size of the problems we want to solve increases dramatically. And this is the moment when we have to start talking about more ‘“serious’” real-world applications, such as weather forecasting, scientific research, and artificial intelligence.Remember when we were driving to the supermarket and we mentioned that we could switch drivers for each part of the way? Wouldn't this only end up taking us more time? This was due to context switching – we would have to find a place to park, then switch drivers, then drive the car until the next stop. But why are we talking about this again? Because most of the time, we need a ‘“serious’” real-world application to make it worthwhile working through all the details of parallel programming. One exception could be using parallel programming to accelerate graphics and physics processing in video games; although...

A quick start guide to the different types of parallelism

So far, we've only been talking about parallelism. In this section, we'll quickly discuss the different types of parallelism before we dive into GPUs – which is what we're all waiting for!

An overview of GPU architecture

After all that cooking, it's time for a change. Let's talk about GPUs.First, let me say that I’ve decided to explain GPUs first before comparing them with CPUs. I’m doing this on the assumption that you're already somewhat familiar with the (basic) architecture of a modern CPU.GPUs were originally thought to accelerate the output of processing graphics, since modern computer usage takes place almost exclusively in graphical environments. This differs from computing in the past, where the character-based interfaces that were used weren't graphically demanding. However, a shift occurred when it was noticed that a processing unit that was capable of dealing with the computations necessary for computer graphics could also be used for anything that could be expressed in terms of matrix computations, which is what linear algebra is all about.In the next chapter, we're going to focus specifically on NVIDIA GPUs...

Comparing CPUs and GPUs

Now that we know a little more about how GPUs are organized, we can compare them with CPUs to understand the impact of using these devices.Modern CPUs are typically composed of many cores, so they're capable of executing parallel applications by using threads. But while they’re capable of handling tens of threads, a simple GPU can handle thousands of threads.As mentioned previously, the fact that GPU cores execute the same instruction on many pieces of data is an interesting difference from CPU cores. On a CPU, each core is a complete processor that can execute either different applications or different threads of the same parallel application. This means that branch execution on a CPU core doesn't affect the performance of other CPU core executions.Another important distinction is that CPUs can switch between tasks quickly, while GPU cores are controlled by their stream multiprocessor.Regarding memory management, most of the time...

Advantages and challenges of GPU programming

So far, we've learned why parallelism matters, considered the various GPU device components, and compared GPUs with CPUs. Now it's time to understand how GPUs can enhance the performance of our solutions and how to overcome the challenges that come with these benefits.Since we've already talked about some of the benefits, let's start with the challenges that come with GPU programming.

Summary

In this chapter, we learnt about various concepts regarding parallel software and its main patterns. We learned about the architecture of GPUs, and we compared GPUs with CPUs in order to understand the differences in developing software for each. Finally, we discussed the challenges and the benefits of using GPUs in our applications.At this point, armed with this new knowledge, a practical side-benefit is that we can organize a meeting with our friends much more efficiently, making faster trips to the supermarket and preparing appetizers accordingly.In the next chapter, we'll learn how to configure our environment so that we can start programming. There are a few steps we must take to get everything is in place ready for that. We'll discuss two alternatives: using Docker and installing it directly on our machines.