- The fact that atomicExch is thread-safe doesn't guarantee that all threads will execute this function at the same time (which is not the case since different blocks in a grid can be executed at different times).
- A block of size 100 will be executed over multiple warps, which will not be synchronized within the block unless we use __syncthreads. Thus, atomicExch may be called at multiple times.
- Since a warp executes in lockstep by default, and blocks of size 32 or less are executed with a single warp, __syncthreads would be unnecessary.
- We use a naïve parallel sum within the warp, but otherwise, we are doing as many sums withatomicAdd as we would do with a serial sum. While CUDA automatically parallelizes many of these atomicAdd invocations, we could reduce the total number of required atomicAdd invocations by implementing...
- Tech Categories
- Best Sellers
- New Releases
- Books
- Videos
- Audiobooks
Tech Categories Popular Audiobooks
- Articles
- Newsletters
- Free Learning
You're reading from Hands-On GPU Programming with Python and CUDA
Dr. Brian Tuomanen has been working with CUDA and General-Purpose GPU Programming since 2014. He received his Bachelor of Science in Electrical Engineering from the University of Washington in Seattle, and briefly worked as a Software Engineer before switching to Mathematics for Graduate School. He completed his Ph.D. in Mathematics at the University of Missouri in Columbia, where he first encountered GPU programming as a means for studying scientific problems. Dr. Tuomanen has spoken at the US Army Research Lab about General Purpose GPU programming, and has recently lead GPU integration and development at a Maryland based start-up company. He currently lives and works in the Seattle area.
Read more about Dr. Brian Tuomanen
Unlock this book and the full library FREE for 7 days
Author (1)
Dr. Brian Tuomanen has been working with CUDA and General-Purpose GPU Programming since 2014. He received his Bachelor of Science in Electrical Engineering from the University of Washington in Seattle, and briefly worked as a Software Engineer before switching to Mathematics for Graduate School. He completed his Ph.D. in Mathematics at the University of Missouri in Columbia, where he first encountered GPU programming as a means for studying scientific problems. Dr. Tuomanen has spoken at the US Army Research Lab about General Purpose GPU programming, and has recently lead GPU integration and development at a Maryland based start-up company. He currently lives and works in the Seattle area.
Read more about Dr. Brian Tuomanen