Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Accelerate Model Training with PyTorch 2.X

You're reading from  Accelerate Model Training with PyTorch 2.X

Product type Book
Published in Apr 2024
Publisher Packt
ISBN-13 9781805120100
Pages 230 pages
Edition 1st Edition
Languages
Author (1):
Maicon Melo Alves Maicon Melo Alves
Profile icon Maicon Melo Alves

Table of Contents (17) Chapters

Preface 1. Part 1: Paving the Way
2. Chapter 1: Deconstructing the Training Process 3. Chapter 2: Training Models Faster 4. Part 2: Going Faster
5. Chapter 3: Compiling the Model 6. Chapter 4: Using Specialized Libraries 7. Chapter 5: Building an Efficient Data Pipeline 8. Chapter 6: Simplifying the Model 9. Chapter 7: Adopting Mixed Precision 10. Part 3: Going Distributed
11. Chapter 8: Distributed Training at a Glance 12. Chapter 9: Training with Multiple CPUs 13. Chapter 10: Training with Multiple GPUs 14. Chapter 11: Training with Multiple Machines 15. Index 16. Other Books You May Enjoy

What is a computing cluster?

A computing cluster is a system of powerful servers interconnected by a high-performance network, as shown in Figure 11.1. This environment can be provisioned on-premises or in the cloud:

Figure 11.1 – A computing cluster

Figure 11.1 – A computing cluster

The computing power provided by these machines is combined to solve complex problems or to execute highly intensive computing tasks. A computing cluster is also known as a high-performance computing (HPC) system.

Each server has powerful computing resources such as multiple CPUs and GPUs, fast memory devices, ultra-fast disks, and special network adapters. Moreover, a computing cluster often has a parallel filesystem, which provides high transfer I/O rates.

Although not formally defined, we conventionally use the term “cluster” to reference environments comprised of four machines at least. Some computing clusters have a half-dozen machines, while others have more than two or three...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €14.99/month. Cancel anytime}