Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Parallel Programming with Python

You're reading from  Parallel Programming with Python

Product type Book
Published in Jun 2014
Publisher
ISBN-13 9781783288397
Pages 124 pages
Edition 1st Edition
Languages

Table of Contents (16) Chapters

Parallel Programming with Python
Credits
About the Author
Acknowledgments
About the Reviewers
www.PacktPub.com
Preface
1. Contextualizing Parallel, Concurrent, and Distributed Programming 2. Designing Parallel Algorithms 3. Identifying a Parallelizable Problem 4. Using the threading and concurrent.futures Modules 5. Using Multiprocessing and ProcessPoolExecutor 6. Utilizing Parallel Python 7. Distributing Tasks with Celery 8. Doing Things Asynchronously Index

Decomposing tasks with pipeline


The pipeline technique is used to organize tasks that must be executed in a collaborative way to resolve a problem. Pipeline breaks large tasks into smaller independent tasks that run in a parallel manner. The pipeline model could be compared to an assembly line at a vehicle factory where the chassis is the raw material, the input. As the raw material goes through different stages of production, several workers perform different actions one after another until the end of the process so that we can have a car ready. This model is very similar to the sequential paradigm of development; tasks are executed on data one after another, and normally, a task gets an input, which is the result of the previous task. So what differentiates this model from the sequential technique? Each stage of the pipeline technique possesses its own workers that act in a parallel way on the problem.

An example in the context of computing could be one in which a system processes images...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}