Practical Deep Learning at Scale with MLflow

By Yong Liu
    What do you get with a Packt Subscription?

  • Instant access to this title and 7,500+ eBooks & Videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Free Chapter
    Chapter 1: Deep Learning Life Cycle and MLOps Challenges
About this book

The book starts with an overview of the deep learning (DL) life cycle and the emerging Machine Learning Ops (MLOps) field, providing a clear picture of the four pillars of deep learning: data, model, code, and explainability and the role of MLflow in these areas.

From there onward, it guides you step by step in understanding the concept of MLflow experiments and usage patterns, using MLflow as a unified framework to track DL data, code and pipelines, models, parameters, and metrics at scale. You’ll also tackle running DL pipelines in a distributed execution environment with reproducibility and provenance tracking, and tuning DL models through hyperparameter optimization (HPO) with Ray Tune, Optuna, and HyperBand. As you progress, you’ll learn how to build a multi-step DL inference pipeline with preprocessing and postprocessing steps, deploy a DL inference pipeline for production using Ray Serve and AWS SageMaker, and finally create a DL explanation as a service (EaaS) using the popular Shapley Additive Explanations (SHAP) toolbox.

By the end of this book, you’ll have built the foundation and gained the hands-on experience you need to develop a DL pipeline solution from initial offline experimentation to final deployment and production, all within a reproducible and open source framework.

Publication date:
July 2022


Section 1 - Deep Learning Challenges and MLflow Prime

In this section, we will learn about the five stages of the full life cycle of deep learning (DL), and understand the emerging field of machine learning operations (MLOps) and the role of MLflow. We will provide an overview of the challenges in the four pillars of a DL process: data, model, code, and explainability. Then, we will learn how to set up a basic local MLflow development environment and run our first MLflow experiment for a natural language processing (NLP) model built on top of PyTorch Lightning Flash. Finally, we will explain the foundational MLflow concepts such as experiments, runs, and many more, through this first MLflow experiment example.

This section comprises the following chapters:

  • Chapter 1, Deep Learning Life Cycle and MLOps Challenges
  • Chapter 2, Getting Started with MLflow for Deep Learning
About the Author
  • Yong Liu

    Yong Liu has been working in big data science, machine learning, and optimization since his doctoral student years at the University of Illinois at Urbana-Champaign (UIUC) and later as a senior research scientist and principal investigator at the National Center for Supercomputing Applications (NCSA), where he led data science R&D projects funded by the National Science Foundation and Microsoft Research. He then joined Microsoft and AI/ML start-ups in the industry. He has shipped ML and DL models to production and has been a speaker at the Spark/Data+AI summit and NLP summit. He has recently published peer-reviewed papers on deep learning, linked data, and knowledge-infused learning at various ACM/IEEE conferences and journals.

    Browse publications by this author
Practical Deep Learning at Scale with MLflow
Unlock this book and the full library FREE for 7 days
Start now