Reader small image

You're reading from  Essential PySpark for Scalable Data Analytics

Product typeBook
Published inOct 2021
Reading LevelBeginner
PublisherPackt
ISBN-139781800568877
Edition1st Edition
Languages
Tools
Concepts
Right arrow
Author (1)
Sreeram Nudurupati
Sreeram Nudurupati
author image
Sreeram Nudurupati

Sreeram Nudurupati is a data analytics professional with years of experience in designing and optimizing data analytics pipelines at scale. He has a history of helping enterprises, as well as digital natives, build optimized analytics pipelines by using the knowledge of the organization, infrastructure environment, and current technologies.
Read more about Sreeram Nudurupati

Right arrow

Tracking experiments with MLflow

In real life, building a single model is never sufficient. A typical model-building process requires iterating over the process several times, sometimes changing the model parameters and other times tweaking the training dataset, until the desired level of model accuracy is achieved. Sometimes, a model that's suitable for a certain use case might not be useful for another. This means that a typical data science process involves experimenting with several models to solve a single business problem and keeping track of all the datasets, model parameters, and model metrics for future reference. Traditionally, experiment tracking is done using rudimentary tools such as spreadsheets, but this slows down the time to production and is also a tedious process that's prone to mistakes.

The MLflow Tracking component solves this problem with its API and UI for logging ML experiments, including model parameters, model code, metrics, the output of the...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Essential PySpark for Scalable Data Analytics
Published in: Oct 2021Publisher: PacktISBN-13: 9781800568877

Author (1)

author image
Sreeram Nudurupati

Sreeram Nudurupati is a data analytics professional with years of experience in designing and optimizing data analytics pipelines at scale. He has a history of helping enterprises, as well as digital natives, build optimized analytics pipelines by using the knowledge of the organization, infrastructure environment, and current technologies.
Read more about Sreeram Nudurupati