Relating the evaluation metric to success
Defining success in a machine learning project is crucial and should be done at the early stages of the project as introduced in the Defining success section in Chapter 1, Deep Learning Life Cycle. Success can be defined as achieving higher-level objectives, such as improving the efficiency of processes or increasing the accuracy of processes in comparison to manual labor. In some rare cases, machine learning can enable processes that were previously impossible due to human limitations. The ultimate success of achieving these objectives is to save costs or earn more revenue for an organization.
A model with a metric performance score of 0.80 F1 score or 0.00123 RMSE doesn’t really mean anything at face value and has to be translated to something tangible in the use case. For instance, one should answer questions such as what estimated model score can allow the project to achieve the targeted cost savings or revenue improvements. Quantifying...
