Chapter 13. Best Practices

"The purpose of a storyteller is not to tell you how to think, but to give you questions to think upon."

– Brandon Sanderson, The Way of Kings

Imagine the field of AI as a huge national park. In previous chapters, we guided you along several exciting trails and showed you the most interesting sights for mobile developers. But there is still so much more that is unexplored. So, in this chapter, we want to provide you with a map of the common paths, from idea to production. We've outlined dangerous zones and left notes on solo hiking best practices! We also want to point out several interesting directions for your future exploration.

In this chapter, we will discuss the following topics:

• The path from idea to production
• Common pitfalls in machine learning projects also known as machine learning gremlins
• Machine learning best practices
• Recommended study resources

Mobile machine learning project life cycle

When developing a mobile machine learning product, you typically go through several stages:

• Preparatory stage
• Prototype creation
• Porting to a mobile platform or deployment of the trained model
• Production

Depending on your situation, your route may be shorter or longer; but usually, if you have skipped some stage, it just means that someone else did it for you. In the following explanation, we are omitting all the steps that are common to all kinds of mobile app projects and focusing only on the steps specific to machine learning.

Preparatory stage

This is the stage where you basically decide what you will do. There can be two possible outcomes for this stage: you have a plan on how to proceed, or you decide that you will not proceed:

Figure 13.1: Preparatory stage map

Formulate the problem

If you can solve your problem without machine learning, don't use it. If the task can be solved with traditional programming techniques, congratulations! You don't need...

Best practices

In this section, we've collected some general ideas worth keeping in mind during the whole development process.

Note

It's impossible to collect all important thoughts in one place, so here is a list of some really insightful guides from seasoned machine learning engineers on the best practices they recommend:

• A Few Useful Things to Know about Machine Learning by Pedro Domingos, at: https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf
• Best Practices for Applying Deep Learning to Novel Applications by Leslie N. Smith, at: https://arxiv.org/abs/1704.01568
• Rules of Machine Learning: Best Practices for ML Engineering by Martin Zinkevich, at: http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
• Best Practices for Machine Learning Applications by Brett Wujek, Patrick Hall, and Funda Güneș, at: https://support.sas.com/resources/papers/proceedings16/SAS2360-2016.pdf

Benchmarking

When you are creating a model for solving a popular machine learning task, how do you know it is any better...

Machine learning gremlins

Ben Hamner, a data scientist at Kaggle, referred to common machine learning gotchas as ML gremlins.

Note

You can watch Ben's original talk at: https://www.youtube.com/watch?v=tleeC-KlsKA.

I like the metaphor because it makes my brain think about evil characters rather than some vague, abstract concepts. In addition to the original gremlins presented by Ben, I want to add several of my own and also present a taxonomy of gremlins (see the following diagram). I employed this metaphor throughout this chapter to avoid boring issues and problems when discussing how to identify and neutralize those pests:

Figure 13.3: The simplified taxonomy of machine learning problems

Data kobolds

Dealing with data is hard; that's why we call it data science and data mining! Many different things can go wrong at different stages. Ben mentions data insufficiency, data leakage, non-stationary distributions, poor data sampling and splitting, data quality, and poorly anonymized data. Let's add...

Recommended learning resources

In this book, we've only scratched the surface of the immense body of knowledge behind the term machine learning. If you want to learn more, we highly recommend the following resources.

The main criteria for choosing the courses and books were clarity of presentation and a CS-oriented approach. Other criteria for the books were free online availability and open source code samples. All courses mentioned in this list are free (as of May 2017) and of introductory level.

Mathematical background

The handwritten comic-style lectures on Calculus by Robert Ghrist from the University of Pennsylvania can be found on YouTube or Coursera. This teaches single-variable calculus: Taylor series, Newton method. This should be your choice if you don't know how to take a derivative of a sigmoid function or which functions are differentiable. For more information refer to: https://www.math.upenn.edu/~ghrist/.

Coding The Matrix: Linear Algebra Through Computer Science Applications...

Summary

This was the final chapter of the book; so we discussed a machine learning app's life cycle, and common problems in AI projects and how to solve them. We also provided a list of good study material for further progress of our readers. We hope that you were not disappointed and wish you many successes in your own AI experiments!