Exploring Supervised Deep Learning

Chapters 2 to 6 explored the core workhorse behind deep learning (DL) technology and included some minimal technical implementations for easy digestion. It is important to understand the intricacies of how different neural networks (NNs) work. One reason is that when things go wrong with any NN model, you can identify what the root cause is and mitigate it. Those chapters are also important to showcase how flexible DL architectures are to solve different types of real-world problems. But what are the problems exactly? Also, how should we train a DL model effectively in varying situations?

In this chapter, we will attempt to answer the preceding two points specifically for supervised deep learning, but we will leave answering the same questions for unsupervised deep learning for the next chapter. This chapter will cover the following topics:

• Exploring supervised use cases and problem types
• Implementing neural network layers for foundational...

Technical requirements

This chapter includes some practical implementations in the Python programming language. To complete it, you will need to have a computer with the following libraries installed:

• pytorch
• catalyst==22.04
• numpy
• scikit-learn

You can find the code files for this chapter on GitHub at https://github.com/PacktPublishing/The-Deep-Learning-Architect-Handbook/tree/main/CHAPTER_8.

Exploring supervised use cases and problem types

Supervised learning requireslabeled data. Labels, targets, and ground truth all refer to the same thing. The provided labels essentially supervise the learning process of the machine learning (ML) model and provide the feedback needed for a DL model to generate gradients and update itself. Labels can exist in many different forms. They are continuous numerical format, categorical format, text format, multiple categorical formats, image format, video format, audio format, and multiple target formats. All of these are then categorized as either of the following supervised problem types:

• Binary classification: This is when the target has categorical data with only two unique values.
• Multiclassification: This is when the target has categorical data with more than two unique values.
• Regression: This is when the target has continuous numerical data.
• Multi-target/problem:
  • Multilabel: This is when the target has more than...

Implementing neural network layers for foundational problem types

In Chapters 2 to 7, although many types of NN layers were introduced, the core layers for the problem types were either not used or not explained. Here, we will go through each of them for clarity and intuition.

Implementing the binary classification layer

Binary means two options for categorical data. Note that this does not necessarily mean a strict rule for the categories to be true or false nor positive or negative in the raw data. The two options can be in any format possible in terms of raw data, in strings, numbers, or symbols. However, note that NNs can always only produce numerical outputs. This means that the target itself has to be represented numerically, for which the optimal numbers are the binary values of zero and one. This means that the data column to be used as a target for training with only two unique values must go through preprocessing to map itself into zero or one.

Generally, there are...

Training supervised deep learning models effectively

In Chapter 1, Deep Learning Life Cycle, it is emphasized that ML projects have a cyclical life cycle. In other words, a lot of iterative processes are carried out in the course of the project’s lifetime. To train supervised deep learning models effectively, there are a lot of general directions that should be taken based on different conditions, but the one that absolutely stands out across every problem is proper tooling. The tooling is more commonly known as ML operations (MLOps). Good MLOps systems for DL are easy to use and provide versioning methods for datasets and model experiments, visualization methods, easy ways to use DL libraries such as pytorch or keras with tensorflow, ease of deployment, ease of model comparisons using different metrics, ease of model tuning, good visualization of model training monitoring, and, finally, good feedback about the progress (this can be sent through messages and notifications for...

Exploring general techniques to realize and improve supervised deep learning based solutions

Notice that earlier in the chapter we focused on use cases based on problem types and not the problems themselves. Solutions in turn solve and take care of the problem. DL and ML in general are great solvers of issues related to staffing difficulties and for the automation of mundane tasks. Furthermore, ML models in computers can process data much quicker than an average human can, allowing a much quicker response time and much more efficient scaling of any process. In many cases, ML models can help to increase the accuracy and efficiency of processes. Sometimes, they improve current processes, and other times, they make previously unachievable processes possible. However, a single DL model may or may not be enough to solve the problem. Let’s take an example of a solution that can be solved sufficiently with a single DL model.

Consider the use case of using a DL model to predict the...

Breaking down the multitask paradigm in supervised deep learning

Multitask is a paradigm that covers a wide spectrum of tasks that involves the execution of ML models on multiple problems coupled with their respective datasets to achieve a goal. This paradigm is usually built based on two reasons:

• To achieve better predictive performance and generalization.
• To break down complicated goals into smaller tasks that are directly solvable using separate ML models. This reiterates the point made in the previous topic.

Let’s dive into four multitask techniques, starting with multitask pipelines.

Multitask pipelines

This variation of multitask systems revolves around realizing solutions that can’t be directly solved by using a single ML model. Breaking down highly complicated tasks into smaller tasks can allow solutions to be made with multiple ML models handling different smaller tasks. These tasks can be sequential or parallel in their paths and generally...

Summary

In this chapter, we explored supervised deep learning, including the types of problems it can be used to solve and the techniques for implementing and training DL models. Supervised deep learning involves training a model on labeled data to make predictions on new data. We also covered a variety of supervised learning use cases on different problem types, including binary classification, multiclassification, regression, and multitask and representation learning. The chapter also covered techniques for training DL models effectively, including regularization and hyperparameter tuning, and provided practical implementations in the Python programming language using popular DL frameworks.

Supervised deep learning can be used for a wide range of real-world applications in tasks such as image classification, natural language processing (NLP), and speech recognition. With the knowledge provided in this chapter, you should be able to identify supervised learning applications and...