The TensorFlow Workshop

4 (2 reviews total)
By Matthew Moocarme , Anthony So , Anthony Maddalone
    What do you get with a Packt Subscription?

  • Instant access to this title and 7,500+ eBooks & Videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Free Chapter
    2. Loading and Processing Data
About this book

Getting to grips with tensors, deep learning, and neural networks can be intimidating and confusing for anyone, no matter their experience level. The breadth of information out there, often written at a very high level and aimed at advanced practitioners, can make getting started even more challenging.

If this sounds familiar to you, The TensorFlow Workshop is here to help. Combining clear explanations, realistic examples, and plenty of hands-on practice, it’ll quickly get you up and running.

You’ll start off with the basics – learning how to load data into TensorFlow, perform tensor operations, and utilize common optimizers and activation functions. As you progress, you’ll experiment with different TensorFlow development tools, including TensorBoard, TensorFlow Hub, and Google Colab, before moving on to solve regression and classification problems with sequential models.

Building on this solid foundation, you’ll learn how to tune models and work with different types of neural network, getting hands-on with real-world deep learning applications such as text encoding, temperature forecasting, image augmentation, and audio processing.

By the end of this deep learning book, you’ll have the skills, knowledge, and confidence to tackle your own ambitious deep learning projects with TensorFlow.

Publication date:
December 2021
Publisher
Packt
Pages
600
ISBN
9781800205253

 

1. Introduction to Machine Learning with TensorFlow

Overview

In this chapter, you will learn how to create, utilize, and apply linear transformations to the fundamental building blocks of programming with TensorFlow: tensors. You will then utilize tensors to understand the complex concepts associated with neural networks, including tensor reshaping, transposition, and multiplication.

 

Introduction

Machine learning (ML) has permeated various aspects of daily life that are unknown to many. From the recommendations of your daily social feeds to the results of your online searches, they are all powered by machine learning algorithms. These algorithms began in research environments solving niche problems, but as their accessibility broadened, so too have their applications for broader use cases. Researchers and businesses of all types recognize the value of using models to optimize every aspect of their respective operations. Doctors can use machine learning to decide diagnosis and treatment options, retailers can use ML to get the right products to their stores at the right time, and entertainment companies can use ML to provide personalized recommendations to their customers.

In the age of data, machine learning models have proven to be valuable assets to any data-driven company. The large quantities of data available allow powerful and accurate models to be created...

 

Implementing Artificial Neural Networks in TensorFlow

The advanced flexibility that TensorFlow offers lends itself well to creating artificial neural networks (ANNs). ANNs are algorithms that are inspired by the connectivity of neurons in the brain and are intended to replicate the process in which humans learn. They consist of layers through which information propagates from the input to the output.

Figure 1.1 shows a visual representation of an ANN. An input layer is on the left-hand side, which, in this example, has two features (X1 and X2). The input layer is connected to the first hidden layer, which has three units. All the data from the previous layer gets passed to each unit in the first hidden layer. The data is then passed to the second hidden layer, which also has three units. Again, the information from each unit of the prior layer is passed to each unit of the second hidden layer. Finally, all the information from the second hidden layer is passed to the output layer...

 

The TensorFlow Library in Python

TensorFlow can be used in Python by importing certain libraries. You can import libraries in Python using the import statement:

import tensorflow as tf

In the preceding command, you have imported the TensorFlow library and used the shorthand tf.

In the next exercise, you will learn how to import the TensorFlow library and check its version so that you can utilize the classes and functions supplied by the library, which is an important and necessary first step when utilizing the library.

Exercise 1.01: Verifying Your Version of TensorFlow

In this exercise, you will load TensorFlow and check which version is installed on your system.

Perform the following steps:

  1. Open a Jupyter notebook to implement this exercise by typing jupyter notebook in the terminal.
  2. Import the TensorFlow library by entering the following code in the Jupyter cell:
    import tensorflow as tf
  3. Verify the version of TensorFlow using the following command...
 

Introduction to Tensors

Tensors can be thought of as the core components of ANNs—the input data, output predictions, and weights that are learned throughout the training process are all tensors. Information propagates through a series of linear and nonlinear transformations to turn the input data into predictions. This section demonstrates how to apply linear transformations such as additions, transpositions, and multiplications to tensors. Other linear transformations, such as rotations, reflections, and shears, also exist. However, their applications as they pertain to ANNs are less common.

Scalars, Vectors, Matrices, and Tensors

Tensors can be represented as multi-dimensional arrays. The number of dimensions a tensor spans is known as the tensor's rank. Tensors with ranks 0, 1, and 2 are used often and have their own names, which are scalars, vectors, and matrices, respectively, although the term tensors can be used to describe each of them. Figure 1.2 shows...

 

Tensor Addition

Tensors can be added together to create new tensors. You will use the example of matrices in this chapter, but the concept can be extended to tensors with any rank. Matrices may be added to scalars, vectors, and other matrices under certain conditions in a process known as broadcasting. Broadcasting refers to the process of array arithmetic on tensors of different shapes.

Two matrices may be added (or subtracted) together if they have the same shape. For such matrix-matrix addition, the resultant matrix is determined by the element-wise addition of the input matrices. The resultant matrix will therefore have the same shape as the two input matrices. You can define the matrix Z = [Zij] as the matrix sum Z = X + Y, where zij = xij + yij and each element in Z is the sum of the same element in X and Y.

Matrix addition is commutative, which means that the order of X and Y does not matter, that is, X + Y = Y + X. Matrix addition is also associative, which ...

 

Reshaping

Some operations, such as addition, can only be applied to tensors if they meet certain conditions. Reshaping is one method for modifying the shape of tensors so that such operations can be performed. Reshaping takes the elements of a tensor and rearranges them into a tensor of a different size. A tensor of any size can be reshaped so long as the number of total elements remains the same.

For example, a (4x3) matrix can be reshaped into a (6x2) matrix since they both have a total of 12 elements. The rank, or number, of dimensions, can also be changed in the reshaping process. For instance, a (4x3) matrix that has a rank equal to 2 can be reshaped into a (3x2x2) tensor that has a rank equal to 3. The (4x3) matrix can also be reshaped into a (12x1) vector in which the rank has changed from 2 to 1.

Figure 1.13 illustrates tensor reshaping. On the left is a tensor with shape (3x2), which can be reshaped to a tensor of shape equal to either (2x3), (6), or (6x1). Here...

 

Tensor Multiplication

Tensor multiplication is another fundamental operation that is used frequently in the process of building and training ANNs since information propagates through the network from the inputs to the result via a series of additions and multiplications. While the rules for addition are simple and intuitive, the rules for tensors are more complex. Tensor multiplication involves more than simple element-wise multiplication of the elements. Rather, a more complicated procedure is implemented that involves the dot product between the entire rows/columns of each of the tensors to calculate each element of the resulting tensor. This section will explain how multiplication works for two-dimensional tensors or matrices. However, tensors of higher orders can also be multiplied.

Given a matrix, X = [xij]m x n, and another matrix, Y = [yij]n x p, the product of the two matrices is Z = XY = [zij]m x p, and each element, zij, is defined element-wise as Formula. The shape of the resultant...

 

Optimization

In this section, you will learn about some optimization approaches that are fundamental to training machine learning models. Optimization is the process by which the weights of the layers of an ANN are updated such that the error between the predicted values of the ANN and the true values of the training data is minimized.

Forward Propagation

Forward propagation is the process by which information propagates through ANNs. Operations such as a series of tensor multiplications and additions occur at each layer of the network until the final output. Forward propagation is explained in Figure 1.37, showing a single hidden layer ANN. The input data has two features, while the output layer has a single value for each input record.

The weights and biases for the hidden layer and output are shown as matrices and vectors with the appropriate indexes. For the hidden layer, the number of rows in the weight matrix is equal to the number of features of the input, and...

 

Activation functions

Activation functions are mathematical functions that are generally applied to the outputs of ANN layers to limit or bound the values of the layer. The reason that values may want to be bounded is that without activation functions, the value and corresponding gradients can either explode or vanish, thereby making the results unusable. This is because the final value is the cumulative product of the values from each subsequent layer. As the number of layers increases, the likelihood of values and gradients exploding to infinity or vanishing to zero increases. This concept is known as the exploding and vanishing gradient problem. Deciding whether a node in a layer should be activated is another use of activation functions, hence their name. Common activation functions and their visual representation in Figure 1.36 are as follows:

  • Step function: The value is non-zero if it is above a certain threshold, otherwise it is zero. This is shown in Figure 1.36a...
 

Summary

In this chapter, you were introduced to the TensorFlow library. You learned how to use it in the Python programming language. You created the building blocks of ANNs (tensors) with various ranks and shapes, performed linear transformations on tensors using TensorFlow, and implemented addition, reshaping, transposition, and multiplication on tensors—all of which are fundamental for understanding the underlying mathematics of ANNs.

In the next chapter, you will improve your understanding of tensors and learn how to load data of various types and pre-process it such that it is appropriate for training ANNs in TensorFlow. You will work with tabular, visual, and textual data, all of which must be pre-processed differently. By working with visual data (that is, images), you will also learn how to use training data in which the size of the training data cannot fit into memory.

About the Authors
  • Matthew Moocarme

    Matthew Moocarme is an accomplished data scientist with more than eight years of experience in creating and utilizing machine learning models. He comes from a background in the physical sciences, in which he holds a Ph.D. in physics from the Graduate Center of CUNY. Currently, he leads a team of data scientists and engineers in the media and advertising space to build and integrate machine learning models for a variety of applications. In his spare time, Matthew enjoys sharing his knowledge with the data science community through published works, conference presentations, and workshops.

    Browse publications by this author
  • Anthony So

    Anthony So is a renowned leader in data science. He has extensive experience in solving complex business problems using advanced analytics and AI in different industries including financial services, media, and telecommunications. He is currently the chief data officer of one of the most innovative fintech start-ups. He is also the author of several best-selling books on data science, machine learning, and deep learning. He has won multiple prizes at several hackathon competitions, such as Unearthed, GovHack, and Pepper Money. Anthony holds two master's degrees, one in computer science and the other in data science and innovation.

    Browse publications by this author
  • Anthony Maddalone

    Anthony Maddalone is a research engineer at TieSet, a Silicon Valley-based leader in distributed artificial intelligence and federated learning. He is a former founder and CEO of a successful start-up. Anthony lives with his wife and two children in Colorado, where they enjoy spending time outdoors. He is also a master's candidate in analytics with a focus on industrial engineering at the Georgia Institute of Technology.

    Browse publications by this author
Latest Reviews (2 reviews total)
More traditional topic but still very useful
I've been interested in TensorFlow for a while now. I think this course will help me get a good foundation for using it.
The TensorFlow Workshop
Unlock this book and the full library FREE for 7 days
Start now