Keras, a High-Level API for TensorFlow 2

In this chapter, we will discuss Keras, which is a high-level API for TensorFlow 2. Keras was developed by François Chollet at Google. Keras has become extremely popular for fast prototyping, for building and training deep learning models, and for research and production. Keras is a very rich API; it supports eager execution and data pipelines, and other features, as we will see.

Keras has been available for TensorFlow since 2017, but its use has been extended and further integrated into TensorFlow with the release of TensorFlow 2.0. TensorFlow 2.0 has embraced Keras as the API of choice for the majority of deep learning development work.

It is possible to import Keras as a standalone module, but in this book, we will concentrate on using Keras from within TensorFlow 2. The module is, thus, tensorflow.keras.

In this chapter, we will...

The adoption and advantages of Keras

Keras is widely used by industry and research, as shown in the following diagram. The Power Score rankings were devised by Jeff Hale, who used 11 data sources across 7 distinct categories to gauge framework usage, interest, and popularity. Then, he weighted and combined the data as shown in this Medium article from September 2018: https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a:

Keras has a number of advantages, including the following:

• It's designed for both new users and experts alike, offering consistent and simple APIs
• It's user friendly with a simple, consistent interface that is optimized for common use cases
• It provides excellent feedback for user errors that is are easily understood and often accompanied by helpful advice
• It's modular and composable; models in Keras are constructed...

The features of Keras

If you want to know which version of Keras came with your TensorFlow, use the following command:

import tensorflow as tf
print(tf.keras.__version__)

At the time of writing, this produced the following (from the alpha build of TensorFlow 2):

2.2.4-tf

Other features of Keras include built-in support for multi-GPU data parallelism, and also the fact that Keras models can be turned into TensorFlow Estimators and trained on clusters of GPUs on Google Cloud.

Keras is, perhaps, unusual in that it is has a reference implementation maintained as an independent open source project, located at www.keras.io.

It's maintained independently of TensorFlow, although TensorFlow does have a full implementation of Keras in the tf.keras module. The implementation has TensorFlow-specific augmentations, including support for eager execution, by default.

Eager execution means...

The default Keras configuration file

The default configuration file for Linux users is as follows:

$HOME/.keras/keras.json

For Windows users, replace $HOME with %USERPROFILE%.

It is created the first time you use Keras, and may be edited to change the defaults. Here's what that .json file contains:

{ "image_data_format": "channels_last",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow" }

The defaults are as follows:

• image_data_format: This is a string, either "channels_last" or channels_first, for the image format. Keras running on top of TensorFlow uses the default.
• epsilon: This is a float, being a numeric fuzzing constant used to avoid division by zero in some operations.
  
• floatx: This is a string and specifies the default float precision, being one of "float16", "...

The Keras backend

Due to its model-level library structure, Keras may have different tensor manipulation engines that handle low-level operations, such as convolutions, tensor products, and the like. These engines are called backends. Other backends are available; we will not consider them here.

The same https://keras.io/backend/ link takes you to a wealth of keras.backend functions.

The canonical way to use the Keras backend is with the following:

from keras import backend as K

For example, here is the signature of a useful function:

K.constant(value, dtype=None, shape=None, name=None)

Here value is the value to be given to the constant, dtype is the type of the tensor that is created, shape is the shape of the tensor that is created, and name is an optional name.

An instantiation of this is as follows:

from tensorflow.keras import backend as K
const = K.constant([[42,24],[11...

Keras data types

Keras data types (dtypes) are the same as TensorFlow Python data types, as shown in the following table:

Python type	Description
tf.float16	16-bit floating point
tf.float32	32-bit floating point
tf.float64	64-bit floating point
tf.int8	8-bit signed integer
tf.int16	16-bit signed integer
tf.int32	32-bit signed integer
tf.int64	64-bit signed integer
tf.uint8	8-bit unsigned integer
tf.string	Variable-length byte arrays
tf.bool	Boolean
tf.complex64	Complex number made of two 32-bit floating points—one real and imaginary part
tf.complex128	Complex number made of two 64-bit floating points—one real and one imaginary part
tf.qint8	8-bit signed integer used in quantized Ops
tf.qint32	32-bit signed integer used in quantized Ops
tf.quint8	8-bit unsigned integer used in quantized Ops

...

Keras models

Keras is based on the concept of a neural network model. The predominant model is called a Sequence, being a linear stack of layers. There is also a system using the Keras functional API.

The Keras Sequential model

To build a Keras Sequential model, you add layers to it in the same order that you want the computations to be undertaken by the network.

After you have built your model, you compile it; this optimizes the computations that are to be undertaken, and is where you allocate the optimizer and the loss function you want your model to use.

The next stage is to fit the model to the data. This is commonly known as training the model, and is where all the computations take place. It is possible to present the...

Summary

In this chapter, we explored the Keras API with general comments and insights followed by the same basic architecture expressed in four different ways, for training with the mnist dataset.

In the next chapter, we will start to use TensorFlow in earnest by exploring a number of supervised learning scenarios, including linear regression, logistic regression, and k-nearest neighbors.