First Look at TensorFlow

TensorFlow is mathematical software and an open-source software library for Machine Intelligence, developed in 2011, by Google Brain Team. The initial target of TensorFlow was to conduct research in machine learning and in deep neural networks. However, the system is general enough to be applicable in a wide variety of other domains as well.

The name is derived from the data model which is represented by tensors and from the data flow graph that stands for the TensorFlow's execution model. In 2015, Google has open-sourced the TensorFlow and all of its reference implementation and made all the source code available on GitHub under the Apache 2.0 license. After that, TensorFlow has achieved wide adaption, form academia and research to industry and following that recently the most stable version 1.0 has been released with a unified API.

Keeping in mind your needs and based on all the latest...

General overview

TensorFlow is an open source software library for numerical computation using data flow graphs that enables machine learning practitioners to do more data-intensive computing. It provides some robust implementations of widely used deep learning algorithms. Nodes in the flow graph represent mathematical operations. On the other hand, the edges represent multidimensional tensors that ensure communication between edges and nodes. TensorFlow offers you a very flexible architecture that enables you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

What's new with TensorFlow 1.x?

The APIs in TensorFlow 1.0 have changed in ways that are not all backward-compatible. That is, TensorFlow programs...

Installing TensorFlow on Linux

In this section, we will show how to install TensorFlow on Ubuntu 14.04 or higher. The instructions presented here also might be applicable for other Linux distros.

Which TensorFlow to install on your platform?

However, before proceeding with the formal steps, we need to determine which TensorFlow to install on your platform. TensorFlow has been developed such that you can run data-intensive tensor application on GPU as well as CPU. Thus, you should choose one of the following types of TensorFlow to install on your platform:

• TensorFlow with CPU support only: If there is no GPU such as NVIDIA installed on your machine, you must install and start computing using this version. This is very easy and you can do it in just 5 to 10 minutes...

Requirements for running TensorFlow with GPU from NVIDIA

The GPU-enabled version of TensorFlow has several requirements such as 64-bit Linux, Python 2.7 (or 3.3+ for Python 3), NVIDIA CUDA 7.5 (CUDA 8.0 required for Pascal GPUs) and NVIDIA, cuDNN v4.0 (minimum) or v5.1 (recommended). More specifically, the current development of TensorFlow supports only GPU computing using NVIDIA toolkits and software. Now the following software must be installed on your machine.

Step 1: Install NVIDIA CUDA

To use TensorFlow with NVIDIA GPUs, CUDA Toolkit 8.0 and associated NVIDIA drivers with CUDA Toolkit 8+ need to be installed.

For more details, refer to this NVIDIA's documentation
https://developer.nvidia.com/cuda-downloads.

Now download and install the required package from...

How to install TensorFlow

You can install TensorFlow on your machine in a number of ways; such as using virtualenv, pip, Docker, and Anaconda. However, using Docker and Anaconda are a bit advanced and this is why we have decided to use pip and virtualenv instead.

Interested readers can try using Docker and Anaconda from this URL at https://www.tensorflow.org/install/.

Installing TensorFlow with native pip

If Steps 1 to 6 have been completed, install TensorFlow by invoking one of the following commands, for Python 2.7 and of course with only CPU support:

$ pip install tensorflow  

For Python 3.x and of course with only CPU support:

$ pip3 install tensorflow

For Python 2.7 and of course with GPU support:

$ pip install tensorflow-gpu

For Python 3.x and of course...

Installing TensorFlow on Windows

If you can't get a Linux-based system, you must install Ubuntu on a virtual machine; just use a free application called VirtualBox, which lets you create a virtual PC on Windows and install Ubuntu in it. TensorFlow only supports version 3.5.x of Python on Windows. Note that Python 3.5.x comes with the pip3 package manager, which is the program you'll use to install TensorFlow. To install TensorFlow, start a command prompt and then issue the appropriate pip3 install command in that terminal. To install the CPU-only version of TensorFlow, enter the following command:

C:> pip3 install --upgrade tensorflow 

To install the GPU version of TensorFlow, enter the following command:

C:> pip3 install --upgrade tensorflow-gpu 

Installation from source

...

Computational graphs

When performing an operation, for example training a neural network, or the sum of two integers, TensorFlow internally represent, its computation using a data flow graph (or computational graph).

This is a directed graph consisting of the following:

• A set of nodes, each one representing an operation
• A set of directed arcs, each one representing the data on which the operations are performed

TensorFlow has two types of edge:

• Normal: They are only carriers of data structures, between the nodes. The output of one operation (from one node) becomes the input for another operation. The edge connecting two nodes carry the values.
• Special: This edge doesn't carry values. It represents a control dependency between two nodes A and B. It means that the node B will be executed only if the operation in A will be ended before the relationship between operations on the data.

The TensorFlow implementation...

Why a computational graph?

Another key idea in TensorFlow is the deferred execution, during the building phase of the computational graph, you can compose very complex expressions (we say it is highly compositional), when you want to evaluate them through the running session phase, TensorFlow schedules the running in the most efficient manner (for example, parallel execution of independent parts of the code using the GPU).

In this way, a graph helps to distribute the computational load if one must deal with complex models containing a large number of nodes and layers.

Finally, a neural network can be compared to a composite function where each network layer can be represented as a function.

This consideration leads us to the next section, where the role of the computational graph in implementing a neural network is explained.

...

The programming model

A TensorFlow program is generally divided into three phases:

• Construction of the computational graph
• Running a session, which is performed for the operations defined in the graph
• Resulting data collection and analysis

These main steps define the programming model in TensorFlow.

Consider the following example, in which we want to multiply two numbers.

import tensorflow as tf 
with tf.Session() as session: 
    x = tf.placeholder(tf.float32,[1],name="x") 
    y = tf.placeholder(tf.float32,[1],name="y") 
    z = tf.constant(2.0) 
    y = x * z 
x_in = [100] 
y_output = session.run(y,{x:x_in}) 
print(y_output)

Surely TensorFlow is not necessary to multiply two numbers; also the number of lines of the code for this simple operation is so many. However, the example wants to clarify how to structure any code, from the simplest as in this instance, to the most complex.

Furthermore, the...

Data model

The data model in TensorFlow is represented by tensors. Without using complex mathematical definitions, we can say that a tensor (in TensorFlow) identifies a multidimensional numerical array.

This data structure is characterized by three parameters--Rank, Shape, and Type.

Rank

Each tensor is described by a unit of dimensionality called rank. It identifies the number of dimensions of the tensor, for this reason, a rank is a known-as order or n-dimensions of a tensor. A rank zero tensor is a scalar, a rank one tensor ID a vector, while a rank two tensor is a matrix.

The following code defines a TensorFlow scalar, a vector, a matrix and a cube_matrix, in the next example we show how the rank works:

 
import tensorflow as tf 

scalar = tf.constant(100) 
vector...

TensorBoard

When training a neural network, it may be useful to keep track of network parameters, typically the inputs and outputs from the nodes, so you can see whether your model is learning such verifying after each training step if the function error is minimized or not. Of course, writing code to display the behavior of the network during the learning phase, it can be not easy.

Installing TensorBoard is pretty straight forward. Just issue the following command on Terminal (On Ubuntu for Python 2.7+):
$ sudo pip install tensorboard

Fortunately, TensorFlow provides TensorBoard which is a framework designed for analysis and debugging of neural network models. TensorBoard uses the so-called summaries to view the parameters of the model; once a TensorFlow code is executed, we can call TensorBoard to view summaries in a graphical user interface (GUI).

Furthermore, TensorBoard can be used to display and study the...

Implementing a single input neuron

In this example, we will take a closer look at TensorFlow's and TensorBoard's main concepts and try to do some basic operations to get you started. The model we want to implement simulates a single neuron. For reference, see the following diagram:

A schema representing the single input neuron

The output will be the input product for the weight.

The preceding schema consists of the following:

• An input value, which stimulates the neuron.
• A single weight, which is multiplied by the input, provides the output of the neuron. The weight value will vary in the course of the training procedure.
• The output is given by the product input x weight. The neuron will learn when the given output will be issued next to the expected value.
• These elements are enough to define the model. The input value represents a stimulus to the neuron. It is a constant which is defined with the TensorFlow...

Source code for the single input neuron

We reported for the entire source code for the example previously described is as follows:

import tensorflow as tf 

input_value = tf.constant(0.5,name="input_value") 
weight = tf.Variable(1.0,name="weight") 
expected_output = tf.constant(0.0,name="expected_output") 
model = tf.multiply(input_value,weight,"model")loss_function = tf.pow(expected_output - model,2,name="loss_function") 
optimizer = tf.train.GradientDescentOptimizer(0.025).minimize(loss_function) 

for value in [input_value,weight,expected_output,model,loss_function]: 
    tf.summary.scalar(value.op.name,value) 
summaries = tf.summary.merge_all()sess = tf.Session() 
summary_writer = tf.summary.FileWriter('log_simple_stats',sess.graph) 
sess.run(tf.global_variables_initializer())
for i in range(100): 
    summary_writer.add_summary(sess.run(summaries),i) 
    sess.run...

Migrating to TensorFlow 1.x

The latest release of TensorFlow 1.0 has left in ways such that all the computing and using reusing the previous codes that all of them are not backward compatible. This means an application developed on TensorFlow 0.x won't necessarily work on TensorFlow 1.x directly. Now to upgrade 0.x codes to 1.x compatible, there are two ways using the upgrade script or manually.

How to upgrade using the script

The tf_upgrade.py script helps you to upgrade your 0.x source codes to 1.x (or higher) compatible. This script can be downloaded from GitHub at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/compatibility. To convert a single 0.x TensorFlow source file to 1.x (or higher) issue a command in following format:

For Python...

Summary

TensorFlow is designed to make distributed machine and deep learning easy for everyone, but using it does require understanding some general principles and algorithms. Furthermore, the latest release of TensorFlow comes with lots of exciting features. Thus we also tried to cover them so that you can use them with ease. We have shown how to install TensorFlow on different platforms including Linux, Windows, and Mac OS. Before, covering this in even greater depth, we showed some example of how to upgrade the source code from the previous version of TensorFlow to the latest version 1.x.

In summary, here is a brief recap of the key concepts of TensorFlow explained in this chapter:

• Graph: A TensorFlow computation is represented as dataflow graph. Each graph is built as a set of operations (http://www.tensorflow.org/api_docs/python/framework.html#Operation) objects.
• Operation: Each Operation object represents...