Chapter 7. Other Important Deep Learning Libraries

In this chapter, we'll talk about other deep learning libraries, especially libraries with programming languages other than Java. The following are the most famous, well-developed libraries:

• Theano

• TensorFlow

• Caffe

You'll briefly learn about each of them. Since we'll mainly implement them using Python here, you can skip this chapter if you are not a Python developer. All the libraries introduced in this chapter support GPU implementations and have other special features, so let's dig into them.

Theano

Theano was developed for deep learning, but it is not actually a deep learning library; it is a Python library for scientific computing. The documentation is available at http://deeplearning.net/software/theano/. There are several characteristics introduced on the page such as the use of a GPU, but the most striking feature is that Theano supports computational differentiation or automatic differentiation, which ND4J, the Java scientific computing library, doesn't support. This means that, with Theano, we don't have to calculate the gradients of model parameters by ourselves. Theano automatically does this instead. Since Theano undertakes the most complicated parts of the algorithm, implementations of math expressions can be less difficult.

Let's see how Theano computes gradients. To begin with, we need to install Theano on the machine. Installation can be done just by using pip install Theano or easy_install Theano. Then, the following are the lines to import and use Theano:

import...

TensorFlow

TensorFlow is the library for machine learning and deep learning developed by Google. The project page is https://www.tensorflow.org/ and all the code is open to the public on GitHub at https://github.com/tensorflow/tensorflow. TensorFlow itself is written with C++, but it provides a Python and C++ API. We focus on Python implementations in this book. The installation can be done with pip, virtualenv, or docker. The installation guide is available at https://www.tensorflow.org/versions/master/get_started/os_setup.html. After the installation, you can import and use TensorFlow by writing the following code:

import tensorflow as tf

TensorFlow recommends you implement deep learning code with the following three parts:

• inference(): This makes predictions using the given data, which defines the model structure

• loss(): This returns the error values to be optimized

• training(): This applies the actual training algorithms by computing gradients

We'll follow this guideline. A tutorial on...

Caffe

Caffe is a library famous for its speed. The official project page is http://caffe.berkeleyvision.org/ and the GitHub page is https://github.com/BVLC/caffe. Similar to TensorFlow, Caffe has been developed mainly with C++, but it provides a Python and MATLAB API. In addition, what is unique to Caffe is that you don't need any programming experience, you just write the configuration or protocol files, that is .prototxt files, to perform experiments and research with deep learning. Here, we focus on the protocol-based approach.

Caffe is a very powerful library that enables quick model building, training, and testing; however, it's a bit difficult to install the library to get a lot of benefits from it. As you can see from the installation guide at http://caffe.berkeleyvision.org/installation.html, you need to install the following in advance:

• CUDA

• BLAS (ATLAS, MKL, or OpenBLAS)

• OpenCV

• Boost

• Others: snappy, leveldb, gflags, glog, szip, lmdb, protobuf, and hdf5

Then, clone the repository from...

Summary

In this chapter, you learned how to implement deep learning algorithms and models using Theano, TensorFlow, and Caffe. All of them have special and powerful features and each of them is very useful. If you are interested in other libraries and frameworks, you can have Chainer (http://chainer.org/), Torch (http://torch.ch/), Pylearn2 (http://deeplearning.net/software/pylearn2/), Nervana (http://neon.nervanasys.com/), and so on. You can also reference some benchmark tests (https://github.com/soumith/convnet-benchmarks and https://github.com/soumith/convnet-benchmarks/issues/66) when you actually consider building your application with one of the libraries mentioned earlier.

Throughout this book, you learned the fundamental theories and algorithms of machine learning and deep learning and how deep learning is applied to study/business fields. With the knowledge and techniques you've acquired here, you should be able to cope with any problems that confront you. While it is true that...