Supervised learning is the most common form of machine learning. With supervised learning, a set of examples, the training set, is submitted as input to the system during the training phase, where each example is labeled with the respective desired output value. For example, let's consider a classification problem, where the system must attribute some experimental observations in one of the N different classes already known. In this problem, the training set is presented as a sequence of pairs of the type {(X1, Y1), ....., (Xn, Yn)} where Xi are the input vectors (feature vectors) and Yi represents the desired class for the corresponding input vector. Most supervised learning algorithms share one characteristic: the training is performed by the minimization of a particular loss function (cost function), which represents the output error with respect to the desired output system.

The cost function most used for this type of training calculates the standard deviation between the desired output and the one supplied by the system. After training, the accuracy of the model is measured on a set of disjointed examples from the training set, the so-called validation set.

Supervised learning workflow

In this phase the model's generalization capability is then verified: we will test if the output is correct for an unused input during the training phase.

Unsupervised learning

In unsupervised learning, the training examples provided by the system are not labeled with the related belonging class. The system, therefore, develops and organizes the data, looking for common characteristics among them, and changing them based on their internal knowledge.

Unsupervised learning algorithms are particularly used in clustering problems, in which a number of input examples are present, you do not know the class a priori, and you do not even know what the possible classes are, or how numerous they are. This is a clear case when you cannot use supervised learning, because you do not know a priori the number of classes.

Unsupervised learning workflow

Deep learning

Deep learning techniques represent a remarkable step forward taken by machine learning in recent decades, having provided results never seen before in many applications, such as image and speech recognition or Natural Language Processing (NLP). There are several reasons that led to deep learning being developed and placed at the center of the field of machine learning only in recent decades. One reason, perhaps the main one, is surely represented by progress in hardware, with the availability of new processors, such as graphics processing units (GPUs), which have greatly reduced the time needed for training networks, lowering them by a factor of 10 or 20. Another reason is certainly the ever more numerous datasets on which to train a system, needed to train architectures of a certain depth and with a high dimensionality for the input data.

Deep learning workflow

Deep learning is based on the way the human brain processes information and learns, responding to external stimuli. It consists in a machine learning model at several levels of representation in which the deeper levels take as input the outputs of the previous levels, transforming them and always abstracting more. Each level corresponds in this hypothetical model to a different area of the cerebral cortex: when the brain receives images, it processes them through various stages such as edge detection and form perception, that is, from a primitive representation level to the most complex. For example, in an image classification problem, each block gradually extracts the features, at various levels of abstraction, inputting of data already processed, by means of filtering operations.

In Python, a line terminator is not required, and the blocks are specified with the indentation. Indent to begin a block and remove indentation to conclude it, that's all. Instructions that require an indented block end with a colon (:). Comments begin with the hash sign (#) and are single-line. Strings on multiple lines are used for multi-line comments. Assignments are accomplished with the equal sign (=). For equality tests we use the double equal (==) symbol. You can increase and decrease a value by using += and -= followed by the addend. This works with many data types, including strings. You can assign and use multiple variables on the same line.

Following are some examples:

>>> myvar = 3
>>> myvar += 2
>>> myvar
5
>>> myvar -= 1
>>> myvar
4
"""This is a comment"""
>>> mystring = "Hello"
>>> mystring += " world."
>>> print mystring
Hello world.

The following code swaps two variables in one line:

>>> myvar, mystring = mystring, myvar

The most significant structures in Python are lists, tuples, and dictionaries. The sets are integrated in Python since version 2.5 (for previous versions, they are available in the sets library). Lists are similar to single-dimensional arrays but you can create lists that contain other lists. Dictionaries are arrays that contain pairs of keys and values (hash table), and tuples are immutable mono-dimensional objects. In Python arrays can be of any type, so you can mix integers, strings, and so on in your lists/dictionaries and tuples. The index of the first object in any type of array is always zero. Negative indices are allowed and counting from the end of the array, -1 is the last element. Variables can refer to functions.

>>> example = [1, ["list1", "list2"], ("one", "tuple")]
>>> mylist = ["Element 1", 2, 3.14]
>>> mylist [0] 
"Element 1"
>>> mylist [-1]
3.14
>>> mydict = {"Key 1": "Val 1", 2: 3, "pi": 3.14}
>>> mydict ["pi"]
3.14
>>> mytuple = (1, 2, 3)
>>> myfunc = len
>>> print myfunc (mylist)
3

You can get an array range using a colon (:). Not specifying the starting index of the range implies the first element; not indicating the final index implies the last element. Negative indices count from the last element (-1 is the last element). Then run the following command:

>>> mylist = ["first element", 2, 3.14]
>>> print mylist [:]
['first element', 2, 3.1400000000000001]
>>> print mylist [0:2]
['first element', 2]
>>> print mylist [-3:-1]
['first element', 2]
>>> print mylist [1:]
[2, 3.14]

Python strings are indicated either with a single quotation mark (') or double (") and are allowed to use a notation within a delimited string on the other ("He said' hello '."It is valid). Strings of multiple lines are enclosed in triple (or single) quotes ("""). Python supports unicode; just use the syntax: "This is a unicode string". To insert values into a string , use the % operator (modulo) and a tuple. Each % is replaced by a tuple element, from left to right, and is permitted to use a dictionary for the replacements.

>>> print "Nome: %s\nNumber: %s\nString: %s" % (myclass.nome, 3, 3 * "-")
Name: Poromenos
Number: 3
String: ---
strString = """this is a string
on multiple lines."""
>>> print "This %(verbo)s un %(name)s." % {"name": "test", "verb": "is"}
This is a test.

The instructions for flow control are if, for, and while. There is the select control flow; in its place, we use if. The for control flow is used to enumerate the members of a list. To get a list of numbers, you use range (number).

rangelist = range(10)
>>> print rangelist
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Let's check if number is one of the numbers in the tuple:

for number in rangelist:
    if number in (3, 4, 7, 9):
        # "Break" ends the for instruction without the else clause
        break
    else:
        # "Continue" continues with the next iteration of the loop
        continue
else:
    # this is an optional "else" 
    # executed only if the loop is not interrupted with "break".
    pass # it does nothing
if rangelist[1] == 2:
    print "the second element (lists are 0-based) is 2"
elif rangelist[1] == 3:
    print "the second element is 3"
else:
    print "I don't know"
while rangelist[1] == 1:
    pass

Functions are declared with the keyword def. Any optional arguments must be declared after those that are mandatory and must have a value assigned. When calling functions using arguments to name you must also pass the value. Functions can return a tuple (tuple unpacking enables the return of multiple values). Lambda functions are in-line. Parameters are passed by reference, but immutable types (tuples, integers, strings, and so on) cannot be changed in the function. This happens because it is only passed through the position of the element in memory, and assigning another object to the variable results in the loss of the object reference earlier.

For example:

# equal to a def f(x): return x + 1
funzionevar = lambda x: x + 1
>>> print funzionevar(1)
2
def passing_example(my_list,my_int):
    my_list.append("new element")
    my_int = 4
    return my_list, my_int
>>> input_my_list = [1, 2, 3]
>>> input_my_int = 10
>>> print passing_example(input_my_list, input_my_int)
([1, 2, 3, 'new element'], 10)
>>> my_list
[1, 2, 3, 'new element']
>>> my_int
10

Python supports multiple inheritance of classes. The variables and private methods are declared by convection (it is not a rule of language) by preceding them with two underscores (__). We can assign attributes (properties) to arbitrary instances of a class.

The following is an example:

class Myclass:
    common = 10
    def __init__(self):
        self.myvariable= 3
    def myfunc(self, arg1, arg2):
        return self.myvariable
# We create an instance of the class
>>> instance= Myclass()
>>> instance.myfunc(1, 2)
3
# This variable is shared by all instances
>>> instance2= Myclass()
>>> instance.common
10
>>> instance2.common
10
# Note here how we use the class name
# Instead of the instance.
>>> Myclass.common = 30
>>> instance.common
30
>>> instance2.common
30
# This does not update the variable in the class, 
# Instead assign a new object to the variable
# of the first instance.
>>> instance.common = 10
>>> instance.common
10
>>> instance2.common
30
>>> Myclass.common = 50
# The value is not changed because "common" is an instance variable.
>>> instance.common
10
>>> instance2.common
50
# This class inherits from Myclass. Multiple inheritance
# is declared like this:
# class AltraClasse(Myclass1, Myclass2, MyclassN)
class AnotherClass(Myclass):
    # The topic "self" is automatically passed 
    # and makes reference to instance of the class, so you can set 
    # of instance variables as above, but within the class.    
def __init__(self, arg1):
        self.myvariable= 3
        print arg1
>>> instance= AnotherClass ("hello")
hello
>>> instance.myfunc(1, 2)
3
# This class does not have a member (property) .test member, but
# We can add one all instance when we want. Note
# .test That will be a member of only one instance.
>>> instance.test = 10
>>> instance.test
10

Exceptions in Python are handled with try-except blocks [exception_name]:

def my_func():
    try:
        # Division by zero causes an exception
        10 / 0
    except ZeroDivisionError:
        print "Oops, error"
    else:
        # no exception, let's proceed
        pass
    finally:
# This code is executed when the block
    # Try..except is already executed and all exceptions
    # Were handled, even if there is a new
    # Exception directly in the block.
        print "finish"
>>> my_func()
Oops, error.
finish

External libraries are imported with import [library name]. You can also use the form [libraryname] import [funcname] to import individual features. Here's an example:

import random
from time import clock
randomint = random.randint(1, 100)
>>> print randomint
64

The following are the steps to install TensorFlow on Mac and Linux system:

               For Ubuntu/Linux 64-bit, CPU:

(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl

                 For Mac OS X, CPU:

(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl

If you want to use your GPU card with TensorFlow, then install another package. I recommend you visit the official documentation to see if your GPU meets the specifications required to support TensorFlow.

Finally, when you've finished, you must disable the virtual environment:

(tensorflow)$ deactivate

If you can't get a Linux-based system, you can install Ubuntu on a virtual machine; just use a free application called VirtualBox, which lets you create a virtual PC on Windows and install Ubuntu in the latter. So you can try the operating system without creating partitions or dealing with cumbersome procedures.

However, it may happen that the Pip installation causes problems, particularly when using the visualization tool TensorBoard (see https://github.com/tensorflow/tensorflow/issues/530 ). To fix this problem, I suggest you build and install TensorFlow, starting form source files, through the following steps:

Open a terminal and type the following lines of code:

>>> import tensorflow as tf
>>> hello = tf.constant("hello TensorFlow!")
>>> sess=tf.Session()

To verify your installation, just type:

>>> print(sess.run(hello))

You should have the following output:

Hello TensorFlow!
>>>

TensorBoard is a visualization tool, devoted to analyzing Data Flow Graph and also to better understand the machine learning models. It can view different types of statistics about the parameters and details of any part of a computer graph graphically. It often happens that a graph of computation can be very complex. A deep neural network can have up to 36,000 nodes. For this reason, TensorBoard collapses nodes in high-level blocks, highlighting the groups with identical structures. Doing so allows a better analysis of the graph, focusing only on the core sections of the computation graph. Also, the visualization process is interactive; user can pan, zoom, and expand the nodes to display the details.

The following figure shows a neural network model with TensorBoard:

A TensorBoard visualization example

TensorBoard's algorithms collapse nodes into high-level blocks and highlight groups with the same structures, while also separating out high-degree nodes. The visualization tool is also interactive: the users can pan, zoom in, expand, and collapse the nodes.

TensorBoard is equally useful in the development and tuning of a machine learning model. For this reason, TensorFlow lets you insert so-called summary operations into the graph. These summary operations monitor changing values (during the execution of a computation) written in a log file. Then TensorBoard is configured to watch this log file with summary information and display how this information changes over time.

Let's consider a basic example to understand the usage of TensorBoard. We have the following example:

import tensorflow as tf
a = tf.constant(10,name="a")
b = tf.constant(90,name="b")
y = tf.Variable(a+b*2, name="y")
model = tf.initialize_all_variables()
with tf.Session() as session:
    merged = tf.merge_all_summaries()
    writer = tf.train.SummaryWriter\
                      ("/tmp/tensorflowlogs",session.graph)   
     session.run(model)
    print(session.run(y))

That gives the following result:

190

Let's point into the session management. The first instruction to consider is as follows:

merged = tf.merge_all_summaries()

This instruction must merge all the summaries collected in the default graph.

Then we create SummaryWriter. It will write all the summaries (in this case the execution graph) obtained from the code's execution into the /tmp/tensorflowlogs directory:

writer = tf.train.SummaryWriter\
                    ("/tmp/tensorflowlogs",session.graph)

Finally, we run the model and so build the Data Flow Graph:

session.run(model)
print(session.run(y))

The use of TensorBoard is very simple. Let's open a terminal and enter the following:

$tensorboard --logdir=/tmp/tensorflowlogs

A message such as the following should appear:

startig tensorboard on port 6006

Then, by opening a web browser, we should display the Data Flow Graph with auxiliary nodes:

Data Flow Graph display with TensorBoard

Now we will be able to explore the Data Flow Graph:

Explore the Data Flow Graph display with TensorBoard

TensorBoard uses special icons for constants and summary nodes. To summarize, we report in the next figure the table of node symbols displayed:

Node symbols in TensorBoard