Getting Started with TensorFlow – Linear Regression and Beyond
In this example, we will take a closer look at TensorFlow's and TensorBoard's main concepts and try to do some basic operations to get you started. The model we want to implement simulates the linear regression.
In the statistics and machine learning realm, linear regression is a technique frequently used to measure the relationship between variables. This is also a quite simple but effective algorithm that can be used in predictive modeling too. Linear regression models the relationship between a dependent variable yi, an interdependent variable xi, and a random term b. This can be seen as follows:
Now to conceptualize the preceding equation, I am going to write a simple Python program for creating data into a 2D space. Then I will use TensorFlow to look for the line that best fits in the data points:
If your compiler does not make any complaints, you should observe the following graph:
Well, so far we have just created a few data points without any associated model that could be executed through TensorFlow. So the next step is to create a linear regression model to be able to obtain the output values y that is estimated from the input data points–that is, x_data. In this context, we have only two associated parameters–that is, W and b. Now the objective is to create a graph that allows finding the values for these two parameters based on the input data x_data by adjusting them to y_data–that is, optimization problem.
So the target function in our case would be as follows:
If you recall, we defined W = 0.1 and b = 0.4 while creating the data points in the 2D space. Now TensorFlow has to optimize these two values so that W tends to 0.1 and b to 0.4, but without knowing any optimization function, TensorFlow does not even know anything.
A standard way to solve such optimization problems is to iterate through each value of the data points and adjust the value of W and b in order to get a more precise answer on each iteration. Now to realize if the values are really improving, we need to define a cost function that measures how good a certain line is.
In our case, the cost function is the mean squared error that helps find the average of the errors based on the distance function between the real data points and the estimated ones on each iteration. We start by importing the TensorFlow library:
In the preceding code segment, we are generating a random point using a different strategy and storing in variable W. Now let's define a loss function loss=mean [(y−y_data) 2] and this returns a scalar value with the mean of all distances between our data and the model prediction. In terms of TensorFlow convention, the loss function can be expressed as follows:
Without going into further detail, we can use some widely used optimization algorithms such as gradient descent. At a minimal level, the gradient descent is an algorithm that works on a set of given parameters that we already have. It starts with an initial set of parameter values and iteratively moves toward a set of values that minimize the function by taking another parameter called learning rate. This iterative minimization is achieved by taking steps in the negative direction of the function called gradient.
Before running this optimization function, we need to initialize all the variables that we have so far. Let's do it using TensorFlow convention as follows:
Since we have created a TensorFlow session, we are ready for the iterative process that helps us find the optimal values of W and b:
You should observe the following output:
Thus you can see the algorithm starts with the initial values of W = 0.18418592 and b = 0.47198644 where the loss is pretty high. Then the algorithm iteratively adjusted the values by minimizing the cost function. In the eighth iteration, all the values tend to our desired values.
Now what if we could plot them? Let's do it by adding the plotting line under the for loop as follows:
Now let's iterate the same up to the 16th iteration:
Much better and we're closer to the optimized values, right? Now, what if we further improve our visual analytics through TensorFlow that help visualize what is happening in these graphs. TensorBoard provides a web page for debugging your graph as well as inspecting the used variables, node, edges, and their corresponding connections.
However, to get the facility of the preceding regression analysis, you need to annotate the preceding graphs with the variables such as loss function, W, b, y_data, x_data, and so on. Then you need to generate all the summaries by invoking the function tf.summary.merge_all().
Now, we need to make the following changes to the preceding code. However, it is a good practice to group related nodes on the graph using the tf.name_scope() function. Thus, we can use tf.name_scope() to organize things on the TensorBoard graph view, but let's give it a better name:
Then let's annotate the loss function in a similar way, but by giving a suitable name such as LossFunction:
Let's annotate the loss, weights, and bias that are needed for the TensorBoard:
Well, once you annotate the graph, it's time to configure the summary by merging them:
Now before running the training (after the initialization), write the summary using the tf.summary.FileWriter() API as follows:
Then start the TensorBoard as follows:
In our case, it could be something like the following:
Now let's move to http://localhost:6006 and on clicking on the GRAPHS tab, you should see the following graph:
Source Code for the Linear Regression
We reported for the entire source code for the example previously described:
Ubuntu may ask you to install the python-tk package. You can do it by executing the following command on Ubuntu:
Argentina
Australia
Austria
Belgium
Brazil
Bulgaria
Canada
Chile
Colombia
Cyprus
Czechia
Denmark
Ecuador
Egypt
Estonia
Finland
France
Germany
Great Britain
Greece
Hungary
India
Indonesia
Ireland
Italy
Japan
Latvia
Lithuania
Luxembourg
Malaysia
Malta
Mexico
Netherlands
New Zealand
Norway
Philippines
Poland
Portugal
Romania
Russia
Singapore
Slovakia
Slovenia
South Africa
South Korea
Spain
Sweden
Switzerland
Taiwan
Thailand
Turkey
Ukraine
United States
Filter
