A quick review of all the models

Let's recap each of the models we built, to model these fonts and some of their strengths and weaknesses:

At a glance, recall that we slowly built up more complicated models and took into account the structure of the data to improve our accuracy.

The logistic regression model

First, we started with a simple logistic regression model:

This has 36x36 pixels plus 1 bias times 5 classes total weights, or 6,485 parameters that we need to train. After 1,000 training epochs, this model achieved about 40 percent validation accuracy. Your results may vary. This is relatively poor, but the model has some advantages.

Let's glance back at the code:

# These will be inputs
## Input pixels, flattened
x = tf.placeholder("float", [None, 1296])
## Known labels
y_ = tf.placeholder("float", [None,5])

# Variables
W = tf.Variable(tf.zeros([1296,5]))
b = tf.Variable(tf.zeros([5]))

# Just initialize
sess.run(tf.initialize_all_variables())

# Define model
y = tf.nn.softmax(tf.matmul...