Yes, in fact it is, and let's show you how. Consider an input vector as the coordinates of a point. For a vector with n elements, the point would like it's in a n-dimensional space. Take a sheet of paper, and on this paper, draw a set of points. Now separate those two points by a single straight line. Your piece of paper should now look something like the following:
As you can see, the points are now divided into two sets, one set on each side of the line. If we can take a single line and clearly separate all the points, then those two sets are what is known as linearly separable.
Our single perceptron, believe it or not, will be able to learn where this line is, and when your program is complete, the perceptron will also be able to tell whether a single point is above or below the line (or to the left or the right of it, depending upon how the line was drawn).
Let's quickly code a Perceptron class, just so it becomes clearer for those of you who love to read code more than words (like me!). The goal will be to create a simple perceptron that can determine which side of the line a point should be on, just like the previous diagram:
class Perceptron {
float[] weights;
The constructor could receive an argument indicating the number of inputs (in this case three: x, y, and a bias) and size the array accordingly:
Perceptron(int n) {
weights = new float[n];
for (int i = 0; i<weights.length; i++) {
The weights are picked randomly to start with:
weights[i] = random(-1,1);
}
}
Next, we'll need a function for the perceptron to receive its information, which will be the same length as the array of weights, and then return the output value to us. We'll call this feedforward:
int feedforward(float[] inputs) {
float sum = 0;
for (int i = 0; i<weights.length; i++) {
sum += inputs[i]*weights[i];
}
The result is the sign of the sum, which will be either -1 or +1. In this case, the perceptron is attempting to guess which side of the line the output should be on:
return activate(sum);
}
Thus far, we have a minimally functional perceptron that should be able to make an educated guess as to where our point will lie.
Create the Perceptron:
Perceptron p = new Perceptron(3);
The input is 3 values: x, y, and bias:
float[] point = {5,-2,19};
Obtain the answer:
int result = p.feedforward(point);
The only thing left that will make our perceptron more valuable is the ability to train it rather than have it make educated guesses. We do that by creating a train function such as this:
- We will introduce a new variable to control the learning rate:
float c = 0.01;
- We will also provide the inputs and the known answer:
void train(float[] inputs, int desired) {
- And we will make an educated guess according to the inputs provided:
int guess = feedforward(inputs);
- We will compute the error, which is the difference between the answer and our guess:
float error = desired - guess;
- And, finally, we will adjust all the weights according to the error and learning constant:
for (int i = 0; i<weights.length; i++) {
weights[i] += c * error * inputs[i];
So, now that you know and see what a perceptron is, let's add activation functions into the mix and take it to the next level!