Solving a system of linear equations

The matrix-vector multiplication operation gives rise to a system of equations. In a typical machine learning algorithm, data comes in the form of a matrix, X, and the target outcome is a vector, y. When the model that’s used is a straightforward linear model, we assume the input-output relationship as Xw = y, where w represents the vector of features/coefficients. An n x p matrix of input data multiplies a p x 1 vector, w, of features to produce, as expected, an n x 1 output vector, y. The essence of linear regress is thus to solve for the exact values in w such that the system of linear equations in Xw = y are satisfied.

The equivalence between matrix-vector multiplication and the system of linear equations may take some time to become noticeable. Let’s pause and look at this equivalence.

System of linear equations

We are already familiar with the process of calculating a matrix-vector multiplication operation. A 2x2 matrix...