Introduction to Matrices and Matrix Arithmetic for Machine Learning
Matrices are a foundational element of linear algebra.
Matrices are used throughout the field of machine learning in the description of algorithms and processes such as the input data variable (X) when training an algorithm.
In this tutorial, you will discover matrices in linear algebra and how to manipulate them in Python.
After completing this tutorial, you will know:
- What a matrix is and how to define one in Python with NumPy.
- How to perform element-wise operations such as addition, subtraction, and the Hadamard product.
- How to multiply matrices together and the intuition behind the operation.
Let’s get started.
A Gentle Introduction to Matrices for Machine Learning
Photo by Maximiliano Kolus, some rights reserved.
Tutorial Overview
This tutorial is divided into 6 parts; they are:
- What is a Matrix?
- Defining a Matrix
- Matrix Arithmetic
- Matrix-Matrix Multiplication (Dot Product)
- Matrix-Vector Multiplication
- Matrix-Scalar Multiplication
Need help with Linear Algebra for Machine Learning?
Take my free 7-day email crash course now (with sample code).
Click to sign-up and also get a free PDF Ebook version of the course.
What is a Matrix?
A matrix is a two-dimensional array of scalars with one or more columns and one or more rows.
A matrix is a two-dimensional array (a table) of numbers.
The notation for a matrix is often an uppercase letter, such as A, and entries are referred to by their two-dimensional subscript of row (i) and column (j), such as aij. For example:
| 1 | A = ((a11, a12), (a21, 22), (a31, a32)) |
It is more common to see matrices defined using a horizontal notation.
| 123 | a11, a12A = (a21, a22) a31, a32 |
A likely first place you may encounter a matrix in machine learning is in model training data comprised of many rows and columns and often represented using the capital letter “X”.
The geometric analogy used to help understand vectors and some of their operations does not hold with matrices. Further, a vector itself may be considered a matrix with one column and multiple rows.
Often the dimensions of the matrix are denoted as m and n for the number of rows and the number of columns.
Now that we know what a matrix is, let’s look at defining one in Python.
Defining a Matrix
We can represent a matrix in Python using a two-dimensional NumPy array.
A NumPy array can be constructed given a list of lists. For example, below is a 3 row, 2 column matrix.
| 1234 | # create matrixfrom numpy import arrayA=array([[1,2,3],[4,5,6]])print(A) |
Running the example prints the created matrix showing the expected structure.
| 12 | [[1 2 3] [4 5 6]] |
Matrix Arithmetic
In this section will demonstrate simple matrix-matrix arithmetic, where all operations are performed element-wise between two matrices of equal size to result in a new matrix with the same size.
Matrix Addition
Two matrices with the same dimensions can be added together to create a new third matrix.
| 1 | C = A + B |
The scalar elements in the resulting matrix are calculated as the addition of the elements in each of the matrices being added.
| 123 | a11 + b11, a12 + b12A + B = (a21 + b21, a22 + b22) a31 + b31, a32 + b32 |
Or, put another way:
| 123456 | C[0,0] = A[0,0] + B[0,0]C[1,0] = A[1,0] + B[1,0]C[2,0] = A[2,0] + B[2,0]C[0,1] = A[0,1] + B[0,1]C[1,1] = A[1,1] + B[1,1]C[2,1] = A[2,1] + B[2,1] |
We can implement this in python using the plus operator directly on the two NumPy arrays.
| 12345678 | # add matricesfrom numpy import arrayA=array([[1,2,3],[4,5,6]])print(A)B=array([[1,2,3],[4,5,6]])print(B)C=A+Bprint(C) |
The example first defines two 2×3 matrices and then adds them together.
Running the example first prints the two parent matrices and then the result of adding them together.
| 12345678 | [[1 2 3] [4 5 6]][[1 2 3] [4 5 6]][[ 2 4 6] [ 8 10 12]] |
Matrix Subtraction
Similarly, one matrix can be subtracted from another matrix with the same dimensions.
| 1 | C = A - B |
The scalar elements in the resulting matrix are calculated as the subtraction of the elements in each of the matrices.
| 123 | a11 - b11, a12 - b12A - B = (a21 - b21, a22 - b22) a31 - b31, a32 - b32 |
Or, put another way:
| 123456 | C[0,0] = A[0,0] - B[0,0]C[1,0] = A[1,0] - B[1,0]C[2,0] = A[2,0] - B[2,0]C[0,1] = A[0,1] - B[0,1]C[1,1] = A[1,1] - B[1,1]C[2,1] = A[2,1] - B[2,1] |
We can implement this in python using the minus operator directly on the two NumPy arrays.
| 12345678 | # subtract matricesfrom numpy import arrayA=array([[1,2,3],[4,5,6]])print(A)B=array([[0.5,0.5,0.5],[0.5,0.5,0.5]])print(B)C=A-Bprint(C) |
The example first defines two 2×3 matrices and then subtracts one from the other.
Running the example first prints the two parent matrices and then subtracts the first matrix from the second.
| 12345678 | [[1 2 3] [4 5 6]][[ 0.5 0.5 0.5] [ 0.5 0.5 0.5]][[ 0.5 1.5 2.5] [ 3.5 4.5 5.5]] |
Matrix Multiplication (Hadamard Product)
Two matrices with the same size can be multiplied together, and this is often called element-wise matrix multiplication or the Hadamard product.
It is not the typical operation meant when referring to matrix multiplication, therefore a different operator is often used, such as a circle “o”.
| 1 | C = A o B |
As with element-wise subtraction and addition, element-wise multiplication involves the multiplication of elements from each parent matrix to calculate the values in the new matrix.
| 123 | a11 * b11, a12 * b12A o B = (a21 * b21, a22 * b22) a31 * b31, a32 * b32 |
Or, put another way:
| 123456 | C[0,0] = A[0,0] * B[0,0]C[1,0] = A[1,0] * B[1,0]C[2,0] = A[2,0] * B[2,0]C[0,1] = A[0,1] * B[0,1]C[1,1] = A[1,1] * B[1,1]C[2,1] = A[2,1] * B[2,1] |
We can implement this in python using the star operator directly on the two NumPy arrays.
| 12345678 | # element-wise multiply matricesfrom numpy import arrayA=array([[1,2,3],[4,5,6]])print(A)B=array([[1,2,3],[4,5,6]])print(B)C=A *Bprint(C) |
The example first defines two 2×3 matrices and then multiplies them together.
Running the example first prints the two parent matrices and then the result of multiplying them together with a Hadamard Product.
| 12345678 | [[1 2 3] [4 5 6]][[1 2 3] [4 5 6]][[ 1 4 9] [16 25 36]] |
Matrix Division
One matrix can be divided by another matrix with the same dimensions.
| 1 | C = A / B |
The scalar elements in the resulting matrix are calculated as the division of the elements in each of the matrices.
| 123 | a11 / b11, a12 / b12A / B = (a21 / b21, a22 / b22) a31 / b31, a32 / b32 |
Or, put another way:
| 123456 | C[0,0] = A[0,0] / B[0,0]C[1,0] = A[1,0] / B[1,0]C[2,0] = A[2,0] / B[2,0]C[0,1] = A[0,1] / B[0,1]C[1,1] = A[1,1] / B[1,1]C[2,1] = A[2,1] / B[2,1] |
We can implement this in python using the division operator directly on the two NumPy arrays.
| 12345678 | # divide matricesfrom numpy import arrayA=array([[1,2,3],[4,5,6]])print(A)B=array([[1,2,3],[4,5,6]])print(B)C=A/Bprint(C) |
The example first defines two 2×3 matrices and then divides the first from the second matrix.
Running the example first prints the two parent matrices and then divides the first matrix by the second.
| 12345678 | [[1 2 3] [4 5 6]][[1 2 3] [4 5 6]][[ 1. 1. 1.] [ 1. 1. 1.]] |