Matrix Multiplication

When you first learned multiplication, you learned that $3 \times 4 = 12$. Simple: take two numbers, multiply them, get a number. You might expect matrix multiplication to work the same way, just multiplying corresponding entries. But it does not. Matrix multiplication follows a completely different rule, one that seems strange at first but turns out to be exactly what you need for describing transformations, solving systems of equations, and countless applications across science and engineering.

The rule for multiplying matrices was not invented arbitrarily. It was designed to capture something fundamental: the idea of doing one thing after another. When you rotate an image and then scale it, the combined effect is another transformation. Matrix multiplication gives you the matrix for that combined transformation. This connection to composition, to doing one operation followed by another, is why matrix multiplication works the way it does.

In this lesson, you will learn the mechanics of matrix multiplication, understand why the rule makes sense, and see how this operation differs from ordinary number multiplication in some surprising ways.

Core Concepts

Why Matrix Multiplication Is Defined the Way It Is

Before diving into the mechanics, let us understand why matrix multiplication has such an unusual definition. The key insight is that matrices represent linear transformations, functions that take vectors as input and produce vectors as output.

Suppose matrix $A$ transforms vectors in one way, and matrix $B$ transforms vectors in another way. If you want to apply $B$ first, then $A$, you need a single matrix that captures the combined effect. That matrix is the product $AB$.

The definition of matrix multiplication is designed so that:

$$(AB)\vec{x} = A(B\vec{x})$$

In words: multiplying a vector by $AB$ gives the same result as first multiplying by $B$, then multiplying the result by $A$. This composition property is what makes matrix multiplication useful, and it is what dictates the specific rule we use.

The Rule: Row Times Column

Here is the fundamental rule for matrix multiplication. To find the entry in row $i$ and column $j$ of the product $AB$:

Take row $i$ of $A$ and column $j$ of $B$, then compute their dot product.

The dot product means you multiply corresponding entries and add up the results. If row $i$ of $A$ is $(a_1, a_2, \ldots, a_n)$ and column $j$ of $B$ is $\begin{pmatrix} b_1 \ b_2 \ \vdots \ b_n \end{pmatrix}$, then:

$$(AB)_{ij} = a_1 b_1 + a_2 b_2 + \cdots + a_n b_n$$

Let us see this with a small example. Suppose:

$$A = \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} \quad \text{and} \quad B = \begin{pmatrix} 5 & 6 \ 7 & 8 \end{pmatrix}$$

To find the $(1,1)$-entry of $AB$, take row 1 of $A$ which is $(1, 2)$ and column 1 of $B$ which is $\begin{pmatrix} 5 \ 7 \end{pmatrix}$:

$$(AB)_{11} = 1 \cdot 5 + 2 \cdot 7 = 5 + 14 = 19$$

To find the $(1,2)$-entry of $AB$, take row 1 of $A$ and column 2 of $B$:

$$(AB)_{12} = 1 \cdot 6 + 2 \cdot 8 = 6 + 16 = 22$$

To find the $(2,1)$-entry of $AB$, take row 2 of $A$ and column 1 of $B$:

$$(AB)_{21} = 3 \cdot 5 + 4 \cdot 7 = 15 + 28 = 43$$

To find the $(2,2)$-entry of $AB$, take row 2 of $A$ and column 2 of $B$:

$$(AB)_{22} = 3 \cdot 6 + 4 \cdot 8 = 18 + 32 = 50$$

So:

$$AB = \begin{pmatrix} 19 & 22 \ 43 & 50 \end{pmatrix}$$

Dimension Requirements

Not every pair of matrices can be multiplied. For the row-times-column rule to work, the length of each row of $A$ must equal the length of each column of $B$. This means:

The number of columns of $A$ must equal the number of rows of $B$.

If $A$ is an $m \times n$ matrix (m rows, n columns) and $B$ is an $n \times p$ matrix (n rows, p columns), then the product $AB$ exists and is an $m \times p$ matrix.

$$(m \times n)(n \times p) = m \times p$$

Notice that the inner dimensions ($n$ and $n$) must match, and the outer dimensions ($m$ and $p$) give you the size of the result.

Here are some examples:

• $(2 \times 3)(3 \times 4) = 2 \times 4$ (valid, result is $2 \times 4$)
• $(3 \times 2)(2 \times 2) = 3 \times 2$ (valid, result is $3 \times 2$)
• $(2 \times 3)(2 \times 2)$ is undefined (3 does not equal 2)
• $(4 \times 1)(1 \times 4) = 4 \times 4$ (valid, result is $4 \times 4$)

Two matrices whose dimensions allow multiplication are called conformable for multiplication.

Matrix Multiplication Is NOT Commutative

Here is one of the biggest surprises about matrix multiplication: in general, $AB \neq BA$.

This is a fundamental difference from ordinary number multiplication, where $3 \times 5 = 5 \times 3$. With matrices, the order matters. Let us see why.

First, $AB$ and $BA$ might not even have the same dimensions. If $A$ is $2 \times 3$ and $B$ is $3 \times 4$, then $AB$ is $2 \times 4$, but $BA$ is not even defined (you cannot multiply a $3 \times 4$ by a $2 \times 3$).

Even when both products are defined and have the same dimensions, they usually give different results. Consider:

$$A = \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} \quad \text{and} \quad B = \begin{pmatrix} 0 & 1 \ 1 & 0 \end{pmatrix}$$

We can compute $AB$:

$$AB = \begin{pmatrix} 1 \cdot 0 + 2 \cdot 1 & 1 \cdot 1 + 2 \cdot 0 \ 3 \cdot 0 + 4 \cdot 1 & 3 \cdot 1 + 4 \cdot 0 \end{pmatrix} = \begin{pmatrix} 2 & 1 \ 4 & 3 \end{pmatrix}$$

And $BA$:

$$BA = \begin{pmatrix} 0 \cdot 1 + 1 \cdot 3 & 0 \cdot 2 + 1 \cdot 4 \ 1 \cdot 1 + 0 \cdot 3 & 1 \cdot 2 + 0 \cdot 4 \end{pmatrix} = \begin{pmatrix} 3 & 4 \ 1 & 2 \end{pmatrix}$$

Clearly, $\begin{pmatrix} 2 & 1 \ 4 & 3 \end{pmatrix} \neq \begin{pmatrix} 3 & 4 \ 1 & 2 \end{pmatrix}$.

This makes sense when you think about transformations. Rotating and then scaling gives a different result than scaling and then rotating. The order of operations matters, and matrix multiplication reflects this.

Matrix-Vector Multiplication as Linear Combination

There is a beautiful alternative way to think about matrix-vector multiplication. When you multiply a matrix $A$ by a vector $\vec{x}$, the result is a linear combination of the columns of $A$, with the entries of $\vec{x}$ as the coefficients.

Let us see how this works. Suppose:

$$A = \begin{pmatrix} 1 & 2 \ 3 & 4 \end{pmatrix} \quad \text{and} \quad \vec{x} = \begin{pmatrix} x_1 \ x_2 \end{pmatrix}$$

The columns of $A$ are $\begin{pmatrix} 1 \ 3 \end{pmatrix}$ and $\begin{pmatrix} 2 \ 4 \end{pmatrix}$.

Using the row-times-column rule:

$$A\vec{x} = \begin{pmatrix} 1 \cdot x_1 + 2 \cdot x_2 \ 3 \cdot x_1 + 4 \cdot x_2 \end{pmatrix}$$

But notice this is exactly:

$$A\vec{x} = x_1 \begin{pmatrix} 1 \ 3 \end{pmatrix} + x_2 \begin{pmatrix} 2 \ 4 \end{pmatrix}$$

The vector $\vec{x}$ tells you how much of each column to take. This column-based view is extremely useful for understanding what matrix multiplication really does: it builds new vectors from combinations of the columns.

The Identity Matrix

Just as 1 is the multiplicative identity for numbers ($1 \cdot a = a \cdot 1 = a$), there is a multiplicative identity for matrices. The identity matrix $I$ has 1s on the main diagonal and 0s everywhere else:

$$I_2 = \begin{pmatrix} 1 & 0 \ 0 & 1 \end{pmatrix} \qquad I_3 = \begin{pmatrix} 1 & 0 & 0 \ 0 & 1 & 0 \ 0 & 0 & 1 \end{pmatrix}$$

For any matrix $A$ of compatible dimensions:

$$AI = IA = A$$

Multiplying by the identity matrix leaves any matrix unchanged. When you multiply a vector by $I$, you get the same vector back: $I\vec{x} = \vec{x}$. The identity matrix represents the “do nothing” transformation.

Powers of Matrices

For a square matrix $A$, we can multiply $A$ by itself. We define:

• $A^1 = A$
• $A^2 = AA$ (multiply $A$ by itself)
• $A^3 = AAA = A^2 \cdot A$
• In general, $A^n = \underbrace{A \cdot A \cdot \cdots \cdot A}_{n \text{ times}}$

By convention, $A^0 = I$ (the identity matrix), just as $a^0 = 1$ for numbers.

Powers of matrices are useful for many applications. In Markov chains, $A^n$ tells you the state of a system after $n$ steps. In graph theory, the $(i,j)$-entry of $A^n$ counts the number of paths of length $n$ between vertices $i$ and $j$.

Computing powers by repeated multiplication can be slow, but there are clever techniques (like eigenvalue decomposition) that make it much faster for large $n$.

Notation and Terminology

Term	Meaning	Example
$AB$	Matrix product	$(AB)_{ij} = \text{row } i \text{ of } A \cdot \text{col } j \text{ of } B$
Conformable	Dimensions allow multiplication	$3 \times 2$ and $2 \times 4$ are conformable
$A^n$	$A$ multiplied by itself $n$ times	$A^2 = AA$
$(AB)_{ij}$	Entry in row $i$, column $j$ of $AB$	The dot product of row $i$ of $A$ with column $j$ of $B$
$I$ or $I_n$	Identity matrix ($n \times n$)	$\begin{pmatrix} 1 & 0 \ 0 & 1 \end{pmatrix}$
Non-commutative	Order of multiplication matters	$AB \neq BA$ in general

Examples

Key Properties and Rules

Properties of Matrix Multiplication

Matrix multiplication satisfies some familiar algebraic properties, but notably lacks commutativity.

Associativity: For matrices $A$, $B$, and $C$ of compatible dimensions:

$$(AB)C = A(BC)$$

You can group multiplications however you like, as long as you keep the order the same.

Distributivity: Matrix multiplication distributes over addition:

$$A(B + C) = AB + AC$$ $$(A + B)C = AC + BC$$

Scalar multiplication: For a scalar $k$:

$$k(AB) = (kA)B = A(kB)$$

Identity: For any matrix $A$:

$$AI = IA = A$$

(where $I$ is the identity matrix of appropriate size)

NOT Commutative: In general:

$$AB \neq BA$$

Dimension Rules

If $A$ is…	And $B$ is…	Then $AB$ is…
$m \times n$	$n \times p$	$m \times p$
$2 \times 3$	$3 \times 4$	$2 \times 4$
$3 \times 2$	$2 \times 1$	$3 \times 1$
$m \times n$	$p \times q$ where $n \neq p$	undefined

Properties of Matrix Powers

For a square matrix $A$ and non-negative integers $m$ and $n$:

• $A^0 = I$
• $A^1 = A$
• $A^{m+n} = A^m A^n$
• $A^{mn} = (A^m)^n$

The Transpose of a Product

The transpose of a product reverses the order:

$$(AB)^T = B^T A^T$$

This “reversal rule” is important. The transpose of $AB$ is not $A^T B^T$; you must reverse the order.

Real-World Applications

Composing Transformations

When you apply one transformation after another, the combined effect is described by multiplying the corresponding matrices. In computer graphics, this is used constantly. A game engine might rotate a character, scale it, and then translate (move) it. Each operation has its own matrix, and the combined transformation is:

$$T_{\text{total}} = T_{\text{translate}} \cdot T_{\text{scale}} \cdot T_{\text{rotate}}$$

By multiplying these matrices once, the engine can apply the combined transformation to every vertex of a 3D model efficiently. This is far faster than applying three separate transformations to each vertex.

Markov Chains

A Markov chain models systems that transition between states with fixed probabilities. The transition matrix $P$ has entries $p_{ij}$ representing the probability of going from state $j$ to state $i$.

If $\vec{v}$ represents the current probability distribution across states, then $P\vec{v}$ gives the distribution after one step. After $n$ steps, the distribution is $P^n\vec{v}$.

Applications include:

• Weather prediction (sunny/cloudy/rainy states)
• PageRank algorithm (Google’s original search ranking)
• Genetics (allele frequency over generations)
• Economics (market state transitions)

Neural Networks

Modern artificial intelligence relies heavily on matrix multiplication. In a neural network, each layer performs a computation of the form:

$$\vec{y} = \sigma(W\vec{x} + \vec{b})$$

where $W$ is a weight matrix, $\vec{x}$ is the input vector, $\vec{b}$ is a bias vector, and $\sigma$ is an activation function.

The weight matrix $W$ contains the learned parameters of the network. Training a neural network means finding the right values for all these matrix entries. When the network makes a prediction, it multiplies matrices over and over, layer after layer. GPUs are specifically designed to perform these matrix multiplications quickly.

Computer Graphics

In 3D graphics, points and objects are transformed using matrices:

• Rotation matrices rotate objects around an axis
• Scaling matrices make objects larger or smaller
• Translation matrices (in homogeneous coordinates) move objects
• Projection matrices convert 3D scenes to 2D images on your screen

By combining these matrices through multiplication, graphics systems can efficiently handle complex scenes with thousands of objects, each with its own position, orientation, and scale.

Self-Test Problems

Summary

• Matrix multiplication uses the row-times-column rule: $(AB)_{ij}$ is the dot product of row $i$ of $A$ with column $j$ of $B$.

• Dimension requirement: To compute $AB$, the number of columns of $A$ must equal the number of rows of $B$. If $A$ is $m \times n$ and $B$ is $n \times p$, then $AB$ is $m \times p$.

• Two matrices whose dimensions allow multiplication are called conformable.

• Matrix multiplication is NOT commutative: In general, $AB \neq BA$. The order matters.

• Matrix-vector multiplication can be viewed as a linear combination of the columns of the matrix, with the vector entries as coefficients.

• The identity matrix $I$ satisfies $AI = IA = A$ for any compatible matrix $A$.

• Matrix powers are defined for square matrices: $A^2 = AA$, $A^3 = AAA$, and so on.

• Matrix multiplication is associative $(ABC) = A(BC)$ and distributive $A(B + C) = AB + AC$.

• The transpose of a product reverses the order: $(AB)^T = B^T A^T$.

• Applications include composing transformations (graphics), Markov chains (probability), neural networks (AI), and combining rotations, scales, and translations in computer graphics.

• The unusual definition of matrix multiplication is designed so that $(AB)\vec{x} = A(B\vec{x})$, capturing the idea of composition: doing one transformation, then another.