Why matrix multiplication is defined the way it is
Multiplying matrices means doing one transformation after another. The rows-times-columns rule, the failure of AB = BA and associativity all follow from that.
AssumesA matrix is a transformation of spaceWhat is a vector? Arrows, lists of numbers, and adding them
Matrices are added entry by entry, so you might expect them to be multiplied entry by entry too. The rule that is actually taught looks nothing like that:
Each entry of the answer mixes a row of the first matrix with a column of the second. Worse, the result depends on the order: $AB$ and $BA$ are usually different matrices, which never happens with numbers. A rule this strange must have been forced on us by something, and this lesson shows by what. Once you see the source, you will be able to rebuild the rule from nothing, and you will know why order matters before computing anything.
What I rely on from the previous lesson fits in three sentences. A map $T$ of the plane is linear if $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$ and $T(c\,\mathbf{v}) = c\,T(\mathbf{v})$ for all vectors $\mathbf{u}, \mathbf{v}$ and numbers $c$. A linear map is completely determined by where it sends $\mathbf{i} = (1, 0)$ and $\mathbf{j} = (0, 1)$, and writing those two landing spots as columns gives its matrix. Applying the matrix to a vector is
One transformation after another
If a matrix is a motion of the plane, the natural thing to do with two matrices is to perform one motion and then the other. Take two that are easy to picture. The quarter turn $R$ rotates the plane $90^\circ$ counterclockwise; it sends $\mathbf{i}$ to $(0, 1)$ and $\mathbf{j}$ to $(-1, 0)$. The shear $S$ slides each point sideways by its height; it leaves $\mathbf{i}$ at $(1, 0)$ and sends $\mathbf{j}$ to $(1, 1)$. As matrices,
Turn the plane with $R$, then shear the result with $S$. The combined motion takes each vector $\mathbf{v}$ to $S(R\mathbf{v})$. The combined motion is again given by a matrix provided it is linear, and that is quick to check for any two linear maps.
So “turn, then shear” has a matrix, and we know how to find the matrix of any linear map: see where $\mathbf{i}$ and $\mathbf{j}$ end up.
Pause and try
Follow $\mathbf{i}$ through the quarter turn and then the shear. Where does it end? Do the same for $\mathbf{j}$, and write the two results as the columns of a matrix. (The shear sends $(x, y)$ to $(x + y, y)$.)
Show answerHide answer
The turn sends $\mathbf{i}$ to $(0, 1)$, and the shear sends $(0, 1)$ to $(0 + 1, 1) = (1, 1)$. The turn sends $\mathbf{j}$ to $(-1, 0)$, and the shear, which does not move points of height 0, leaves it at $(-1, 0)$. The matrix of “turn, then shear” is $\begin{pmatrix} 1 & -1 \\ 1 & 0 \end{pmatrix}$.
The rule, derived
Nothing in that computation was special to a turn and a shear. Let $A$ and $B$ be any two matrices,
and apply $B$ first, then $A$. The vector $\mathbf{i}$ goes first to $B\mathbf{i} = (e, g)$, the first column of $B$. Then $A$ acts on that vector. Here is the step that produces the rule: $(e, g)$ is $e$ steps of $\mathbf{i}$ and $g$ steps of $\mathbf{j}$, so $A$ carries it to $e$ steps of $A$’s first column and $g$ steps of $A$’s second,
The figure runs this step on numbers: $A$ is the shear $S$ and $B = \begin{pmatrix} 2 & -1 \\ 1 & 1 \end{pmatrix}$.
The same reasoning applied to $\mathbf{j}$, which $B$ sends to its second column $(f, h)$, gives $A(f, h) = (af + bh,\ cf + dh)$. Those two vectors are where “$B$, then $A$” sends the basis, so they are the columns of its matrix. That matrix is what we define the product to be.
The definition arranges for $AB$ to do the right thing to $\mathbf{i}$ and $\mathbf{j}$. We should confirm that it does the right thing to every vector.
Now look back at the entry in row 1, column 1: $ae + bg$. It pairs the first row of $A$, $(a, b)$, with the first column of $B$, $(e, g)$. The reason is visible in the derivation. A column of $B$ is a vector being fed to $A$, and each row of $A$ produces one component of the output. So the entry in row $r$ and column $s$ of $AB$ is always row $r$ of $A$ paired with column $s$ of $B$. Entry-by-entry multiplication would be simpler to carry out, but it would not describe one motion followed by another, or anything else of geometric interest.
The order of the letters deserves a comment. $AB$ applies $B$ first because the vector stands on the right: in $A(B\mathbf{v})$ the matrix next to $\mathbf{v}$ acts first, as with functions, where $f(g(x))$ applies $g$ first.
Order matters
We found that “turn, then shear” is $SR = \begin{pmatrix} 1 & -1 \\ 1 & 0 \end{pmatrix}$. Now reverse the order.
Pause and try
Compute $RS$, the matrix of “shear, then turn”, by tracking $\mathbf{i}$ and $\mathbf{j}$. Compare it with $SR$. (The turn sends $(x, y)$ to $(-y, x)$.)
Show answerHide answer
The shear leaves $\mathbf{i}$ at $(1, 0)$, and the turn sends that to $(0, 1)$. The shear sends $\mathbf{j}$ to $(1, 1)$, and the turn sends that to $(-1, 1)$. So $RS = \begin{pmatrix} 0 & -1 \\ 1 & 1 \end{pmatrix}$, which is not $SR$. In the animation at the top of the page these are the right and left panels.
This is no accident of the example. Shearing slides points along horizontal lines. If the turn comes first, the lines that were horizontal are vertical by the time the shear arrives, so the shear acts on different material. For motions, order matters far more often than not: putting on socks and then shoes is not the same as shoes and then socks. Numbers commute because stretching a line by 2 and then by 3 is the same as by 3 and then by 2, and that is a special feature of stretches.
Some pairs of matrices do commute, and it is worth knowing which. Two rotations about the origin commute: turning by $30^\circ$ and then by $60^\circ$ gives the same quarter turn as $60^\circ$ and then $30^\circ$.
Grouping does not matter
With three matrices there is a second worry. The expression $ABC$ could mean $(AB)C$, where we first compute the matrix $AB$ and then multiply by $C$, or $A(BC)$. Checking that these agree by brute force means expanding eight entries, each a sum of four triple products. The transformation view makes the question disappear.
No entry was ever expanded. Both groupings describe “do $C$, then $B$, then $A$”, and the brackets only say where you pause to write down an intermediate matrix. This is the standard example of a fact that is tedious in coordinates and immediate once the objects are understood as functions, because composition of functions is always associative.
Doing nothing, repeating, and losing everything
The identity. The motion that leaves every vector alone sends $\mathbf{i}$ to $\mathbf{i}$ and $\mathbf{j}$ to $\mathbf{j}$, so its matrix is $I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$, the identity matrix. Doing nothing before or after $A$ changes nothing: $AI = IA = A$. It plays the part of the number 1.
Powers. $A^2$ means $AA$: perform the motion twice. Two quarter turns make a half turn, which sends every $(x, y)$ to $(-x, -y)$, so $R^2 = \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix} = -I$ without any arithmetic. (If you have seen complex numbers, this is $i^2 = -1$ again: multiplying by $i$ is a quarter turn.) For the shear, each application slides a point of height $y$ a further $y$ to the right.
Pause and try
Compute $S^2$ and $S^3$ for $S = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ and guess $S^n$.
Show answerHide answer
$S$ fixes $\mathbf{i}$, so the first column is always $(1, 0)$. For the second column: $S\mathbf{j} = (1, 1)$, then $S(1, 1) = (2, 1)$, then $S(2, 1) = (3, 1)$. So $S^2 = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}$, $S^3 = \begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix}$, and the pattern is $S^n = \begin{pmatrix} 1 & n \\ 0 & 1 \end{pmatrix}$ (Problem 5 asks for a proof).
Zero without a zero factor. With numbers, $xy = 0$ forces $x = 0$ or $y = 0$. Let $B = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}$, which flattens the plane onto the vertical axis, and $A = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$, which flattens it onto the horizontal axis. Neither is the zero matrix. But $B$ puts everything on the vertical axis, and $A$ sends the whole vertical axis to the origin, so $AB$ sends every vector to $\mathbf{0}$.
One consequence is that you cannot cancel freely: from $AB = AC$ it does not follow that $B = C$. (Here $AB = A0$, where $0$ is the zero matrix, and $B \neq 0$.) Which matrices can be cancelled is settled in the next lesson: exactly those with an inverse.
Looking back, the definition of the product is “the matrix of one linear map followed by another”, and the rule at the top of the page is the bookkeeping for it. The harder a fact about the bookkeeping looks (non-commutativity, associativity), the more it helps to restate it about the maps, where it is usually plain. The next lesson asks what a matrix does to areas, and the fact that areas scaled by $B$ and then by $A$ are scaled by $AB$ will do real work there.
Check yourself
$A$ is a reflection and $B$ is a rotation. Which matrix describes “reflect first, then rotate”?
Problems
The first three are computations, the next three ask you to reason with the composition picture, and the last two connect to other topics.
Problem 1
Let $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $B = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$. Compute $AB$ and $BA$. Describe in words what multiplying by $B$ on the right did to $A$, and what multiplying on the left did.
Show solutionHide solution
The columns of $AB$ are $A$ applied to the columns of $B$: $A(0, 1) = (2, 4)$ and $A(1, 0) = (1, 3)$, so $AB = \begin{pmatrix} 2 & 1 \\ 4 & 3 \end{pmatrix}$. The columns of $BA$ are $B$ applied to the columns of $A$; since $B$ swaps the two coordinates, $B(1, 3) = (3, 1)$ and $B(2, 4) = (4, 2)$, so $BA = \begin{pmatrix} 3 & 4 \\ 1 & 2 \end{pmatrix}$. Multiplying by $B$ on the right swapped the columns of $A$; multiplying on the left swapped its rows.
Problem 2
Let $F = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}$ be the reflection across the horizontal axis and $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ the quarter turn. Find the matrix of “reflect, then turn” and of “turn, then reflect”, and identify each as a single familiar motion.
Show solutionHide solution
“Reflect, then turn” is $RF$. Track the basis: $F\mathbf{i} = (1, 0)$, and $R(1, 0) = (0, 1)$; $F\mathbf{j} = (0, -1)$, and $R(0, -1) = (1, 0)$. So $RF = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$, which swaps coordinates: the reflection across the line $y = x$. “Turn, then reflect” is $FR$: $R\mathbf{i} = (0, 1)$ and $F(0, 1) = (0, -1)$; $R\mathbf{j} = (-1, 0)$ and $F(-1, 0) = (-1, 0)$. So $FR = \begin{pmatrix} 0 & -1 \\ -1 & 0 \end{pmatrix}$, which sends $(x, y)$ to $(-y, -x)$: the reflection across the line $y = -x$.
Problem 3
Let $D = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$ and $E = \begin{pmatrix} 5 & 0 \\ 0 & -1 \end{pmatrix}$. Compute $DE$ and $ED$, and explain geometrically why they are equal.
Show solutionHide solution
$DE$ has columns $D(5, 0) = (10, 0)$ and $D(0, -1) = (0, -3)$, so $DE = \begin{pmatrix} 10 & 0 \\ 0 & -3 \end{pmatrix}$. $ED$ has columns $E(2, 0) = (10, 0)$ and $E(0, 3) = (0, -3)$: the same matrix. Each of $D$ and $E$ scales the horizontal direction by one number and the vertical direction by another, without mixing the two. Along each axis we are just multiplying numbers, $2 \cdot 5 = 5 \cdot 2$ and $3 \cdot (-1) = (-1) \cdot 3$, and numbers commute.
Problem 4
Find all matrices $B = \begin{pmatrix} e & f \\ g & h \end{pmatrix}$ that commute with the shear $S = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$, that is, with $SB = BS$.
Show solutionHide solution
$SB$ has columns $S(e, g) = (e + g, g)$ and $S(f, h) = (f + h, h)$, so $SB = \begin{pmatrix} e + g & f + h \\ g & h \end{pmatrix}$. $BS$ has columns $B(1, 0) = (e, g)$ and $B(1, 1) = (e + f, g + h)$, so $BS = \begin{pmatrix} e & e + f \\ g & g + h \end{pmatrix}$. Comparing entries: $e + g = e$ gives $g = 0$; $f + h = e + f$ gives $h = e$; the bottom-left entries agree; and $h = g + h$ gives $g = 0$ again. So $B$ commutes with $S$ exactly when $B = \begin{pmatrix} e & f \\ 0 & e \end{pmatrix}$ for some numbers $e, f$: a uniform scaling by $e$ combined with a horizontal shear. Check with $e = 2, f = 5$: $SB = \begin{pmatrix} 2 & 7 \\ 0 & 2 \end{pmatrix} = BS$.
Problem 5
Prove that $S^n = \begin{pmatrix} 1 & n \\ 0 & 1 \end{pmatrix}$ for every whole number $n \geq 1$. More generally, show that shears $\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}$ and $\begin{pmatrix} 1 & m \\ 0 & 1 \end{pmatrix}$ multiply by adding $k$ and $m$.
Show solutionHide solution
Do the general claim first. The product $\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}\begin{pmatrix} 1 & m \\ 0 & 1 \end{pmatrix}$ has first column $(1, 0)$, since both matrices fix $\mathbf{i}$, and second column $\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}$ applied to $(m, 1)$, which is $m\,(1, 0) + 1\,(k, 1) = (m + k, 1)$. So the product is $\begin{pmatrix} 1 & k + m \\ 0 & 1 \end{pmatrix}$. For the powers, argue by induction. The claim holds for $n = 1$. If $S^n = \begin{pmatrix} 1 & n \\ 0 & 1 \end{pmatrix}$, then $S^{n+1} = S\,S^n$ is the product of shears with $k = 1$ and $m = n$, which is $\begin{pmatrix} 1 & n + 1 \\ 0 & 1 \end{pmatrix}$. Geometrically: sliding each point by $m$ times its height and then by $k$ times its height slides it by $(k + m)$ times its height.
Problem 6
Let $P = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$ and $N = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$. Compute $P^2$ and $N^2$, and explain each result by describing what the map does.
Show solutionHide solution
$P$ sends $(x, y)$ to $(x, 0)$: it drops each point straight onto the horizontal axis. A point already on that axis does not move, so doing $P$ twice is the same as doing it once: $P^2 = P$. By columns: $P(1, 0) = (1, 0)$ and $P(0, 0) = (0, 0)$. $N$ sends $(x, y)$ to $(y, 0)$; in particular $N\mathbf{i} = \mathbf{0}$ and $N\mathbf{j} = \mathbf{i}$. Every output of $N$ lies on the horizontal axis, and $N$ sends that whole axis to the origin, so $N^2$ sends everything to $\mathbf{0}$: $N^2 = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix}$. By columns: $N(N\mathbf{i}) = N\mathbf{0} = \mathbf{0}$ and $N(N\mathbf{j}) = N\mathbf{i} = \mathbf{0}$. So a nonzero matrix can have square zero.
Problem 7
Use $R_\theta R_\theta = R_{2\theta}$, where $R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}$, to derive formulas for $\cos 2\theta$ and $\sin 2\theta$. Check them at $\theta = 30^\circ$.
Show solutionHide solution
The first column of $R_\theta R_\theta$ is $R_\theta$ applied to $(\cos\theta, \sin\theta)$: $\cos\theta\,(\cos\theta, \sin\theta) + \sin\theta\,(-\sin\theta, \cos\theta) = (\cos^2\theta - \sin^2\theta,\ 2\sin\theta\cos\theta)$. It must equal the first column of $R_{2\theta}$, namely $(\cos 2\theta, \sin 2\theta)$. Hence $\cos 2\theta = \cos^2\theta - \sin^2\theta$ and $\sin 2\theta = 2\sin\theta\cos\theta$. At $\theta = 30^\circ$: $\cos^2 - \sin^2 = \frac34 - \frac14 = \frac12 = \cos 60^\circ$, and $2 \cdot \frac12 \cdot \frac{\sqrt3}{2} = \frac{\sqrt3}{2} = \sin 60^\circ$.
Problem 8
For real numbers $a, b$ let $M(a, b) = \begin{pmatrix} a & -b \\ b & a \end{pmatrix}$. (a) Show that $M(a, b)\,M(c, d) = M(ac - bd,\ ad + bc)$. (b) Deduce that any two matrices of this form commute. (c) Compare with the product of complex numbers $(a + bi)(c + di)$, and say which matrix plays the part of $i$.
Show solutionHide solution
(a) The first column of the product is $M(a, b)$ applied to $(c, d)$: $c\,(a, b) + d\,(-b, a) = (ac - bd,\ bc + ad)$. The second column is $M(a, b)$ applied to $(-d, c)$: $-d\,(a, b) + c\,(-b, a) = (-(ad + bc),\ ac - bd)$. So the product is $\begin{pmatrix} ac - bd & -(ad + bc) \\ ad + bc & ac - bd \end{pmatrix} = M(ac - bd,\ ad + bc)$. (b) Swapping the roles of $(a, b)$ and $(c, d)$ gives $M(c, d)M(a, b) = M(ca - db,\ cb + da)$, which is the same matrix because multiplication of numbers commutes. (c) $(a + bi)(c + di) = (ac - bd) + (ad + bc)i$: the same rule. The matrix $M(a, b)$ acts on the plane the way multiplication by $a + bi$ acts on complex numbers. The part of $i$ is played by $M(0, 1) = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$, the quarter turn $R$, and $R^2 = M(-1, 0) = -I$ matches $i^2 = -1$.