Eigenvectors: the directions a transformation leaves alone
Most arrows are knocked off their line by a matrix. The few that are only stretched are its eigenvectors, and they turn matrix powers into powers of numbers.
AssumesA matrix is a transformation of spaceThe determinant is how much area gets scaled
Apply the matrix
to an arrow, and the arrow usually changes direction. The arrow $(0, 1)$ goes to $(1, 2)$, which points somewhere else entirely. The arrow $(1, 1)$ goes to $(4, 2)$, again off its original line. You would expect this of every arrow: a matrix stretches and slants the plane, so why would any direction survive? Yet in the animation the arrow $(-1, 1)$ goes to $(-2, 2)$, exactly twice itself, and $(1, 0)$ goes to $(3, 0)$, three times itself. These arrows are stretched and nothing else happens to them.
Arrows like that are rare, and they are the most useful thing to know about a matrix. Along such a direction the matrix acts like a plain number, and questions that are hard for matrices, such as “what does applying $A$ fifty times do?”, become questions about powers of numbers. We will find out how to locate these directions, when they exist, and how to use them.
Here is what I use from earlier. A $2 \times 2$ matrix is a transformation of the plane: $\begin{pmatrix} a & b \\ c & d \end{pmatrix}$ sends $(x, y)$ to $(ax + by,\ cx + dy)$, and the map is linear, so $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$ and $A(k\mathbf{v}) = k\,A\mathbf{v}$. The determinant $ad - bc$ is the factor by which the matrix scales areas, and it is zero exactly when the two columns are parallel and the plane is flattened onto a line or a point. $I$ is the identity matrix, which leaves every vector alone.
Hunting for the special directions
Let us look for the special arrows of $A$ systematically. Take an arrow $\mathbf{v}$ of fixed length, swing it around the origin, and at each position draw $A\mathbf{v}$ next to it. The quantity to watch is the angle between $\mathbf{v}$ and $A\mathbf{v}$. When it is zero, $A\mathbf{v}$ lies along $\mathbf{v}$.
Pause and try
Before watching: compute $A\mathbf{v}$ for $\mathbf{v} = (1, 0)$, $(2, 1)$, $(1, -1)$ and $(-3, 3)$. For which of these is $A\mathbf{v}$ a multiple of $\mathbf{v}$, and what is the multiple?
Show answerHide answer
$A(1, 0) = (3, 0) = 3\,(1, 0)$. $A(2, 1) = (7, 2)$, which is not a multiple of $(2, 1)$ since $7/2 \neq 2/1$. $A(1, -1) = (3 - 1, -2) = (2, -2) = 2\,(1, -1)$. $A(-3, 3) = (-9 + 3, 6) = (-6, 6) = 2\,(-3, 3)$. So three of the four are only stretched, and the last two, which lie on the same line through the origin, are stretched by the same factor 2.
In half a turn the angle reaches zero exactly twice: on the line through $(-1, 1)$, where $A$ doubles every arrow, and on the $x$ axis, where $A$ triples every arrow. (The other half turn passes through the same two lines.) Everywhere else $A\mathbf{v}$ leaves the line of $\mathbf{v}$.
The clause $\mathbf{v} \neq \mathbf{0}$ is there because $A\mathbf{0} = \lambda\mathbf{0}$ for every number $\lambda$, so the zero vector would tell us nothing. The eigenvalue, on the other hand, may be any number. If $\lambda$ is negative the arrow is flipped to the other side of the origin, still on its line. If $\lambda = 0$ the arrow is sent to $\mathbf{0}$.
Eigenvectors come in whole lines. If $A\mathbf{v} = \lambda\mathbf{v}$ and $k \neq 0$, then $A(k\mathbf{v}) = k\,A\mathbf{v} = k\lambda\mathbf{v} = \lambda(k\mathbf{v})$, so $k\mathbf{v}$ is an eigenvector for the same $\lambda$. That is why $(1, -1)$ and $(-3, 3)$ shared the factor 2 above. It is the line that matters, and I will call it an eigenline. (“Eigen” is German for “own”: these are the matrix’s own directions.) For our matrix we have found the eigenvalue 3 with eigenline the $x$ axis, and the eigenvalue 2 with eigenline along $(-1, 1)$.
Finding eigenvalues without searching
Swinging an arrow around is not a method. For a matrix whose eigenlines are at awkward angles we would never hit them exactly, and we could not be sure we had found them all. We need an equation.
The trouble with $A\mathbf{v} = \lambda\mathbf{v}$ is that it has two unknowns of different kinds, the number $\lambda$ and the vector $\mathbf{v}$. Move everything to one side. Since $\lambda\mathbf{v} = \lambda I\mathbf{v}$, the equation says
Read this as a statement about the matrix $A - \lambda I$, which is $A$ with $\lambda$ subtracted from both diagonal entries. It says that this matrix sends some nonzero vector to the origin. Most matrices do not do that. A matrix that sends a nonzero vector to $\mathbf{0}$ has crushed a whole line onto a point, so it flattens the plane, and flattening is what a zero determinant detects. So we can look for $\lambda$ first, with no $\mathbf{v}$ in sight: find the values of $\lambda$ for which $A - \lambda I$ flattens the plane.
For our matrix, $A - \lambda I = \begin{pmatrix} 3 - \lambda & 1 \\ 0 & 2 - \lambda \end{pmatrix}$ has determinant $(3 - \lambda)(2 - \lambda) - 1 \cdot 0$, which is zero exactly for $\lambda = 3$ and $\lambda = 2$. The search found both eigenvalues, and now we know there are no others. Here is the general statement, with the step “flattens the plane” made exact.
The quadratic $\lambda^2 - (a + d)\lambda + (ad - bc)$ is called the characteristic polynomial of $A$. Its coefficients are familiar: $ad - bc$ is the determinant, and $a + d$, the sum of the diagonal entries, is called the trace. Because a quadratic with roots $\lambda_1, \lambda_2$ factors as $(\lambda - \lambda_1)(\lambda - \lambda_2)$, the eigenvalues add up to the trace and multiply to the determinant. For our $A$: $3 + 2 = 5$ and $3 \cdot 2 = 6$. The second fact makes sense as geometry, since stretching by 3 along one line and by 2 along another multiplies areas by 6.
Once an eigenvalue is known, its eigenvectors are the nonzero solutions of $(A - \lambda I)\mathbf{v} = \mathbf{0}$, a pair of linear equations of which one is guaranteed to be a multiple of the other.
Three kinds of matrix
A quadratic equation with real coefficients has two different real roots, one repeated root, or no real roots. All three happen for matrices, and each has a clear geometric meaning. The quantity that decides is the discriminant of the characteristic polynomial, $(a + d)^2 - 4(ad - bc)$.
Two different real eigenvalues. This is the case of $A$ and $B$ above: two eigenlines with different stretch factors.
A repeated eigenvalue. Take the shear $S = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$, which slides each point sideways by its height.
Pause and try
Picture the shear $S$, which sends $(x, y)$ to $(x + y, y)$. Which arrows stay on their own line? Then confirm with the characteristic equation.
Show answerHide answer
An arrow on the $x$ axis has height 0 and does not move at all, so it is an eigenvector with eigenvalue 1. Any arrow with $y \neq 0$ keeps its height while its tip slides sideways by $y$, so its direction changes. Equation: the trace is 2 and the determinant is 1, so $\lambda^2 - 2\lambda + 1 = (\lambda - 1)^2 = 0$ and the only eigenvalue is 1. Then $S - I = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$, and $(S - I)(x, y) = (y, 0) = \mathbf{0}$ forces $y = 0$: only the $x$ axis.
So a repeated eigenvalue can come with only one eigenline. It does not have to: the matrix $2I = \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix}$ has characteristic polynomial $(\lambda - 2)^2$ as well, and it doubles every vector, so every nonzero vector is an eigenvector. A repeated root tells you to look at $A - \lambda I$ directly.
No real eigenvalues. Take the quarter turn $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$, which sends $(x, y)$ to $(-y, x)$. A rotation by an angle other than $0^\circ$ or $180^\circ$ turns every arrow off its line, so it should have no eigenvectors, and the algebra agrees: the trace is 0 and the determinant is 1, so the characteristic equation is $\lambda^2 + 1 = 0$, which no real number satisfies.
The equation $\lambda^2 + 1 = 0$ does have solutions among the complex numbers, $\lambda = \pm i$, and in a fuller treatment these complex eigenvalues encode the rotation. In the real plane there is no arrow that a quarter turn leaves on its line, and that is the statement we need here.
Eigenvectors for different eigenvalues point in different directions
In the first case we found two eigenlines, and they were different lines. That was no accident, and the consequence is what makes eigenvectors usable.
Two vectors of which neither is a multiple of the other are called linearly independent, and the theorem says they form a basis of the plane: a pair of arrows out of which everything else can be mixed, just like $\mathbf{i}$ and $\mathbf{j}$. A basis made of eigenvectors is an eigenbasis. The shear shows that the hypothesis $\lambda_1 \neq \lambda_2$ cannot be dropped, since all its eigenvectors lie on one line and cannot reach a point off the $x$ axis.
Why eigenvectors make powers easy
Here is the payoff. Suppose $\mathbf{x} = \alpha\mathbf{v}_1 + \beta\mathbf{v}_2$ with $\mathbf{v}_1, \mathbf{v}_2$ eigenvectors. Applying $A$ once, by linearity,
The result has the same form with new coefficients, so applying $A$ again multiplies them by $\lambda_1$ and $\lambda_2$ once more.
Now a concrete model. A region has 900 thousand people, some in the city and some in the suburbs. Each year 10% of the city’s residents move to the suburbs and 20% of the suburbs’ residents move to the city. If $c$ and $s$ are this year’s populations in thousands, next year’s are $0.9c + 0.2s$ and $0.1c + 0.8s$:
Start with 300 thousand in the city and 600 thousand in the suburbs. Where do things stand after 15 years? Done directly, that is fifteen matrix multiplications.
The formula says more than the answer for year 15. Because $(0.7)^n$ shrinks toward 0, the second piece dies away and the population approaches $300\,(2, 1) = (600, 300)$: twice as many people in the city as in the suburbs. And that limit depends on the start only through the total.
Pause and try
Suppose instead that all 900 thousand people start in the suburbs. Without multiplying any matrices, find the populations after $n$ years and the long-run split.
Show answerHide answer
Write $(0, 900) = \alpha\,(2, 1) + \beta\,(1, -1)$: $2\alpha + \beta = 0$ and $\alpha - \beta = 900$, so $3\alpha = 900$, $\alpha = 300$, $\beta = -600$. Then $M^n(0, 900) = (600 - 600\,(0.7)^n,\ 300 + 600\,(0.7)^n)$. The long-run split is $(600, 300)$ again. The reason is that $\mathbf{v}_2 = (1, -1)$ has coordinates adding to 0, so the total of 900 has to be carried entirely by $\alpha\,(2, 1)$, which forces $\alpha = 300$ for every start with that total.
The same reasoning explains the long-run behavior of any matrix with an eigenbasis. In $\alpha\lambda_1^n\mathbf{v}_1 + \beta\lambda_2^n\mathbf{v}_2$, the term whose eigenvalue is larger in absolute value outgrows the other, so after many steps the result points almost along that eigenvector (provided its coefficient is not zero). For our first matrix $A$, with eigenvalues 3 and 2, the ratio of the two terms shrinks like $(2/3)^n$, and repeated application lines every arrow up with the $x$ axis, unless it started on the other eigenline. The Fibonacci numbers obey the same law: each step is one multiplication by a fixed matrix, and the ratio of consecutive terms settles at the larger eigenvalue of that matrix, the golden ratio. One of the problems below works a similar sequence with whole-number eigenvalues.
Two mistakes to avoid
What was hard here was that the defining equation $A\mathbf{v} = \lambda\mathbf{v}$ hides two unknowns in one line. The step that untangles them is to ask a question about $\lambda$ alone: for which numbers does $A - \lambda I$ flatten the plane? The determinant answers that, and the eigenvectors then come from a small linear system. The reusable idea is the change of viewpoint that follows. Described in the basis of its eigenvectors, a matrix only multiplies each coordinate by a number, and anything done repeatedly with the matrix becomes arithmetic with powers.
Check yourself
The matrix $C$ has characteristic equation $\lambda^2 - 4\lambda + 4 = 0$. What can you say about its eigenvectors?
Problems
The first three are computations, the next three use the eigenbasis, and the last two are proofs.
Problem 1
Let $D = \begin{pmatrix} 1 & 2 \\ 2 & -2 \end{pmatrix}$. Without solving any equation, decide which of $(2, 1)$, $(1, 1)$ and $(1, -2)$ are eigenvectors of $D$, and give the eigenvalues. Check your eigenvalues against the trace and determinant.
Show solutionHide solution
$D(2, 1) = (2 + 2, 4 - 2) = (4, 2) = 2\,(2, 1)$: an eigenvector with eigenvalue 2. $D(1, 1) = (3, 0)$, which is not a multiple of $(1, 1)$. $D(1, -2) = (1 - 4, 2 + 4) = (-3, 6) = -3\,(1, -2)$: an eigenvector with eigenvalue $-3$. Check: $2 + (-3) = -1$ is the trace $1 + (-2)$, and $2 \cdot (-3) = -6$ is the determinant $-2 - 4$.
Problem 2
Find the eigenvalues and eigenvectors of $\begin{pmatrix} 5 & -2 \\ 4 & -1 \end{pmatrix}$, and verify them.
Show solutionHide solution
Trace $4$, determinant $-5 + 8 = 3$: $\lambda^2 - 4\lambda + 3 = (\lambda - 1)(\lambda - 3) = 0$, so $\lambda = 1$ or $3$. For $\lambda = 1$: the matrix minus $I$ is $\begin{pmatrix} 4 & -2 \\ 4 & -2 \end{pmatrix}$, giving $4x - 2y = 0$, so $y = 2x$: multiples of $(1, 2)$. For $\lambda = 3$: $\begin{pmatrix} 2 & -2 \\ 4 & -4 \end{pmatrix}$ gives $x = y$: multiples of $(1, 1)$. Verify: the matrix sends $(1, 2)$ to $(5 - 4, 4 - 2) = (1, 2)$ and $(1, 1)$ to $(3, 3)$.
Problem 3
For each matrix, say how many real eigenvalues it has and how many eigenlines: (a) $\begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix}$, (b) $\begin{pmatrix} 1 & -2 \\ 1 & 3 \end{pmatrix}$, (c) $\begin{pmatrix} 1 & 4 \\ 1 & 1 \end{pmatrix}$.
Show solutionHide solution
(a) Trace 4, determinant 4: $(\lambda - 2)^2 = 0$, one eigenvalue, 2. The matrix minus $2I$ is $\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$, which is not zero and sends $(x, y)$ to $(y, 0)$, so the eigenvectors have $y = 0$: one eigenline, the $x$ axis. (b) Trace 4, determinant $3 + 2 = 5$: $\lambda^2 - 4\lambda + 5 = 0$ has discriminant $16 - 20 = -4 < 0$. No real eigenvalues and no eigenlines. (c) Trace 2, determinant $1 - 4 = -3$: $\lambda^2 - 2\lambda - 3 = (\lambda - 3)(\lambda + 1) = 0$. Two eigenvalues, $3$ and $-1$, and two eigenlines: for $3$, $-2x + 4y = 0$ gives multiples of $(2, 1)$; for $-1$, $2x + 4y = 0$ gives multiples of $(2, -1)$.
Problem 4
The matrix $F = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ reflects the plane across the line $y = x$. Predict its eigenvalues and eigenlines from the geometry, then confirm with the characteristic equation.
Show solutionHide solution
A reflection leaves every point of its mirror line where it is, so the line $y = x$ is an eigenline with eigenvalue 1. An arrow perpendicular to the mirror is sent to its opposite, so the line along $(1, -1)$ is an eigenline with eigenvalue $-1$. Every other arrow lands on the other side of the mirror, off its line. Algebra: trace 0, determinant $-1$, so $\lambda^2 - 1 = 0$ and $\lambda = \pm 1$. Check: $F(1, 1) = (1, 1)$ and $F(1, -1) = (-1, 1) = -(1, -1)$.
Problem 5
For $A = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix}$ and $\mathbf{x} = (1, 1)$, compute $A^5\mathbf{x}$ using the eigenvectors $(1, 0)$ and $(-1, 1)$. Then check by multiplying five times.
Show solutionHide solution
Write $(1, 1) = \alpha\,(1, 0) + \beta\,(-1, 1)$: the second coordinate gives $\beta = 1$, and then $\alpha - 1 = 1$ gives $\alpha = 2$. So $A^5\mathbf{x} = 2 \cdot 3^5\,(1, 0) + 1 \cdot 2^5\,(-1, 1) = (486 - 32,\ 32) = (454, 32)$. Directly, each step sends $(x, y)$ to $(3x + y, 2y)$: $(1, 1) \to (4, 2) \to (14, 4) \to (46, 8) \to (146, 16) \to (454, 32)$.
Problem 6
A sequence starts $x_0 = 1$, $x_1 = 1$ and continues by $x_{n+1} = x_n + 2x_{n-1}$, so it runs $1, 1, 3, 5, 11, 21, \dots$ (a) Check that the pair $(x_{n+1}, x_n)$ is obtained from $(x_n, x_{n-1})$ by the matrix $G = \begin{pmatrix} 1 & 2 \\ 1 & 0 \end{pmatrix}$. (b) Find the eigenvalues and eigenvectors of $G$. (c) Find a formula for $x_n$, and say what $x_{n+1}/x_n$ approaches.
Show solutionHide solution
(a) $G(x_n, x_{n-1}) = (x_n + 2x_{n-1},\ x_n) = (x_{n+1}, x_n)$. (b) Trace 1, determinant $-2$: $\lambda^2 - \lambda - 2 = (\lambda - 2)(\lambda + 1) = 0$. For $\lambda = 2$: $G - 2I = \begin{pmatrix} -1 & 2 \\ 1 & -2 \end{pmatrix}$ gives $x = 2y$, eigenvector $(2, 1)$. For $\lambda = -1$: $G + I = \begin{pmatrix} 2 & 2 \\ 1 & 1 \end{pmatrix}$ gives $x = -y$, eigenvector $(1, -1)$. (c) The pair $(x_{n+1}, x_n)$ equals $G^n(x_1, x_0) = G^n(1, 1)$. Write $(1, 1) = \alpha\,(2, 1) + \beta\,(1, -1)$: $2\alpha + \beta = 1$ and $\alpha - \beta = 1$ give $\alpha = \frac23$, $\beta = -\frac13$. So
and reading the second coordinate, $x_n = \dfrac{2^{n+1} + (-1)^n}{3}$. Check: $x_5 = \frac{64 - 1}{3} = 21$. The term with $2^n$ dominates, so $x_{n+1}/x_n$ approaches the larger eigenvalue, 2. (For example $21/11 \approx 1.91$.)
Problem 7
Let $\mathbf{v}$ be an eigenvector of $A$ with eigenvalue $\lambda$. Prove: (a) $\mathbf{v}$ is an eigenvector of $A^2$ with eigenvalue $\lambda^2$; (b) if $A$ is invertible, then $\lambda \neq 0$ and $\mathbf{v}$ is an eigenvector of $A^{-1}$ with eigenvalue $1/\lambda$.
Show solutionHide solution
(a) $A^2\mathbf{v} = A(A\mathbf{v}) = A(\lambda\mathbf{v}) = \lambda\,A\mathbf{v} = \lambda^2\mathbf{v}$, and $\mathbf{v} \neq \mathbf{0}$. (b) If $\lambda$ were 0 we would have $A\mathbf{v} = \mathbf{0}$, and applying $A^{-1}$ would give $\mathbf{v} = A^{-1}\mathbf{0} = \mathbf{0}$, which is impossible for an eigenvector. So $\lambda \neq 0$. Now apply $A^{-1}$ to both sides of $A\mathbf{v} = \lambda\mathbf{v}$: $\mathbf{v} = \lambda\,A^{-1}\mathbf{v}$, and dividing by $\lambda$ gives $A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}$. Undoing a stretch by $\lambda$ along a line is a stretch by $1/\lambda$ along the same line.
Problem 8
A matrix is symmetric if it has the form $\begin{pmatrix} a & b \\ b & d \end{pmatrix}$. (a) Prove that a symmetric $2 \times 2$ matrix always has real eigenvalues. (b) The dot product of $(v_1, v_2)$ and $(w_1, w_2)$ is $v_1w_1 + v_2w_2$, and two nonzero vectors are perpendicular exactly when it is 0. Prove that for a symmetric matrix $A$, $(A\mathbf{v}) \cdot \mathbf{w} = \mathbf{v} \cdot (A\mathbf{w})$ for all $\mathbf{v}, \mathbf{w}$, and deduce that eigenvectors for two different eigenvalues are perpendicular.
Show solutionHide solution
(a) The discriminant of $\lambda^2 - (a + d)\lambda + (ad - b^2)$ is $(a + d)^2 - 4ad + 4b^2 = (a - d)^2 + 4b^2$, a sum of squares, so it is never negative and the roots are real. (b) $(A\mathbf{v}) \cdot \mathbf{w} = (av_1 + bv_2)w_1 + (bv_1 + dv_2)w_2 = a\,v_1w_1 + b\,(v_2w_1 + v_1w_2) + d\,v_2w_2$. This expression is unchanged when $\mathbf{v}$ and $\mathbf{w}$ are exchanged, so it also equals $(A\mathbf{w}) \cdot \mathbf{v} = \mathbf{v} \cdot (A\mathbf{w})$. Now let $A\mathbf{v}_1 = \lambda_1\mathbf{v}_1$ and $A\mathbf{v}_2 = \lambda_2\mathbf{v}_2$ with $\lambda_1 \neq \lambda_2$. Then $\lambda_1\,(\mathbf{v}_1 \cdot \mathbf{v}_2) = (A\mathbf{v}_1) \cdot \mathbf{v}_2 = \mathbf{v}_1 \cdot (A\mathbf{v}_2) = \lambda_2\,(\mathbf{v}_1 \cdot \mathbf{v}_2)$, so $(\lambda_1 - \lambda_2)(\mathbf{v}_1 \cdot \mathbf{v}_2) = 0$ and $\mathbf{v}_1 \cdot \mathbf{v}_2 = 0$. Problem 4 is an instance: $F$ is symmetric, and its eigenvectors $(1, 1)$ and $(1, -1)$ are perpendicular.