Singular Values Measure Stretch

Eigenvectors ask which input directions leave as parallel output directions. That is a powerful question, but it is not always the most natural one. A matrix can rotate every direction and still have a perfectly understandable geometry.

The singular value decomposition asks a different question:

> Which input directions are stretched the most, and which are stretched the least?

That question always has real answers. In two dimensions, the picture is almost embarrassingly concrete. Take the unit circle, apply the matrix to every point on it, and watch what shape appears. The result is an ellipse.

The long and short radii of that ellipse are the singular values. The directions on the original circle that become those radii are the right singular vectors. The directions of the radii after transformation are the left singular vectors.

Loading runtime...

0.00s

live slideshowessay-2026-singular-values-measure-stretch-1.mcs

param tilt = 0.0
param stretch = 1.0

background = BLACK
camera = Camera(4b)

let M = |x, y, tilt, stretch| [stretch * x + tilt * y, 0.65 * y, 0]

let UpperEllipse = |stretch| block {
    . stroke{BLUE, 3} ExplicitFunc(|x| 0.65 * sqrt(1 - x * x / (stretch * stretch)), [-stretch, stretch, 120])
}

let LowerEllipse = |stretch| block {
    . stroke{BLUE, 3} ExplicitFunc(|x| -0.65 * sqrt(1 - x * x / (stretch * stretch)), [-stretch, stretch, 120])
}

let Diagram = |tilt, stretch| block {
    . stroke{GRAY, 1} center{[-1.55, 0, 0]} Circle(0.75)
    . color{GRAY} center{[-1.55, -1.05, 0]} Text("unit circle", 0.74)
    . UpperEllipse(stretch)
    . LowerEllipse(stretch)
    . stroke{GREEN, 3} Line(start: [0, 0, 0], end: M(1, 0, tilt, stretch))
    . stroke{ORANGE, 3} Line(start: [0, 0, 0], end: M(0, 1, tilt, stretch))
    . color{WHITE} center{[0, 1.65, 0]} Text("a matrix turns circles into ellipses", 0.88)
}

mesh diagram = Diagram($tilt, $stretch)

slide "increase stretch"
    stretch = 1.95
    play Lerp(1.3)

slide "add shear"
    tilt = 1.05
    play Lerp(1.3)

The reason this works is that a linear map sends combinations to combinations. A circle contains every unit direction. When those directions are mapped, the amount of stretching varies smoothly. Somewhere it is largest. Somewhere perpendicular to that, it is smallest. Those two extremal directions become the axes of the ellipse.

The decomposition

The singular value decomposition says every matrix can be written as

A=U\Sigma V^T.

Read from right to left, this has a beautiful geometric story.

First, $V^T$ rotates the input so the special input directions line up with the coordinate axes. Then $\Sigma$ stretches those axes by the singular values. Finally, $U$ rotates the result into its final orientation.

So every linear transformation is:

\text{rotate, stretch along axes, rotate again.}

This is stronger than diagonalization in one important way. Diagonalization needs eigenvectors and can fail over the real plane. The singular value decomposition always exists.

Why the transpose appears

There is a quiet trick behind the computation. The matrix $A^TA$ measures input-side stretch:

\lVert Av\rVert^2=(Av)\cdot(Av)=v^TA^TAv.

So the directions that maximize and minimize output length are eigenvectors of $A^TA$. Since $A^TA$ is symmetric, its eigenvectors are perpendicular and its eigenvalues are real and nonnegative.

The singular values are square roots of those eigenvalues.

\sigma_i=\sqrt{\lambda_i(A^TA)}.

That formula is less important than the meaning. $A^TA$ asks, "if I transform a vector and measure its squared length, which input directions make that number simple?"

Not just a factorization

Singular values tell you the numerical personality of a matrix.

If the largest singular value is huge, the matrix can magnify errors. If the smallest singular value is close to zero, the matrix nearly collapses some direction. If a singular value is exactly zero, information is lost completely along its corresponding input direction.

This is why SVD appears in data compression. An image, a sound, or a table of measurements can be viewed as a matrix. Large singular values capture large-scale structure. Small singular values often capture fine detail or noise. Keeping only the largest ones gives the closest low-rank approximation in a precise least-squares sense.

The SVD is therefore a way to ask a matrix what it does most strongly. It does not demand that directions stay parallel to themselves. It only asks how much each direction is stretched. For geometry, computation, and data, that is often the question you actually wanted.