Projection matrix in 3D: dropping space onto a plane or a line

The idea: drop every point at a right angle

Pick a plane or a line through the origin: the target. The orthogonal projection onto it sends each point v to the point of the target straight "under" it: drop a perpendicular from v to the target, and its foot is Pv. With the sun straight overhead, Pv is the shadow of v on the floor. The map is linear, so it is a 3×3 matrix P.

On the canvas the target is purple, its normal n pink, v the black arrow, the drop the grey dashes and Pv the purple arrow. The unit cube (blue) with an orange F on its face x = 1, and î, ĵ, k̂ (red, green, blue), show what P does to all of space: press Project and every point slides along its own drop until the cube is flat. The grey ghost keeps the cube where it started. (Demo 1: shadow on the floor: drop z)

Two formulas: P = uuᵀ and P = I − nnᵀ

Onto a line with unit direction u, the shadow is the part of v along u: Pv = (v·u)u = u(uᵀv) = (uuᵀ)v. So P = uuᵀ.

Onto a plane with unit normal n, take that part away instead: Pv = v − (v·n)n. So P = I − nnᵀ. If n = m/|m| for a whole-number vector m, then nnᵀ = mmᵀ/(m·m), and every entry of P is a fraction.

target	n or u	P
the xy-plane z = 0	n = (0, 0, 1)	[1 0 0; 0 1 0; 0 0 0]: drop z
the plane x + y + z = 0	n = (1, 1, 1)/√3	⅓[2 −1 −1; −1 2 −1; −1 −1 2]
the plane x = y	n = (1, −1, 0)/√2	[½ ½ 0; ½ ½ 0; 0 0 1]
the plane z = x + y	n = (−1, −1, 1)/√3	⅓[2 −1 1; −1 2 1; 1 1 2]
the z-axis	u = (0, 0, 1)	[0 0 0; 0 0 0; 0 0 1]: keep only z
the line through (1, 1, 1)	u = (1, 1, 1)/√3	⅓[1 1 1; 1 1 1; 1 1 1]

As for any matrix, the columns are where î, ĵ and k̂ land. For a plane all three columns lie in the plane; for a line they are all multiples of u. Pv is worked out row by row: the x of Pv is row 1 of P dotted with v, and so on. For x + y + z = 0 and v = (2, 1, 3): row 1 · v = (⅔, −⅓, −⅓)·(2, 1, 3) = 0, and Pv = (0, −1, 1). (Demo 2: drop at a right angle onto a tilted plane, Demo 3: onto a line: rank 1)

The error is perpendicular, so Pv is the closest point

The error e = v − Pv is the dashed drop. For a plane, e = (v·n)n points along n, at a right angle to every vector in the plane. For a line, e·u = 0. In the example above, e = (2, 2, 2) = 2·(1, 1, 1).

That right angle makes Pv the closest point of the target to v. For any other point w of the target, Pythagoras gives |v − w|² = |v − Pv|² + |Pv − w|², which is never smaller than |v − Pv|². This is the idea behind least squares.

P² = P and Pᵀ = P

A point that is already on the target is its own shadow, so projecting twice changes nothing: P² = P (P is idempotent). Onto the plane x = y, v = (2, 0, 1) goes to Pv = (1, 1, 1), and P(1, 1, 1) = (1, 1, 1) again. (Demo 4: project twice: nothing changes)

P is also symmetric: (uuᵀ)ᵀ = uuᵀ, so Pᵀ = P. Together the two properties are exactly what makes a projection orthogonal. A matrix with P² = P that is not symmetric also projects, but along a slanted direction (an oblique projection).

det = 0, rank and eigenvalues

P flattens the cube: its volume goes to 0, so det P = 0 and there is no inverse. Every point on the same drop line has the same shadow, so you cannot get v back from Pv.

target	rank	eigenvalues	λ = 1 on	λ = 0 on (null space)
plane, normal n	2	1, 1, 0	the plane	the line along n
line, direction u	1	1, 0, 0	the line	the plane ⟂ u

The trace (the sum of the diagonal) equals the rank: 2 for a plane, 1 for a line.

Plane + normal = I

I − P keeps exactly what P throws away. For a plane, I − P = nnᵀ is the projection onto the normal line, so P + nnᵀ = I. Every vector splits into two pieces at right angles, one in the plane and one along n: v = Pv + (I − P)v. (Demo 5: plane + normal = I)

For a line it is the other way round: I − uuᵀ projects onto the plane ⟂ u. A reflection in the plane is I − 2nnᵀ = 2P − I: go to the shadow, then the same distance again (see Reflection Matrix in 3D).

The least-squares formula A(AᵀA)⁻¹Aᵀ

Often a plane is given by two vectors that span it, the columns a1 and a2 of a 3×2 matrix A, and not by its normal. The shadow is on the plane, so Pv = A·x̂ for some x̂ = (x̂1, x̂2). The error must be perpendicular to both columns: Aᵀ(v − A·x̂) = 0. That gives the normal equations AᵀA·x̂ = Aᵀv, so x̂ = (AᵀA)⁻¹Aᵀv and

P = A(AᵀA)⁻¹Aᵀ.

With a1 = (1, 0, 1) and a2 = (0, 1, 1): AᵀA = [2 1; 1 2], (AᵀA)⁻¹ = ⅓[2 −1; −1 2], and P = ⅓[2 −1 1; −1 2 1; 1 1 2]. That is the same matrix as I − nnᵀ with n along a1 × a2 = (−1, −1, 1): one plane, one projection. For v = (1, 2, 0), Aᵀv = (1, 2), x̂ = (0, 1) and Pv = 0·a1 + 1·a2 = (0, 1, 1). (Demo 6: the least-squares formula A(AᵀA)⁻¹Aᵀ)

In least squares, v is the data that A·x = v cannot fit exactly, and x̂ is the best fit (see Least Squares). If the columns are orthonormal, AᵀA = I and P = AAᵀ. That is the reason to orthonormalize them first (Gram–Schmidt).

See also Projection Matrix (the same idea in 2D), Linear Transformations, Vectors and the Dot Product, Plane Equation, Determinant, Eigenvectors, Inverse Matrix, Column Space and Null Space, Rotation Matrix in 3D, Shear Matrix in 3D, Perspective Projection (the camera kind of projection) and SVD and PCA (projection onto the top principal directions).

What the page leaves out

Oblique projections (P² = P but not symmetric), projection onto a plane that does not pass through the origin (an affine map, not a matrix), and perspective projection, where rays meet at the eye instead of running parallel. Projection onto subspaces of higher dimension works the same way with a taller A. In practice x̂ is computed with a QR factorization instead of forming AᵀA, which loses accuracy when the columns are nearly parallel.

References

MIT 18.06 (Gilbert Strang), Lecture 15: Projections onto subspaces

Interactive Linear Algebra (Margalit, Rabinoff), 6.3 Orthogonal Projection

Wikipedia: Projection (linear algebra)