Mathematics · Upper Division

Linear Algebra for Physics

Matrices, eigenvalues, and inner product spaces are the language of quantum mechanics, classical mechanics (normal modes), and every numerical simulation. This chapter develops the tools physicists use most.

PrerequisitesVectors (Ch. 24) · Calculus (Ch. 25) · Basic matrix multiplication
Learning Goals
  • Identify vector spaces and inner product spaces, and apply the Cauchy-Schwarz inequality.
  • Find eigenvalues and eigenvectors of a matrix by solving the characteristic equation.
  • State the spectral theorem for Hermitian operators and explain its role in quantum mechanics.
  • Compute the singular value decomposition and use it for low-rank approximation and least-squares fitting.
  • Determine whether two Hermitian operators share an eigenbasis by checking if they commute.

LA.1 Vector Spaces and Inner Products

Definition LA.1 — Vector Space
A vector space V over a field F (usually ℝ or ℂ) is a set with addition and scalar multiplication satisfying 8 axioms (commutativity, associativity, distributivity, identity, inverses). Key examples in physics:ℝⁿ:column vectors, forces, positions.  ℂⁿ: quantum states (spinors, finite-dimensional Hilbert spaces).L2(a,b)L^{2}(a,b): square-integrable functions — the Hilbert space of quantum wavefunctions.Polynomials, matrices, operators — all are vector spaces under appropriate operations.

An inner product ⟨u, v⟩ on a complex vector space satisfies: linearity in v, conjugate-linearity in u, ⟨v,v⟩ ≥ 0 with equality iff v = 0. For ℂⁿ: ⟨u, v⟩ = u†v = Σᵢ uᵢ* vᵢ. For L²: ⟨f, g⟩ = ∫ f*(x) g(x) dx.

∣⟨u,v⟩∣≤∥u∥∥v∥(Cauchy−Schwarzinequality)|⟨u, v⟩| \le \|u\| \|v\| \qquad (Cauchy-Schwarz inequality)(LA.1)

LA.2 Eigenvalues and Eigenvectors

For a linear operator A: V → V, a nonzero vector v with Av = λv is an eigenvector with eigenvalue λ. Eigenvalues satisfy:

det⁡(A−λI)=0(characteristicequation)\det(A - \lambda I) = 0 \qquad (characteristic equation)(LA.2)

For an n×n matrix, this is an nth-degree polynomial — giving n eigenvalues (counting multiplicity). The eigenvalues determine: stability of linear systems (Re(λ) < 0 → stable), principal stresses in mechanics, normal mode frequencies, and the spectrum of quantum observables.

Example LA.1 — Normal Modes of Two Coupled Oscillators

Two masses m connected by springs (k to walls, κ between them). Find normal frequencies.

Equations of motion:mx¨1=−kx1−κ(x1−x2)mx¨2=−kx2−κ(x2−x1)m\ddot{x}_{1} = -kx_{1} - \kappa(x_{1}-x_{2}) \qquad m\ddot{x}_{2} = -kx_{2} - \kappa(x_{2}-x_{1})
Matrix form:ẍ = −(1/m)Kx-(1/m)Kx    K=(k+κ−κ−κk+κ)K = \begin{pmatrix} k+\kappa & -\kappa \\ -\kappa & k+\kappa \end{pmatrix}
Eigenvalues of K:det⁡(K−μI)=0→(k+κ−μ)2−κ2=0→μ=k\det(K - \mu I) = 0 \to (k+\kappa-\mu)^{2} - \kappa^{2} = 0 \to \mu = k or k+2κk+2\kappa
Normal frequencies:ω1=k/m\omega_{1} = \sqrt{k/m} (in-phase mode), ω2=(k+2κ)/m\omega_{2} = \sqrt{(k+2\kappa)/m} (out-of-phase mode)
Eigenvectors:v1=(1,1)/2v_{1} = (1,1)/\sqrt 2 (both masses move together), v2=(1,−1)/2v_{2} = (1,-1)/\sqrt 2 (masses move oppositely)

LA.3 Hermitian Operators and Spectral Theorem

Definition LA.2 — Hermitian (Self-Adjoint) Operators
An operator A is Hermitian (self-adjoint) if A=A†(AA = A^\dagger (A equals its conjugate transpose). Key properties:1. All eigenvalues are real (proved by ⟨v∣A∣v⟩=⟨Av∣v⟩→λ=λ∗v|A|v⟩ = ⟨Av|v⟩ \to \lambda = \lambda*)2. Eigenvectors for different eigenvalues are orthogonal3. (Spectral theorem) A has a complete orthonormal eigenbasis: A=∑iλi∣vi⟩⟨vi∣A = \sum_{i} \lambda_{i} |v_{i}⟩⟨v_{i}|In quantum mechanics, observables are Hermitian operators — this guarantees real measured values and allows complete state expansion in the eigenstate basis.

Analogously, unitary operators U satisfy U†U = I (preserve inner products). All eigenvalues lie on the unit circle (|λ| = 1). Time evolution e−iHt/ℏe^{-iHt/\hbar} is unitary when H is Hermitian — unitarity is quantum mechanical probability conservation.

LA.4 Singular Value Decomposition

Every m×n matrix A can be written as:

A=U∑V†(SVD)A = U\sum V^\dagger \qquad (SVD)(LA.3)

where U (m×m) and V (n×n) are unitary and Σ is diagonal with non-negative real entries σ₁ ≥ σ₂ ≥ ... ≥ 0 (singular values). The number of nonzero σᵢ is the rank of A. SVD is the foundation of:

Principal Component Analysis (PCA): the columns of V are principal axes — directions of maximum variance in data.

Low-rank approximation: A≈UkΣkVk†A \approx U_k \Sigma_k V_k^\dagger (keep only k largest singular values) — used in image compression, recommendation systems, quantum state compression.

Least squares: the pseudoinverse A⁺ = VΣ⁺U† solves the least-squares problem min‖Ax − b‖ when A is non-square.

Theorem LA.1 — Spectral Theorem for Commuting Observables
Two Hermitian operators A and B can be simultaneously diagonalized (share a complete orthonormal eigenbasis) if and only if they commute: [A,B]=AB−BA=0A, B] = AB - BA = 0. Physical consequence: two quantum observables can be simultaneously measured (sharp values) only if they commute. Position and momentum don't commute ([x^,p^]=iℏ)—\hat{x}, \hat{p}] = i\hbar) — they cannot both be sharp. L2L^{2} and LzL_z commute — both can be simultaneously specified.
Example LA.2 — Diagonalizing the Inertia Tensor

A rigid body has inertia tensor I (symmetric 3×3 matrix). Find the principal axes.

Problem:In an arbitrary frame, the inertia tensor I has off-diagonal elements (products of inertia). L=IωhasLL = I\omega has L not parallel to ω in general.
Spectral theorem:Since I is symmetric (real Hermitian), it has 3 real eigenvalues I1,I2,I3I_{1}, I_{2}, I_{3} (principal moments) and orthogonal eigenvectors (principal axes).
In principal frame:I = diag(I1,I2,I3I_{1}, I_{2}, I_{3}). Euler's equations simplify: I1ω˙1=(I2−I3)ω2ω3I_{1}\dot{\omega}_{1} = (I_{2}-I_{3})\omega_{2}\omega_{3}, etc.
Symmetry:A body with 3-fold or higher symmetry axis has two equal principal moments → axisymmetric top. Spherical symmetry →I1=I2=I3→no\to I_{1} = I_{2} = I_{3} \to no precession.
Definition LA.3 — Common Traps
  • Basis vectors are choices: physical vectors and states are independent of representation.
  • Eigenvectors can be degenerate: a degenerate eigenspace needs a basis choice.
  • Hermitian operators have real eigenvalues: this is why they represent observables in quantum mechanics.
  • Matrix multiplication is not commutative: operator order can change the result.
Exercises — LA.1–LA.4 Linear Algebra for Physics
1.
Find the larger eigenvalue of the matrix A=(4123)A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}.
Straightforward
2.
Find the rank of the matrix A=(123246)A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{pmatrix}.
Straightforward
3.Apply Gram-Schmidt orthogonalization to the vectors v1=(1,1,0),v2=(1,0,1),v3=(0,1,1)v_{1}=(1,1,0), v_{2}=(1,0,1), v_{3}=(0,1,1) to produce an orthonormal basis.
Straightforward
4.Write the matrix representations of the harmonic oscillator Hamiltonian H and position operator x in the energy eigenstate basis {|0⟩, |1⟩, |2⟩, |3⟩}. What selection rule does this imply?
Intermediate
5.Use the Cayley-Hamilton theorem to find a recursion for AnA^{n} where A=(1110)A = \begin{pmatrix} 1 & 1 \\ 1 & 0 \end{pmatrix}. How does this relate to Fibonacci numbers?
Intermediate
6.Discuss the position and momentum operators as operators on L2(R)L^{2}(ℝ). Why do they not have normalizable eigenstates? How does the continuous spectrum relate to the uncertainty principle?
Challenging
Key Takeaways
  • Inner product ⟨u,v⟩: Cauchy-Schwarz gives ‖u∥∥v∥≥∣⟨u,v⟩∣u\|\|v\| \ge |⟨u,v⟩|.
  • Eigenvalues from det⁡(A−λI)=0\det(A-\lambda I)=0; eigenvectors span the eigenspace.
  • Hermitian: A=A†A=A^\dagger. Eigenvalues real, eigenvectors orthogonal, spectral theorem holds.
  • Unitary: U†U=IU^\dagger U=I. Eigenvalues on unit circle. Time evolution is unitary (probability conserved).
  • SVD:A=U∑V†SVD: A = U\sum V^\dagger. Foundation of PCA, low-rank approximation, and least squares.
  • Commuting Hermitian operators share eigenbases — simultaneously measurable observables.