Mathematics · Upper Division

Probability & Statistics for Physics

Physics is inherently probabilistic at the quantum level and practical level alike. Probability theory, distributions, and statistical inference are tools every physicist uses — from analyzing data to formulating quantum mechanics.

PrerequisitesCalculus (Ch. 25) · Basic probability concepts
Learning Goals
  • Identify the Gaussian, Poisson, Maxwell-Boltzmann, and Lorentzian distributions and their physical applications.
  • Apply the central limit theorem to determine how measurement uncertainty decreases with sample size.
  • Propagate errors through a function of multiple independent measurements using partial derivatives.
  • Update beliefs from data using Bayes' theorem, distinguishing prior, likelihood, and posterior.
  • Interpret the chi-squared statistic and reduced chi-squared to assess goodness of fit.

PR.1 Probability Distributions

Definition PR.1 — Key Distributions in Physics
Gaussian (Normal): P(x)=12πσ2 e−(x−μ)2/(2σ2)P(x) = \dfrac{1}{\sqrt{2\pi\sigma^2}}\, e^{-(x-\mu)^2/(2\sigma^2)}. Appears whenever many independent random contributions add (CLT). Quantum measurement uncertainties, thermal fluctuations, noise.Poisson: P(k)=e−λλk/k!P(k) = e^{-\lambda} \lambda^k / k! — probability of k events when mean is λ. Radioactive decay counting, photon shot noise, rare events.Maxwell-Boltzmann: f(v)=4πn(m2πkBT)3/2v2 e−mv2/(2kBT)f(v) = 4\pi n \left(\dfrac{m}{2\pi k_B T}\right)^{3/2} v^2\, e^{-mv^2/(2k_B T)}. Speed distribution of ideal gas molecules.Cauchy (Lorentzian): P(x)=(Γ/π)/(x2+Γ2)P(x) = (\Gamma/\pi)/(x^{2}+\Gamma^{2}). Heavy tails; resonance line shapes, random walk in 2D. No finite mean or variance!

PR.2 Central Limit Theorem and Error Propagation

Theorem PR.1 — Central Limit Theorem
IfX1,X2,⋯ ,XnIf X_{1}, X_{2}, \cdots, X_{n} are independent identically distributed random variables with mean μ and variance σ2\sigma^{2}, then the sample mean Xˉ=(X1+⋯+Xn)/n\bar{X} = (X_{1}+\cdots+X_{n})/n satisfies:(Xˉ−μ)/(σ/n)→N(0,1\bar{X} - \mu) / (\sigma/\sqrt n) \to N(0,1) as n→∞n \to \inftyregardless of the distribution of individual XiX_{i} (provided the variance is finite). The standard error of the mean is σ/n—\sigma/\sqrt n — averaging n measurements reduces uncertainty by n\sqrt n. This is why averaging works and why the Gaussian distribution is universal.

Error propagation: if a quantity z = f(x, y, ...) depends on measured quantities with uncertainties σx,σy\sigma_x, \sigma_y:

σz2=(∂f∂x)2σx2+(∂f∂y)2σy2+2 ∂f∂x∂f∂y σxy(error propagation)\sigma_z^2 = \left(\frac{\partial f}{\partial x}\right)^2 \sigma_x^2 + \left(\frac{\partial f}{\partial y}\right)^2 \sigma_y^2 + 2\,\frac{\partial f}{\partial x}\frac{\partial f}{\partial y}\, \sigma_{xy} \qquad \text{(error propagation)}(PR.1)

For independent measurements (σxy=0\sigma_{xy} = 0): σz2=∑i(∂f/∂xi)2σi2\sigma_z^2 = \sum_i (\partial f/\partial x_i)^2 \sigma_i^2. For z = x·y:(σz/z)2=(σx/x)2+(σy/y)2(\sigma_z/z)^2 = (\sigma_x/x)^2 + (\sigma_y/y)^2 (relative errors add in quadrature).

Example PR.1 — Radioactive Decay Counting Statistics

A Geiger counter records an average count rate λ=100\lambda = 100 counts/minute. How precisely is this known after T=5T = 5 minutes?

Poisson statistics:For N events counted, variance = N (Poisson property).
Total counts:N=λT=100×5=500N = \lambda T = 100\times 5 = 500 counts
Statistical uncertainty:σN=N=500≈22.4\sigma_{N} = \sqrt N = \sqrt 500 \approx 22.4
Uncertainty in rate:σλ=σN/T=22.4/5=4.5\sigma_\lambda = \sigma_{N}/T = 22.4/5 = 4.5 counts/min
Relative precision:σλ/λ=1/N=1/500=4.5\sigma_\lambda/\lambda = 1/\sqrt N = 1/\sqrt 500 = 4.5%. To get 1% precision: need N=10,000N = 10,000 counts (T=100minT = 100 min).
Systematic check:After T minutes, if we see N≠λTN ≠ \lambda T by more than 3λT3\sqrt{\lambda T}, investigate systematic errors. TheNThe \sqrt N rule is fundamental — shot noise is irreducible.

PR.3 Bayesian Inference

Bayesian statistics provides a framework for updating beliefs with data:

P(θ∣data)∝P(data∣θ)×P(θ)(Bayes’ theorem)P(\theta \mid \text{data}) \propto P(\text{data} \mid \theta) \times P(\theta) \qquad \text{(Bayes' theorem)}(PR.2)

Here P(θ) is the prior (belief before seeing data), P(data|θ) is thelikelihood (probability of observing the data given parameter θ), and P(θ|data) is the posterior (updated belief). Bayesian methods are especially powerful when: (1) Prior information is available (previous experiments). (2) Data is limited. (3) Model comparison is needed. In gravitational wave detection, Bayesian parameter estimation recovers black hole masses, spins, and sky location from the noisy LIGO signal.

PR.4 The χ² Distribution and Goodness of Fit

For fitting a model with k free parameters to n data points, thechi-squared statistic is:

χ2=∑i(yi−f(xi))2σi2(chi-squared)\chi^2 = \sum_i \frac{\bigl(y_i - f(x_i)\bigr)^2}{\sigma_i^2} \qquad \text{(chi-squared)}(PR.3)

Under the null hypothesis (model is correct), χ²/(n−k) should be approximately 1 (reduced chi-squared). Values much greater than 1 indicate poor fit or underestimated errors; values much less than 1 indicate overestimated errors or overfitting.

The p-value — probability of observing χ² this large or larger by chance — is computed from the chi-squared distribution with ν = n−k degrees of freedom. Convention: p < 0.05 → reject null hypothesis at 95% confidence; in particle physics, p < 2.87×10⁻⁷ (5σ) is the "discovery" threshold.

Example PR.2 — Fitting a Decay Curve

You measure N(t)=N0e−λtN(t) = N_{0} e^{-\lambda t} at 5 time points. Find the best λ by maximum likelihood.

Likelihood:Each measurement NiN_{i} at time tit_{i} has Poisson uncertainty. Log-likelihood: ln⁡L=∑i[Niln⁡(N0e−λti)−N0e−λti]\ln L = \sum_{i} [N_{i} \ln(N_{0} e^{-\lambda t_{i}}) - N_{0} e^{-\lambda t_{i}}].
Maximize over λ:d(ln⁡L)/dλ=∑i[−Niti+N0tie−λti]=0d(\ln L)/d\lambda = \sum_{i} [-N_{i} t_{i} + N_{0} t_{i} e^{-\lambda t_{i}}] = 0
Equation:∑iNiti=N0∑itie−λti—\sum_{i} N_{i} t_{i} = N_{0} \sum_{i} t_{i} e^{-\lambda t_{i}} — solve numerically for λ^\hat{\lambda}.
Uncertainty:σλ2=−1/(d2lnL/dλ2)=1/(N0∑iti2e−λti)—\sigma_\lambda^{2} = -1/(d^{2}lnL/d\lambda^{2}) = 1/(N_{0} \sum_{i} t_{i}^{2} e^{-\lambda t_{i}}) — from the Fisher information.
Practical:For Gaussian noise, maximum likelihood reduces to least squares. For Poisson data (counting experiments), use Poisson likelihood.
Definition PR.2 — Common Traps
  • Probability density is not probability: integrate density over an interval to get probability.
  • Independence is an assumption: uncorrelated variables need not be independent.
  • Bayes' theorem updates beliefs with evidence: priors and likelihoods play different roles.
  • Standard deviation and standard error differ: one describes spread, the other uncertainty in a mean.
Exercises — PR.1–PR.4 Probability and Statistics
1.
A Geiger counter records 500 total counts. What is the relative statistical uncertainty on the count rate?
%
Straightforward
2.
A count rate of 100 counts/min is measured over T=5minT = 5 min. Find the uncertainty in the measured rate.
counts/min
Straightforward
3.For nitrogen molecules (M=28gM = 28 g/mol) at T=300KT = 300 K, find the mean, rms, and most probable speeds. Compare to the speed of sound.
Straightforward
4.Explain the t-test for comparing two experimental means. When is 2σ2\sigma sufficient and when is 5σ5\sigma required? What is the look-elsewhere effect?
Intermediate
5.Propagate errors for the pendulum measurement g=4π2L/T2g = 4\pi^{2}L/T^{2}: length L=1.000±0.001mL = 1.000 \pm 0.001 m, period T=2.007±0.002sT = 2.007 \pm 0.002 s. What is σg\sigma_{g}?
Intermediate
6.Connect information entropy S=−∑piln⁡piS = -\sum p_{i} \ln p_{i} to thermodynamic entropy via Jaynes's maximum entropy principle. State the Cramér-Rao bound and its quantum analog.
Challenging
Key Takeaways
  • Key distributions: Gaussian (CLT, noise), Poisson (counting), Maxwell-Boltzmann (gas speeds), Lorentzian (resonances).
  • CLT:Xˉ→N(μ,σ/n)CLT: \bar{X} \to N(\mu, \sigma/\sqrt n). Standard error scales as 1/n—1/\sqrt n — more data helps.
  • Error propagation: σz2=∑i(∂f/∂xi)2σi2\sigma_{z}^{2} = \sum_{i} (\partial f/\partial x_{i})^{2} \sigma_{i}^{2}. Relative errors add in quadrature for products.
  • Bayes: posterior ∝ likelihood × prior. Powerful for limited data and model comparison.
  • χ2/ν≈1\chi^{2}/\nu \approx 1 for good fit. p-value measures consistency with null hypothesis.
  • Maximum entropy principle: Boltzmann distribution maximizes S subject to ⟨E⟩=UE⟩ = U.