Mathematics · Upper Division
Probability & Statistics for Physics Physics is inherently probabilistic at the quantum level and practical level alike. Probability theory, distributions, and statistical inference are tools every physicist uses — from analyzing data to formulating quantum mechanics.
Prerequisites C a l c u l u s ( C h .22 ) ⋅ B a s i c p r o b a b i l i t y c o n c e p t s Calculus (Ch. 22) \cdot Basic probability concepts C a l c u l u s ( C h .22 ) ⋅ B a s i c p r o babi l i t y co n ce pt s Learning Goals
Identify the Gaussian, Poisson, Maxwell-Boltzmann, and Lorentzian distributions and their physical applications. Apply the central limit theorem to determine how measurement uncertainty decreases with sample size. Propagate errors through a function of multiple independent measurements using partial derivatives. Update beliefs from data using Bayes' theorem, distinguishing prior, likelihood, and posterior. Interpret the chi-squared statistic and reduced chi-squared to assess goodness of fit. PR.1 Probability Distributions Gaussian (Normal): P ( x ) = ( 1 / ( 2 π σ 2 ) ) e ( − ( x − μ ) 2 / ( 2 σ 2 ) ) . A p p e a r s w h e n e v e r m a n y i n d e p e n d e n t r a n d o m c o n t r i b u t i o P(x) = (1/\sqrt(2\pi\sigma^{2})) e^(-(x-\mu)^{2}/(2\sigma^{2})). Appears whenever many independent random contributio P ( x ) = ( 1/ ( 2 π σ 2 )) e ( − ( x − μ ) 2 / ( 2 σ 2 )) . A pp e a r s w h e n e v er man y in d e p e n d e n t r an d o m co n t r ib u t i o Poisson: P ( k ) = e − λ λ k / k ! — p r o b a b i l i t y o f k e v e n t s w h e n m e a n i s λ . R a d i o a c t i v e d e c a y c o u n t i n g P(k) = e^{-\lambda} \lambdaᵏ/k! — probability of k events when mean is \lambda. Radioactive decay counting P ( k ) = e − λ λ k / k ! — p r o babi l i t y o f k e v e n t s w h e nm e ani s λ . R a d i o a c t i v e d ec a y co u n t in g Maxwell-Boltzmann: f ( v ) = 4 π n ( m / ( 2 \pik B T ) ) 3 / 2 v 2 e ( − m v 2 / ( 2 k B T ) ) . S p e e d d i s t r i b u t i o n o f i d e a l g a s m o l e c u l f(v) = 4\pi n(m/(2\pik_BT))^{3/2} v^{2} e^(-mv^{2}/(2k_BT)). Speed distribution of ideal gas molecul f ( v ) = 4 π n ( m / ( 2 \pik B T ) ) 3/2 v 2 e ( − m v 2 / ( 2 k B T )) . S p ee dd i s t r ib u t i o n o f i d e a l g a s m o l ec u l Cauchy (Lorentzian): P ( x ) = ( Γ / π ) / ( x 2 + Γ 2 ) . H e a v y t a i l s ; r e s o n a n c e l i n e s h a p e s , r a n d o m w a l k i n 2 D . N o f i n i t e m e a P(x) = (Γ/\pi)/(x^{2}+Γ^{2}). Heavy tails; resonance line shapes, random walk in 2D. No finite mea P ( x ) = ( Γ/ π ) / ( x 2 + Γ 2 ) . H e a v y t ai l s ; r eso nan ce l in es ha p es , r an d o m w a l k in 2 D . N o f ini t e m e a PR.2 Central Limit Theorem and Error Propagation I f X 1 , X 2 , ⋯ , X n a r e i n d e p e n d e n t i d e n t i c a l l y d i s t r i b u t e d r a n d o m v a r i a b l e s w i t h m e a n μ a n If X_{1}, X_{2}, \cdots, X_{n} are independent identically distributed random variables with mean \mu an I f X 1 , X 2 , ⋯ , X n a r e in d e p e n d e n t i d e n t i c a l l y d i s t r ib u t e d r an d o m v a r iab l es w i t hm e an μ an d v a r i a n c e σ 2 , t h e n t h e s a m p l e m e a n X ˉ = ( X 1 + ⋯ + X n ) / n s a t i s f i e s d variance \sigma^{2}, then the sample mean X̄ = (X_{1}+\cdots+X_{n})/n satisfies d v a r ian ce σ 2 , t h e n t h es am pl e m e an X ˉ = ( X 1 + ⋯ + X n ) / n s a t i s f i es :
(X ˉ − μ ) / ( σ / \sqrtn ) → N ( 0 , 1 ) a s n → ∞ X̄ - \mu) / (\sigma/\sqrtn) \to N(0,1) as n \to \infty X ˉ − μ ) / ( σ / \sqrtn ) → N ( 0 , 1 ) a s n → ∞ r e g a r d l e s s o f t h e d i s t r i b u t i o n o f i n d i v i d u a l X i ( p r o v i d e d t h e v a r i a n c e i s f i n i t e ) . T h e s t a regardless of the distribution of individual X_{i} (provided the variance is finite). The sta r e g a r d l esso f t h e d i s t r ib u t i o n o f in d i v i d u a l X i ( p r o v i d e d t h e v a r ian ce i s f ini t e ) . T h es t a n d a r d e r r o r o f t h e m e a n i s σ / \sqrtn — a v e r a g i n g n m e a s u r e m e n t s r e d u c e s u n c e r t a i n t y b y \sqrtn . T h i s ndard error of the mean is \sigma/\sqrtn — averaging n measurements reduces uncertainty by \sqrtn. This n d a r d er r or o f t h e m e ani s σ / \sqrtn — a v er a g in g nm e a s u r e m e n t sr e d u ces u n cer t ain t y b y \sqrtn . T hi s is why averaging works and why the Gaussian distribution is universal.
Error propagation: if a quantity z = f(x, y, ...) depends on measured quantities with uncertainties σ_x, σ_y:
σ z 2 = ( \partialf / \partialx ) 2 σ x 2 + ( \partialf / \partialy ) 2 σ y 2 + 2 ( \partialf / \partialx ) ( \partialf / \partialy ) σ x y ( e r r o r p r o p a g a t i o n ) \sigma_z^{2} = (\partialf/\partialx)^{2} \sigma_x^{2} + (\partialf/\partialy)^{2} \sigma_y^{2} + 2(\partialf/\partialx)(\partialf/\partialy) \sigma_xy \qquad (error propagation) σ z 2 = ( \partialf / \partialx ) 2 σ x 2 + ( \partialf / \partialy ) 2 σ y 2 + 2 ( \partialf / \partialx ) ( \partialf / \partialy ) σ x y ( er r or p r o p a g a t i o n ) (PR.1) For independent measurements (σ_xy = 0): σ_z² = Σᵢ (∂f/∂xᵢ)² σᵢ². For z = x·y: (σ_z/z)² = (σ_x/x)² + (σ_y/y)² (relative errors add in quadrature).
A G e i g e r c o u n t e r r e c o r d s a n a v e r a g e c o u n t r a t e λ = 100 c o u n t s / m i n u t e . H o w p r e c i s e l y i s t h i A Geiger counter records an average count rate \lambda = 100 counts/minute. How precisely is thi A G e i g er co u n t er r ecor d s ana v er a g eco u n t r a t e λ = 100 co u n t s / min u t e . H o w p r ec i se l y i s t hi s k n o w n a f t e r T = 5 m i n u t e s s known after T = 5 minutes s k n o w na f t er T = 5 min u t es
Poisson statistics: F o r N e v e n t s c o u n t e d , v a r i a n c e = N ( P o i s s o n p r o p e r t y For N events counted, variance = N (Poisson property F or N e v e n t sco u n t e d , v a r ian ce = N ( P o i sso n p r o p er t y .
Total counts: N = \lambdaT = 100 × 5 = 500 c o u n t s N = \lambdaT = 100\times5 = 500 counts N = \lambdaT = 100 × 5 = 500 co u n t s Statistical uncertainty: σ N = \sqrtN = 5 00 ≈ 22.4 \sigma_N = \sqrtN = \sqrt500 \approx 22.4 σ N = \sqrtN = 5 00 ≈ 22.4 Uncertainty in rate: σ λ = σ N / T = 22.4 / 5 = 4.5 c o u n t s / m i n \sigma_\lambda = \sigma_N/T = 22.4/5 = 4.5 counts/min σ λ = σ N / T = 22.4/5 = 4.5 co u n t s / min Relative precision: σ λ / λ = 1 / \sqrtN = 1 / 5 00 = 4.5 \sigma_\lambda/\lambda = 1/\sqrtN = 1/\sqrt500 = 4.5%. To get 1% precision: need N = 10,000 counts (T = 100 min σ λ / λ = 1/ \sqrtN = 1/ 5 00 = 4.5 .
Systematic check: A f t e r T m i n u t e s , i f w e s e e N ≠ \lambdaT b y m o r e t h a n 3 ( \lambdaT ) , i n v e s t i g a t e s y s t e m a t i c e r r o r s . T h e After T minutes, if we see N ≠ \lambdaT by more than 3\sqrt(\lambdaT), investigate systematic errors. The A f t er T min u t es , i f w esee N = \lambdaT b y m or e t han 3 ( \lambdaT ) , in v es t i g a t esy s t e ma t i cer r or s . T h e \sqrtN r u l e i s f u n d a m e n t a l — s h o t n o i s e i s i r r e d u c i b l e \sqrtN rule is fundamental — shot noise is irreducible \sqrtN r u l e i s f u n d am e n t a l — s h o t n o i se i s i r r e d u c ib l e PR.3 Bayesian Inference Bayesian statistics provides a framework for updating beliefs with data:
P ( θ ∣ d a t a ) ∝ P ( d a t a ∣ θ ) × P ( θ ) ( B a y e s ′ t h e o r e m ) P(\theta|data) \propto P(data|\theta) \times P(\theta) \qquad (Bayes' theorem) P ( θ ∣ d a t a ) ∝ P ( d a t a ∣ θ ) × P ( θ ) ( B a y e s ′ t h eor e m ) (PR.2) Here P(θ) is the prior (belief before seeing data), P(data|θ) is thelikelihood (probability of observing the data given parameter θ), and P(θ|data) is the posterior (updated belief). Bayesian methods are especially powerful when: (1) Prior information is available (previous experiments). (2) Data is limited. (3) Model comparison is needed. In gravitational wave detection, Bayesian parameter estimation recovers black hole masses, spins, and sky location from the noisy LIGO signal.
PR.4 The χ² Distribution and Goodness of Fit For fitting a model with k free parameters to n data points, thechi-squared statistic is:
χ 2 = ∑ i ( y i − f ( x i ) ) 2 / σ i 2 ( c h i − s q u a r e d ) \chi^{2} = \sum_{i} (y_{i} - f(x_{i}))^{2} / \sigma_i^{2} \qquad (chi-squared) χ 2 = i ∑ ( y i − f ( x i ) ) 2 / σ i 2 ( c hi − s q u a r e d ) (PR.3) Under the null hypothesis (model is correct), χ²/(n−k) should be approximately 1 (reduced chi-squared). Values much greater than 1 indicate poor fit or underestimated errors; values much less than 1 indicate overestimated errors or overfitting.
The p-value — probability of observing χ² this large or larger by chance — is computed from the chi-squared distribution with ν = n−k degrees of freedom. Convention: p < 0.05 → reject null hypothesis at 95% confidence; in particle physics, p < 2.87×10⁻⁷ (5σ) is the "discovery" threshold.
Y o u m e a s u r e N ( t ) = N 0 e − \lambdat a t 5 t i m e p o i n t s . F i n d t h e b e s t λ b y m a x i m u m l i k e l i h o o d You measure N(t) = N_{0} e^{-\lambdat} at 5 time points. Find the best \lambda by maximum likelihood Y o u m e a s u r e N ( t ) = N 0 e − \lambdat a t 5 t im e p o in t s . F in d t h e b es t λb y ma x im u m l ik e l ih oo d
Likelihood: E a c h m e a s u r e m e n t N i a t t i m e t i h a s P o i s s o n u n c e r t a i n t y . L o g − l i k e l i h o o d : l n L = ∑ i [ N i l n ( N Each measurement N_{i} at time t_{i} has Poisson uncertainty. Log-likelihood: ln L = \sum_{i} [N_{i} ln(N E a c hm e a s u r e m e n t N i a tt im e t i ha s P o i sso n u n cer t ain t y . L o g − l ik e l ih oo d : l n L = ∑ i [ N i l n ( N 0 e − \lambdat i ) − N 0 e ( − \lambdat i _{0} e^{-\lambdat_{i}}) - N_{0} e^(-\lambdat_{i} 0 e − \lambdat i ) − N 0 e ( − \lambdat i .
Maximize over λ: d ( l n L ) / d λ = ∑ i [ − N i t i + N 0 t i e − \lambdat i ] = 0 d(ln L)/d\lambda = \sum_{i} [-N_{i} t_{i} + N_{0} t_{i} e^{-\lambdat_{i}}] = 0 d ( l n L ) / d λ = ∑ i [ − N i t i + N 0 t i e − \lambdat i ] = 0 Equation: \sum_{i} N_{i} t_{i} = N_{0} \sum_{i} t_{i} e^{-\lambdat_{i}} — solve numerically for \lambdâ
Uncertainty: σ λ 2 = − 1 / ( d 2 l n L / d λ 2 ) = 1 / ( N 0 ∑ i t i 2 e − \lambdat i ) — f r o m t h e F i s h e r i n f o r m a t i o n \sigma_\lambda^{2} = -1/(d^{2}lnL/d\lambda^{2}) = 1/(N_{0} \sum_{i} t_{i}^{2} e^{-\lambdat_{i}}) — from the Fisher information σ λ 2 = − 1/ ( d 2 l n L / d λ 2 ) = 1/ ( N 0 ∑ i t i 2 e − \lambdat i ) — f r o m t h e F i s h er in f or ma t i o n .
Practical: For Gaussian noise, maximum likelihood reduces to least squares. For Poisson data (counting experiments), use Poisson likelihood.
Probability density is not probability: integrate density over an interval to get probability.Independence is an assumption: uncorrelated variables need not be independent.Bayes' theorem updates beliefs with evidence: priors and likelihoods play different roles.Standard deviation and standard error differ: one describes spread, the other uncertainty in a mean.1. A Geiger counter records 500 total counts. What is the relative statistical uncertainty on the count rate?
% Check
Get hint Ask AI tutor Solution Straightforward
2. A c o u n t r a t e o f 100 c o u n t s / m i n i s m e a s u r e d o v e r T = 5 m i n . F i n d t h e u n c e r t a i n t y i n t h e m e a A count rate of 100 counts/min is measured over T = 5 min. Find the uncertainty in the mea A co u n t r a t eo f 100 co u n t s / mini s m e a s u r e d o v er T = 5 min . F in d t h e u n cer t ain t y in t h e m e a sured rate.
counts/min Check
Get hint Ask AI tutor Solution Straightforward
3. F o r n i t r o g e n m o l e c u l e s ( M = 28 g / m o l ) a t T = 300 K , f i n d t h e m e a n , r m s , a n d m o s t p r o b a b l e For nitrogen molecules (M = 28 g/mol) at T = 300 K, find the mean, rms, and most probable F or ni t r o g e nm o l ec u l es ( M = 28 g / m o l ) a tT = 300 K , f in d t h e m e an , r m s , an d m os tp r o bab l e speeds. Compare to the speed of sound.
Straightforward Solution
4. E x p l a i n t h e t − t e s t f o r c o m p a r i n g t w o e x p e r i m e n t a l m e a n s . W h e n i s 2 σ s u f f i c i e n t a n d w h e n i s Explain the t-test for comparing two experimental means. When is 2\sigma sufficient and when is E x pl ain t h e t − t es t f or co m p a r in g tw oe x p er im e n t a l m e an s . W h e ni s 2 σ s u f f i c i e n t an d w h e ni s 5 σ r e q u i r e d ? W h a t i s t h e l o o k − e l s e w h e r e e f f e c t 5\sigma required? What is the look-elsewhere effect 5 σ r e q u i r e d ? W ha t i s t h e l oo k − e l se w h er ee f f ec t Intermediate Solution
5. P r o p a g a t e e r r o r s f o r t h e p e n d u l u m m e a s u r e m e n t g = 4 π 2 L / T 2 : l e n g t h L = 1.000 ± 0.001 m , p e r Propagate errors for the pendulum measurement g = 4\pi^{2}L/T^{2}: length L = 1.000 \pm 0.001 m, per P r o p a g a t eer r or s f or t h e p e n d u l u mm e a s u r e m e n t g = 4 π 2 L / T 2 : l e n g t h L = 1.000 ± 0.001 m , p er i o d T = 2.007 ± 0.002 s . W h a t i s σ g iod T = 2.007 \pm 0.002 s. What is \sigma_g i o d T = 2.007 ± 0.002 s . W ha t i s σ g ?
Intermediate Solution
6. C o n n e c t i n f o r m a t i o n e n t r o p y S = − ∑ p i l n p i t o t h e r m o d y n a m i c e n t r o p y v i a J a y n e s ′ s m a x i m u m Connect information entropy S = -\sum p_{i} ln p_{i} to thermodynamic entropy via Jaynes's maximum C o nn ec t in f or ma t i o n e n t r o p y S = − ∑ p i l n p i t o t h er m o d y nami ce n t r o p y v ia J a y n e s ′ s ma x im u m entropy principle. State the Cramér-Rao bound and its quantum analog.
Challenging Solution
Key Takeaways
Key distributions: Gaussian (CLT, noise), Poisson (counting), Maxwell-Boltzmann (gas speeds), Lorentzian (resonances). C L T : X ˉ → N ( μ , σ / \sqrtn ) . S t a n d a r d e r r o r s c a l e s a s 1 / \sqrtn — m o r e d a t a h e l p s CLT: X̄ \to N(\mu, \sigma/\sqrtn). Standard error scales as 1/\sqrtn — more data helps C L T : X ˉ → N ( μ , σ / \sqrtn ) . S t an d a r d er r or sc a l es a s 1/ \sqrtn — m or e d a t ah e l p s E r r o r p r o p a g a t i o n : σ z 2 = ∑ i ( \partialf / \partialx i ) 2 σ i 2 . R e l a t i v e e r r o r s a d d i n q u a d r a t u r e f o r p r o d u c t s Error propagation: \sigma_z^{2} = \sum_{i} (\partialf/\partialx_{i})^{2} \sigma_{i}^{2}. Relative errors add in quadrature for products E r r or p r o p a g a t i o n : σ z 2 = ∑ i ( \partialf / \partialx i ) 2 σ i 2 . R e l a t i v eer r or s a dd in q u a d r a t u r e f or p r o d u c t s B a y e s : p o s t e r i o r ∝ l i k e l i h o o d × p r i o r . P o w e r f u l f o r l i m i t e d d a t a a n d m o d e l c o m p a r i s o n Bayes: posterior \propto likelihood \times prior. Powerful for limited data and model comparison B a y es : p os t er i or ∝ l ik e l ih oo d × p r i or . P o w er f u l f or l imi t e dd a t aan d m o d e l co m p a r i so n χ 2 / ν ≈ 1 f o r g o o d f i t . p − v a l u e m e a s u r e s c o n s i s t e n c y w i t h n u l l h y p o t h e s i s \chi^{2}/\nu \approx 1 for good fit. p-value measures consistency with null hypothesis χ 2 / ν ≈ 1 f or g oo df i t . p − v a l u e m e a s u r esco n s i s t e n cy w i t hn u l l h y p o t h es i s M a x i m u m e n t r o p y p r i n c i p l e : B o l t z m a n n d i s t r i b u t i o n m a x i m i z e s S s u b j e c t t o ⟨ E ⟩ = U Maximum entropy principle: Boltzmann distribution maximizes S subject to ⟨E⟩ = U M a x im u m e n t r o p y p r in c i pl e : B o l t z mann d i s t r ib u t i o nma x imi z es S s u bj ec tt o ⟨ E ⟩ = U