Complex Systems & Emergence
Complex systems — from neural networks to earthquakes to the internet — exhibit emergent collective behavior not reducible to their parts: power laws, self-organized criticality, phase transitions in networks, and scaling laws that transcend microscopic details.
- Identify power-law distributions on a log-log plot and explain why they have no characteristic scale, dominating statistics with rare extreme events.
- Describe self-organised criticality using the BTW sandpile model and explain how slow drive plus fast relaxation tunes the system to criticality without external parameter adjustment.
- Analyse the Kuramoto synchronisation transition and derive the critical coupling for a Lorentzian frequency distribution.
- Distinguish Erdős-Rényi, small-world, and scale-free network topologies and explain the percolation threshold for each.
- Calculate Shannon entropy for a discrete distribution and state Landauer's principle connecting information erasure to thermodynamic dissipation.
CS.1 Power Laws and Scale Invariance
Many natural distributions follow power laws: . Unlike exponential distributions, power laws have no characteristic scale — they are scale-invariant. In a log-log plot, power laws appear as straight lines with slope −α.
Examples: earthquake magnitudes (Gutenberg-Richter: log N ∝ −b M, b≈1), city sizes (Zipf: rank × size ≈ const), word frequencies (Zipf s=1), internet node degrees (Barabási-Albert: ), lunar craters, wealth distributions (Pareto), solar flares, extinction events.
CS.2 Self-Organized Criticality
Per Bak (1987): some systems naturally evolve to a critical state — without any parameter tuning — and exhibit scale-free fluctuations. The sandpile model: add grains one at a time; when a site exceeds a threshold, it topples and redistributes to neighbors. Avalanche size distribution: , τ ≈ 1.5.
The system self-organizes to the edge of stability — a critical state with long-range spatial and temporal correlations. The order parameter (avalanche size) fluctuates on all scales simultaneously.
SOC examples: earthquakes (Gutenberg-Richter), forest fires (), biological evolution (Bak-Sneppen: fitness of weakest species drives avalanche of co-evolution), neural avalanches (Beggs and Plenz 2003: cascade sizes power-law distributed in cortex), financial markets (volatility clustering, fat tails).
Analyze the Gutenberg-Richter (GR) law for earthquake statistics and extract the total seismic energy release rate.
CS.3 Network Theory
Erdős-Rényi (ER) random graph: N nodes, each pair connected with probability p. Giant component appears at p = 1/N (percolation threshold). Degree distribution: Poisson .
Small-world networks (Watts-Strogatz): start from regular ring lattice, randomly rewire fraction p of edges. For small p: high clustering coefficient C (like regular) + small average path length L (like random). The six degrees of separation arises from the small-world property. .
Percolation theory: fraction p of edges present. At critical , a giant connected component (GCC) appears. Order parameter: GCC size . Correlation length: . In 2D percolation the exponents are β=5/36 and ν=4/3; this is the q=1 Potts universality class, distinct from 2D Ising.
CS.4 Emergence and Collective Behavior
Flocking (Vicsek model): N self-propelled particles with velocity v₀, align with neighbors within radius r, add noise η. Order parameter: . Second-order phase transition in 2D from disorder (high η or low density) to ordered flock (low η, high density). Continuous transition in 2D violates Mermin-Wagner! (resolved: activity breaks time-reversal, enabling long-range order).
Kuramoto model: N coupled oscillators with natural frequencies (drawn from a Lorentzian distribution with width γ):
Order parameter: . Phase transition at : for , r = 0 (incoherent); for , r > 0 (partial synchrony). near — mean-field critical behavior. Applications: firefly synchrony, cardiac pacemaker cells, power grid stability, Josephson junction arrays, circadian rhythms.
CS.5 Information Theory and Entropy
Shannon entropy: (bits). Maximum H = log₂ N for uniform distribution. Minimum H = 0 for deterministic system.
Kolmogorov complexity: K(x) = length of shortest program that outputs x. K(x) ≤ |x| + O(1). A random string has K(x) ≈ |x| (incompressible). Connection to physics: Boltzmann entropy relates to the number of microstates; Shannon entropy H relates to the information needed to specify a microstate. Landauer's principle: erasing 1 bit costs of energy — information is physical.
- Power laws are not proven by one log-log line: finite data can mimic scaling over a narrow range.
- Scale-free does not mean fully connected: most nodes can have low degree while hubs dominate connectivity.
- Self-organized criticality requires slow drive and fast relaxation: continuous forcing can destroy the critical state.
- Information entropy depends on probabilities: a biased source carries less uncertainty per symbol than a uniform one.
- Power laws are scale-invariant. Log-log linear. Common in nature: earthquakes, cities, networks.
- SOC: systems drive themselves to criticality without tuning. Sandpile → avalanche . Key: slow drive, fast relaxation.
- Scale-free networks: from preferential attachment. Robust to random failure; fragile to hub attacks.
- Kuramoto model: for Lorentzian. Synchronization transition: . Mean-field .
- Percolation: giant component at for ER graph. A continuous transition like magnetic ones, but in its own universality class.
- Landauer principle: erasing 1 bit costs . Connects information entropy to thermodynamic entropy.