Probability

Lesson 9 of 14

LLN & Central Limit Theorem

Why averages settle down, and why everything becomes a bell curve.

Law of Large Numbers

Flip a fair coin 1000 times. The proportion of heads won't be exactly 0.5, but it will be close, and closer still with more flips. Formally, for i.i.d. XiX_i with mean μ\mu:

Xˉn=1n∑i=1nXi→μas n→∞\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i \to \mu \quad \text{as } n \to \infty

The mechanism is variance shrinkage: Var(Xˉn)=σ2n→0\text{Var}(\bar{X}_n) = \frac{\sigma^2}{n} \to 0. The LLN is why casinos and insurers profit reliably on thin edges, and why backtests need many independent bets before the sample mean means anything. It does not make tails due after a run of heads; there is no "law of averages". Convergence comes from dilution, not correction.