2.3 Modes of Convergence, Laws of Large Numbers and the CLT
What it means for random variables to converge — almost surely, in probability, in mean and in distribution — and the two great limit theorems built on these modes: the laws of large numbers, which say that averages settle down to the mean, and the central limit theorem, which says how the remaining error is distributed; with Slutsky's theorem and the delta method that make them usable in statistics.
Statistics is built on two experimental facts. Averages of many independent measurements settle down to a fixed number, and the scatter of those averages around that number has a bell-shaped histogram, whatever the distribution of a single measurement. The laws of large numbers and the central limit theorem turn these facts into theorems. They justify estimating a probability by a relative frequency, an integral by a Monte Carlo average and a population mean by a sample mean, and they produce the normal-based confidence intervals and tests of Chapters 4 and 5.
To state these theorems precisely we must first decide what "a sequence of random variables converges" means, and there are four standard answers: almost surely, in probability, in $r$-th mean and in distribution. They are genuinely different, and the implications between them, together with the counterexamples showing that no other implications hold, are among the most frequently examined facts in CSIR Part C. The section then proves the weak laws of large numbers of Chebyshev and Khintchine, states Kolmogorov's strong law (with a complete proof under a fourth-moment assumption), proves the Lindeberg–Lévy central limit theorem with the characteristic functions of §2.2, and develops the tools that statisticians use every day: normal approximations with continuity correction, the continuous mapping theorem, Slutsky's theorem and the delta method.
Four Modes of Convergence
Throughout, $X,X_1,X_2,\dots$ are random variables. For the first three modes they must be defined on one probability space $(\Omega,\mathcal F,P)$, because the definitions compare $X_n(\omega)$ with $X(\omega)$; the fourth mode compares only distributions.
- $X_n\to X$ almost surely (a.s., with probability one), written $X_n\xrightarrow{a.s.}X$, if $P\left(\{\omega:X_n(\omega)\to X(\omega)\}\right)=1$.
- $X_n\to X$ in probability, written $X_n\xrightarrow{P}X$, if $P(|X_n-X|\gt\varepsilon)\to0$ for every $\varepsilon\gt0$.
- For $r\gt0$, $X_n\to X$ in $r$-th mean (in $L^r$), written $X_n\xrightarrow{L^r}X$, if $E|X_n|^r\lt\infty$, $E|X|^r\lt\infty$ and $E|X_n-X|^r\to0$. For $r=2$ this is convergence in mean square (in quadratic mean).
- $X_n\to X$ in distribution, written $X_n\xrightarrow{d}X$, if $F_{X_n}(x)\to F_X(x)$ at every continuity point $x$ of $F_X$ (as defined in §2.2).
The set $\{X_n\to X\}$ is an event: it equals $\bigcap_{k\ge1}\bigcup_{N\ge1}\bigcap_{n\ge N}\{|X_n-X|\le1/k\}$, a countable combination of events. In words, $X_n(\omega)\to X(\omega)$ means that for every $k$ the deviations eventually stay below $1/k$. Almost-sure convergence is a statement about whole sample paths, convergence in probability only about the law of $X_n-X$ at each fixed $n$. The next theorem makes the difference precise.
For $\varepsilon\gt0$ let $A_n(\varepsilon)=\{|X_n-X|\gt\varepsilon\}$.
- $X_n\xrightarrow{a.s.}X$ if and only if, for every $\varepsilon\gt0$, $P\left(A_n(\varepsilon)\text{ i.o.}\right)=0$, if and only if, for every $\varepsilon\gt0$, $$P\Big(\sup_{m\ge n}|X_m-X|\gt\varepsilon\Big)\to0\qquad(n\to\infty).$$
- (Borel–Cantelli criterion) If $\sum_nP(|X_n-X|\gt\varepsilon)\lt\infty$ for every $\varepsilon\gt0$, then $X_n\xrightarrow{a.s.}X$.
- If the $X_n$ are independent and $c$ is a constant, then $X_n\xrightarrow{a.s.}c$ if and only if $\sum_nP(|X_n-c|\gt\varepsilon)\lt\infty$ for every $\varepsilon\gt0$.
Proof. (1) $X_n(\omega)\not\to X(\omega)$ iff there is a $k$ with $|X_n(\omega)-X(\omega)|\gt1/k$ for infinitely many $n$. So $\{X_n\not\to X\}=\bigcup_k\limsup_nA_n(1/k)$. This has probability $0$ iff each $\limsup_nA_n(1/k)$ does, and since $A_n(\varepsilon)$ shrinks as $\varepsilon$ grows, iff $P(\limsup_nA_n(\varepsilon))=0$ for every $\varepsilon\gt0$. Now $\limsup_nA_n(\varepsilon)=\bigcap_nB_n$ with $B_n=\bigcup_{m\ge n}A_m(\varepsilon)=\{\sup_{m\ge n}|X_m-X|\gt\varepsilon\}$, a decreasing sequence, so by continuity from above $P(\limsup A_n(\varepsilon))=\lim_nP(B_n)$.
(2) The first Borel–Cantelli lemma (Theorem 1.22 of §1.2) gives $P(A_n(\varepsilon)\text{ i.o.})=0$; apply (1).
(3) The events $\{|X_n-c|\gt\varepsilon\}$ are independent. If the series diverges for some $\varepsilon$, the second Borel–Cantelli lemma gives $P(|X_n-c|\gt\varepsilon\text{ i.o.})=1$, so $X_n\not\to c$ a.s. The converse is (2). ∎
- $X_n\xrightarrow{a.s.}X\ \Rightarrow\ X_n\xrightarrow{P}X$.
- $X_n\xrightarrow{L^r}X\ \Rightarrow\ X_n\xrightarrow{P}X$, for every $r\gt0$.
- $X_n\xrightarrow{L^s}X\ \Rightarrow\ X_n\xrightarrow{L^r}X$ for $0\lt r\lt s$.
- $X_n\xrightarrow{P}X\ \Rightarrow\ X_n\xrightarrow{d}X$.
- If $c$ is a constant, $X_n\xrightarrow{d}c\ \Rightarrow\ X_n\xrightarrow{P}c$.
- If $X_n\xrightarrow{P}X$, then some subsequence satisfies $X_{n_k}\xrightarrow{a.s.}X$. Consequently $X_n\xrightarrow{P}X$ iff every subsequence has a further subsequence converging to $X$ almost surely.
- (Dominated convergence) If $X_n\xrightarrow{P}X$ and $|X_n|\le Y$ with $E[Y^r]\lt\infty$, then $X_n\xrightarrow{L^r}X$. (More generally, $X_n\xrightarrow{L^1}X$ iff $X_n\xrightarrow{P}X$ and the family $\{X_n\}$ is uniformly integrable; statement only.)
No other implication holds in general (Example 1).
Proof. (1) By Theorem 2.30(1), $$P(|X_n-X|\gt\varepsilon)\le P\Big(\sup_{m\ge n}|X_m-X|\gt\varepsilon\Big)\to0.$$
(2) Markov's inequality (§2.2): $P(|X_n-X|\gt\varepsilon)\le E|X_n-X|^r/\varepsilon^r\to0$.
(3) Lyapunov's inequality (§2.2): $\left(E|X_n-X|^r\right)^{1/r}\le\left(E|X_n-X|^s\right)^{1/s}\to0$.
(4) Let $F_n$, $F$ be the cdfs and $\varepsilon\gt0$. If $X_n\le x$, then either $X\le x+\varepsilon$ or $|X_n-X|\gt\varepsilon$; and if $X\le x-\varepsilon$, then either $X_n\le x$ or $|X_n-X|\gt\varepsilon$. Hence $$F(x-\varepsilon)-P(|X_n-X|\gt\varepsilon)\le F_n(x)\le F(x+\varepsilon)+P(|X_n-X|\gt\varepsilon).$$ Letting $n\to\infty$, $$F(x-\varepsilon)\le\liminf F_n(x)\le\limsup F_n(x)\le F(x+\varepsilon).$$ At a continuity point $x$ of $F$, let $\varepsilon\downarrow0$.
(5) The cdf of the constant $c$ is continuous everywhere except at $c$. So $F_n(c-\varepsilon)\to0$ and $F_n(c+\varepsilon/2)\to1$, and $$P(|X_n-c|\gt\varepsilon)\le F_n(c-\varepsilon)+1-F_n(c+\varepsilon/2)\to0.$$
(6) Choose $n_1\lt n_2\lt\dots$ with $P(|X_{n_k}-X|\gt2^{-k})\lt2^{-k}$. The series $\sum_k2^{-k}$ converges, so by the first Borel–Cantelli lemma, a.s. $|X_{n_k}-X|\le2^{-k}$ for all large $k$, and $X_{n_k}\to X$ a.s. For the "iff": every subsequence of a sequence converging in probability converges in probability, so it has an a.s. convergent sub-subsequence. Conversely, if $X_n\not\xrightarrow{P}X$, there are $\varepsilon,\delta\gt0$ and a subsequence with $P(|X_{n_k}-X|\gt\varepsilon)\gt\delta$ for all $k$; no further subsequence can converge a.s., by (1).
(7) By (6), a subsequence converges a.s., so $|X|\le Y$ a.s. Then $|X_n-X|^r\le(2Y)^r$. For $\varepsilon\gt0$, $$E|X_n-X|^r\le\varepsilon^r+E\left[(2Y)^r\mathbf{1}\{|X_n-X|\gt\varepsilon\}\right].$$ Since $P(|X_n-X|\gt\varepsilon)\to0$ and $(2Y)^r$ is integrable, the last expectation tends to $0$ (absolute continuity of the integral: $E[Z\mathbf{1}_{A_n}]\to0$ whenever $E|Z|\lt\infty$ and $P(A_n)\to0$). So $\limsup E|X_n-X|^r\le\varepsilon^r$ for every $\varepsilon\gt0$. ∎
| implication | true? | reason or counterexample |
|---|---|---|
| a.s. ⟹ in probability | yes | Theorem 2.31(1) |
| $L^r$ ⟹ in probability | yes | Markov's inequality |
| $L^s$ ⟹ $L^r$ ($r\lt s$) | yes | Lyapunov's inequality |
| in probability ⟹ in distribution | yes | Theorem 2.31(4) |
| in probability ⟹ a.s. | no | typewriter; independent $\operatorname{Ber}(1/n)$ (Example 1(b), (c)) |
| a.s. ⟹ $L^1$ | no | $n\mathbf{1}\{U\lt1/n\}$ (Example 1(a)) |
| $L^1$ ⟹ a.s. | no | typewriter (Example 1(b)) |
| in distribution ⟹ in probability | no (yes to a constant) | $X_n=-X$, $X\sim N(0,1)$ (Example 1(d)) |
| $L^r$ ⟹ $L^s$ ($r\lt s$) | no | $n\mathbf{1}\{U\lt1/n\}\to0$ in $L^{1/2}$, not in $L^1$ |
- (Helly–Bray) $X_n\xrightarrow{d}X$ if and only if $E[g(X_n)]\to E[g(X)]$ for every bounded continuous $g:\mathbb{R}\to\mathbb{R}$.
- (Lévy, §2.2) $X_n\xrightarrow{d}X$ if and only if $\varphi_{X_n}(t)\to\varphi_X(t)$ for every $t$.
- (Integer-valued laws) If $X_n$ and $X$ take values in $\mathbb{Z}$, then $X_n\xrightarrow{d}X$ iff $P(X_n=k)\to P(X=k)$ for every $k\in\mathbb{Z}$.
- (Scheffé) If $X_n$ and $X$ have densities $f_n\to f$ almost everywhere, then $\int|f_n-f|\to0$; hence $\sup_B|P(X_n\in B)-P(X\in B)|\to0$ over all Borel sets $B$, and in particular $X_n\xrightarrow{d}X$.
Proof. (1) "If": fix a continuity point $x$ of $F$ and $\delta\gt0$. Let $g$ be $1$ on $(-\infty,x]$, $0$ on $[x+\delta,\infty)$ and linear in between; it is bounded and continuous with $\mathbf{1}_{(-\infty,x]}\le g\le\mathbf{1}_{(-\infty,x+\delta]}$. So $$\limsup F_n(x)\le\lim E[g(X_n)]=E[g(X)]\le F(x+\delta).$$ Similarly, with a function squeezed between $\mathbf{1}_{(-\infty,x-\delta]}$ and $\mathbf{1}_{(-\infty,x]}$, $\liminf F_n(x)\ge F(x-\delta)$. Let $\delta\downarrow0$. "Only if" (sketch): let $U\sim U(0,1)$ and put $Y_n=F_n^{-1}(U)$, $Y=F^{-1}(U)$ with the quantile functions of §1.3. Then $Y_n\overset{d}{=}X_n$, $Y\overset{d}{=}X$ (the quantile transform), and one checks that $F_n^{-1}(u)\to F^{-1}(u)$ at every continuity point $u$ of $F^{-1}$, hence for almost every $u$. So $Y_n\to Y$ a.s. (Skorokhod's representation), and dominated convergence gives $E[g(X_n)]=E[g(Y_n)]\to E[g(Y)]=E[g(X)]$.
(3) "Only if": $P(X_n=k)=F_n(k+\frac12)-F_n(k-\frac12)$, and $k\pm\frac12$ are continuity points of $F$. "If": fix $x$ and $M\in\mathbb{N}$. Finite sums converge, so $$\liminf F_n(x)\ge\lim\sum_{-M\le k\le x}P(X_n=k)=P(-M\le X\le x);$$ letting $M\to\infty$, $\liminf F_n(x)\ge F(x)$. In the same way $\liminf P(X_n\gt x)\ge P(x\lt X\le M)\to P(X\gt x)$, i.e. $\limsup F_n(x)\le F(x)$.
(4) $(f-f_n)^+\le f$ and $(f-f_n)^+\to0$ a.e., so $\int(f-f_n)^+\to0$ by dominated convergence. Since $\int(f-f_n)=1-1=0$, $\int|f-f_n|=2\int(f-f_n)^+\to0$. Finally $$|P(X_n\in B)-P(X\in B)|=\left|\int_B(f_n-f)\right|\le\int|f_n-f|.$$ ∎
Laws of Large Numbers
Let $S_n=X_1+\dots+X_n$ and $\bar X_n=S_n/n$. A weak law asserts $\bar X_n-E[\bar X_n]\xrightarrow{P}0$; a strong law asserts almost-sure convergence. The weak law of Chebyshev needs second moments but allows the variables to be merely uncorrelated and not identically distributed; Khintchine's weak law and Kolmogorov's strong law need iid variables but only a finite mean.
- (Markov's condition) If the $X_j$ have finite variances and $\operatorname{Var}(S_n)/n^2\to0$, then $\dfrac{S_n-E[S_n]}{n}\to0$ in $L^2$ and in probability.
- In particular, if the $X_j$ are uncorrelated with $\operatorname{Var}(X_j)\le C$ for all $j$, then $\bar X_n-\frac1n\sum_{j\le n}E[X_j]\xrightarrow{P}0$. If moreover $E[X_j]=\mu$ for all $j$, then $\bar X_n\xrightarrow{P}\mu$.
- (Bernoulli's law of large numbers) If $S_n\sim\operatorname{Bin}(n,p)$, then $S_n/n\xrightarrow{P}p$.
Proof. (1) $E\left[\left(\frac{S_n-ES_n}{n}\right)^2\right]=\frac{\operatorname{Var}(S_n)}{n^2}\to0$, which is $L^2$ convergence; Chebyshev's inequality (or Theorem 2.31(2)) gives convergence in probability. (2) For uncorrelated variables $\operatorname{Var}(S_n)=\sum_j\operatorname{Var}(X_j)\le Cn$, so $\operatorname{Var}(S_n)/n^2\le C/n\to0$. (3) is (2) for Bernoulli trials, with $C=p(1-p)$. ∎
Proof. Let $\varphi$ be the cf of $X_1$. By Theorem 2.19(2) of §2.2 with $n=1$, $\varphi(s)=1+i\mu s+o(|s|)$ as $s\to0$. By Theorem 2.18, $$\varphi_{\bar X_n}(t)=\varphi\!\left(\frac tn\right)^n=\left(1+\frac{i\mu t+\epsilon_n}{n}\right)^n,\qquad\epsilon_n=n\,o(|t|/n)\to0.$$ For complex $z_n\to z$, $(1+z_n/n)^n\to e^z$ (for large $n$, $|z_n/n|\lt\frac12$, and $n\operatorname{Log}(1+z_n/n)=z_n+O(|z_n|^2/n)\to z$). Hence $\varphi_{\bar X_n}(t)\to e^{i\mu t}$, the cf of the constant $\mu$, which is continuous. Lévy's continuity theorem gives $\bar X_n\xrightarrow{d}\mu$, and Theorem 2.31(5) upgrades this to convergence in probability. ∎
Let $X_1,X_2,\dots$ be iid.
- If $E|X_1|\lt\infty$, then $\bar X_n\xrightarrow{a.s.}\mu=E[X_1]$. (Statement; we prove it below under the extra assumption $E[X_1^4]\lt\infty$.)
- Conversely, if $E|X_1|=\infty$, then $\limsup_n|\bar X_n|=\infty$ a.s.; in particular $\bar X_n$ converges to a finite limit with probability $0$.
So for iid variables, $\bar X_n$ converges a.s. to a finite constant if and only if $E|X_1|\lt\infty$, and the constant is then $E[X_1]$.
Proof of (1) when $E[X_1^4]\lt\infty$. Replacing $X_j$ by $X_j-\mu$ we may assume $\mu=0$. Expand $E[S_n^4]=\sum_{i,j,k,l}E[X_iX_jX_kX_l]$. By independence and $E[X_j]=0$, a term vanishes whenever some index occurs exactly once. The surviving terms have all four indices equal ($n$ terms, each $E[X_1^4]$) or two distinct indices each occurring twice ($3n(n-1)$ ordered quadruples, each $\sigma^4$). Since $\sigma^4\le E[X_1^4]$ (Jensen), $$E[S_n^4]=nE[X_1^4]+3n(n-1)\sigma^4\le3n^2E[X_1^4].$$ Markov's inequality applied to $S_n^4$ gives $P(|\bar X_n|\gt\varepsilon)=P(S_n^4\gt n^4\varepsilon^4)\le\frac{3E[X_1^4]}{n^2\varepsilon^4}$, which is summable in $n$. Theorem 2.30(2) gives $\bar X_n\to0$ a.s. (The general case needs more work: truncation $X_j\mathbf{1}\{|X_j|\le j\}$, Kolmogorov's maximal inequality or Etemadi's subsequence argument.)
Proof of (2). For a non-negative $Y$, the tail formula (Theorem 1.27 of §1.3) gives $E[Y]=\int_0^\infty P(Y\gt t)\,dt\le1+\sum_{n\ge1}P(Y\ge n)$. Fix $c\gt0$ and apply this to $Y=|X_1|/c$: $\sum_nP(|X_n|\ge cn)=\sum_nP(|X_1|/c\ge n)=\infty$, since $E[Y]=\infty$. The events $\{|X_n|\ge cn\}$ are independent, so by the second Borel–Cantelli lemma $|X_n|\ge cn$ infinitely often, a.s. Since $\frac{X_n}{n}=\bar X_n-\frac{n-1}{n}\bar X_{n-1}$, on this event $|\bar X_n|+|\bar X_{n-1}|\ge c$ infinitely often, so $\limsup|\bar X_n|\ge c/2$. Intersect over $c\in\mathbb{N}$. ∎
Consequences used throughout statistics: the relative frequency of an event converges a.s. to its probability (apply the SLLN to indicators; for a fixed $x$ the empirical cdf $\hat F_n(x)\to F(x)$ a.s., and the Glivenko–Cantelli theorem, stated here without proof, gives $\sup_x|\hat F_n(x)-F(x)|\to0$ a.s.); sample moments converge a.s. to population moments; and Monte Carlo averages $\frac1n\sum g(U_j)$ converge a.s. to $\int_0^1g$ whenever $\int_0^1|g|\lt\infty$.
The Central Limit Theorem and Normal Approximations
The law of large numbers says $\bar X_n-\mu\to0$. The central limit theorem describes the size and shape of the error: it is of order $\sigma/\sqrt n$, and after dividing by that scale it is approximately standard normal, whatever the law of $X_1$, provided only that the variance is finite.
Let $X_1,X_2,\dots$ be iid with mean $\mu$ and variance $\sigma^2$, where $0\lt\sigma^2\lt\infty$. Then $$Z_n=\frac{S_n-n\mu}{\sigma\sqrt n}=\frac{\sqrt n\,(\bar X_n-\mu)}{\sigma}\xrightarrow{d}N(0,1),$$ that is, $P(Z_n\le z)\to\Phi(z)$ for every real $z$. Because $\Phi$ is continuous, the convergence is even uniform in $z$ (Pólya's theorem).
Proof. Step 1 (reduce). Let $Y_j=(X_j-\mu)/\sigma$: iid with mean $0$, variance $1$ and cf $\psi$. Then $Z_n=\frac{1}{\sqrt n}\sum_{j\le n}Y_j$ and, by Theorems 2.17(4) and 2.18 of §2.2, $\varphi_{Z_n}(t)=\psi\left(t/\sqrt n\right)^n$.
Step 2 (expand). By Theorem 2.19(2) with two moments, $\psi(s)=1-\frac{s^2}{2}+o(s^2)$ as $s\to0$. So for fixed $t$, $$\psi\!\left(\frac{t}{\sqrt n}\right)=1-\frac{t^2}{2n}+r_n,\qquad n\,r_n\to0.$$
Step 3 (compare products). For complex numbers with $|a_j|,|b_j|\le1$, $$\Big|\prod_{j=1}^na_j-\prod_{j=1}^nb_j\Big|\le\sum_{j=1}^n|a_j-b_j|,$$ by telescoping: $\prod a-\prod b=\sum_ja_1\cdots a_{j-1}(a_j-b_j)b_{j+1}\cdots b_n$. Take $a_j=\psi(t/\sqrt n)$ and $b_j=1-\frac{t^2}{2n}$, which lies in $[-1,1]$ once $n\ge t^2/4$. Then $$\left|\psi\!\left(\tfrac{t}{\sqrt n}\right)^n-\left(1-\tfrac{t^2}{2n}\right)^n\right|\le n|r_n|\to0.$$
Step 4 (conclude). $\left(1-\frac{t^2}{2n}\right)^n\to e^{-t^2/2}$, so $\varphi_{Z_n}(t)\to e^{-t^2/2}$, the continuous cf of $N(0,1)$. Lévy's continuity theorem gives $Z_n\xrightarrow{d}N(0,1)$. For uniformity: given $\varepsilon\gt0$ choose $z_0\lt z_1\lt\dots\lt z_m$ with $\Phi(z_0)\lt\varepsilon$, $1-\Phi(z_m)\lt\varepsilon$ and $\Phi(z_i)-\Phi(z_{i-1})\lt\varepsilon$; convergence at these finitely many points and monotonicity of cdfs give $\sup_z|P(Z_n\le z)-\Phi(z)|\lt2\varepsilon$ for large $n$. ∎
The proof used exactly the second moment: the expansion $\psi(s)=1-\frac{s^2}2+o(s^2)$ needs $E[Y^2]\lt\infty$ but no third moment. If the variance is infinite the conclusion can fail completely: for Cauchy data $Z_n$ is not even defined, and $\bar X_n$ stays Cauchy.
- (De Moivre–Laplace) If $X\sim\operatorname{Bin}(n,p)$ with $0\lt p\lt1$ fixed, then $\frac{X-np}{\sqrt{np(1-p)}}\xrightarrow{d}N(0,1)$. For integers $a\le b$ the continuity-corrected approximation is $$P(a\le X\le b)\approx\Phi\left(\frac{b+\frac12-np}{\sqrt{npq}}\right)-\Phi\left(\frac{a-\frac12-np}{\sqrt{npq}}\right).$$
- (Poisson) If $X_\lambda\sim\operatorname{Poi}(\lambda)$, then $\frac{X_\lambda-\lambda}{\sqrt\lambda}\xrightarrow{d}N(0,1)$ as $\lambda\to\infty$, with the same continuity correction.
- (Lyapunov and Lindeberg, statement) Let $X_k$ be independent with means $\mu_k$, finite variances $\sigma_k^2$ and $s_n^2=\sum_{k\le n}\sigma_k^2$. If for some $\delta\gt0$ $$\frac{1}{s_n^{2+\delta}}\sum_{k=1}^nE|X_k-\mu_k|^{2+\delta}\to0\qquad\text{(Lyapunov's condition)},$$ or more generally if for every $\varepsilon\gt0$ $$\frac{1}{s_n^2}\sum_{k\le n}E\left[(X_k-\mu_k)^2\mathbf{1}\{|X_k-\mu_k|\gt\varepsilon s_n\}\right]\to0$$ (Lindeberg's condition), then $\frac{S_n-E[S_n]}{s_n}\xrightarrow{d}N(0,1)$.
- (Berry–Esseen, statement) In Theorem 2.36, if $\rho=E|X_1-\mu|^3\lt\infty$, then $$\sup_z\left|P(Z_n\le z)-\Phi(z)\right|\le\frac{C\rho}{\sigma^3\sqrt n},$$ with an absolute constant $C$; $C=0.4748$ is known to suffice.
Proof. (1) $X$ has the law of a sum of $n$ iid $\operatorname{Ber}(p)$ variables with mean $p$ and variance $pq\gt0$; apply Theorem 2.36. (2) was proved with cfs in Example 3(b) of §2.2; alternatively, for integer $\lambda=m$, $X_m$ has the law of a sum of $m$ iid $\operatorname{Poi}(1)$ variables. (3) and (4) are stated without proof. ∎
Why the continuity correction? An integer-valued $X$ puts mass $P(X=k)$ at the single point $k$, while the normal curve spreads it over an interval. Representing $P(X=k)$ by the normal area over $[k-\frac12,k+\frac12]$ matches the two, so $P(X\le b)$ is approximated by the area up to $b+\frac12$ and $P(X\ge a)$ by the area from $a-\frac12$. The correction matters most when $\sqrt{npq}$ is small, and for a single value $P(X=k)$ it is indispensable (without it the answer would be $0$). By Berry–Esseen, the error of the uncorrected approximation for the binomial is at most $0.4748\,\frac{p^2+q^2}{\sqrt{npq}}$; a common rule of thumb is to use the normal approximation when $np\ge5$ and $n(1-p)\ge5$.
Slutsky's Theorem, Continuous Mapping and the Delta Method
In statistics the CLT is rarely used in its pure form. The standard deviation $\sigma$ is unknown and is replaced by an estimate, and one needs the limit law of a function of $\bar X_n$, such as $1/\bar X_n$ or $\log\bar X_n$, rather than of $\bar X_n$ itself. Three results handle this.
Proof. Almost surely: if $X_n(\omega)\to X(\omega)\in C$, then $g(X_n(\omega))\to g(X(\omega))$ by continuity at $X(\omega)$; this holds for almost every $\omega$. In probability: by Theorem 2.31(6), every subsequence of $(X_n)$ has a further subsequence converging a.s. to $X$; along it $g(X_n)\to g(X)$ a.s.; by the same criterion $g(X_n)\xrightarrow{P}g(X)$. In distribution, for continuous $g$: if $h$ is bounded and continuous, so is $h\circ g$, and Theorem 2.32(1) gives $E[h(g(X_n))]\to E[h(g(X))]$, so $g(X_n)\xrightarrow{d}g(X)$. (With exceptional points outside $C$ the proof uses Skorokhod's representation and the a.s. case.) ∎
Two facts are used repeatedly. (i) If $X_n\xrightarrow{d}X$, then $X_n=O_p(1)$: choose $K$ with $\pm K$ continuity points of $F_X$ and $P(|X|\gt K)\lt\varepsilon/2$; then $P(|X_n|\gt K)\lt\varepsilon$ for all large $n$, and enlarging $K$ takes care of the finitely many remaining $n$. (ii) $O_p(1)\cdot o_p(1)=o_p(1)$, because $$P(|X_nY_n|\gt\varepsilon)\le P(|X_n|\gt K)+P(|Y_n|\gt\varepsilon/K).$$
If $X_n\xrightarrow{d}X$ and $Y_n\xrightarrow{P}c$, where $c$ is a constant, then
- $X_n+Y_n\xrightarrow{d}X+c$;
- $Y_nX_n\xrightarrow{d}cX$;
- $X_n/Y_n\xrightarrow{d}X/c$, provided $c\ne0$.
Proof. (1) Let $x$ be a continuity point of the cdf of $X+c$, i.e. $x-c$ is a continuity point of $F_X$, and choose $\varepsilon\gt0$ with $x-c\pm\varepsilon$ also continuity points (all but countably many $\varepsilon$ qualify). Then $$P(X_n+Y_n\le x)\le P(X_n\le x-c+\varepsilon)+P(|Y_n-c|\gt\varepsilon)\to F_X(x-c+\varepsilon),$$ $$P(X_n+Y_n\le x)\ge P(X_n\le x-c-\varepsilon)-P(|Y_n-c|\gt\varepsilon)\to F_X(x-c-\varepsilon).$$ Letting $\varepsilon\downarrow0$ along such values, $P(X_n+Y_n\le x)\to F_X(x-c)=P(X+c\le x)$.
(2) Write $Y_nX_n=cX_n+(Y_n-c)X_n$. By the continuous mapping theorem, $cX_n\xrightarrow{d}cX$. By fact (i), $X_n=O_p(1)$, and $Y_n-c=o_p(1)$, so $(Y_n-c)X_n=o_p(1)$ by fact (ii). Part (1) (with the constant $0$) gives the claim.
(3) $g(y)=1/y$ is continuous at $c\ne0$, so $1/Y_n\xrightarrow{P}1/c$ by Theorem 2.38; apply (2). ∎
Slutsky's theorem fails if $Y_n$ converges only in distribution to a non-degenerate limit: with $Z\sim N(0,1)$, $X_n=Z$ and $Y_n=-Z$, both converge in distribution to $N(0,1)$, but $X_n+Y_n=0$. Convergence in distribution of each coordinate says nothing about the joint law.
Suppose $\sqrt n\,(T_n-\theta)\xrightarrow{d}N(0,\tau^2)$, and let $g$ be differentiable at $\theta$.
- (First order) $\sqrt n\left(g(T_n)-g(\theta)\right)\xrightarrow{d}N\left(0,\,g'(\theta)^2\tau^2\right)$. (If $g'(\theta)=0$ the limit is the constant $0$.)
- (Second order) If $g'(\theta)=0$ and $g''(\theta)$ exists, then $n\left(g(T_n)-g(\theta)\right)\xrightarrow{d}\tfrac12g''(\theta)\,\tau^2\,\chi^2_1$.
(Multivariate form, statement: if $\sqrt n(\mathbf{T}_n-\mathbf{t})\xrightarrow{d}N_k(\mathbf 0,\Sigma)$ for a vector $\mathbf{t}\in\mathbb{R}^k$ and $g:\mathbb{R}^k\to\mathbb{R}$ is differentiable at $\mathbf{t}$, then $\sqrt n(g(\mathbf{T}_n)-g(\mathbf{t}))\xrightarrow{d}N(0,\nabla g^{\mathsf T}\Sigma\nabla g)$, the gradient taken at $\mathbf{t}$. It gives the asymptotic variances of ratio estimators in §8.2 and of the sample correlation coefficient.)
Proof. (1) First, $T_n-\theta=\frac{1}{\sqrt n}\cdot\sqrt n(T_n-\theta)=o(1)\cdot O_p(1)\xrightarrow{P}0$. Define $h(t)=\frac{g(t)-g(\theta)}{t-\theta}$ for $t\ne\theta$ and $h(\theta)=g'(\theta)$. Differentiability of $g$ at $\theta$ means exactly that $h$ is continuous at $\theta$, so $h(T_n)\xrightarrow{P}g'(\theta)$ by Theorem 2.38. Now $$\sqrt n\left(g(T_n)-g(\theta)\right)=h(T_n)\cdot\sqrt n(T_n-\theta),$$ and Slutsky's theorem (2) gives the limit $g'(\theta)\,N(0,\tau^2)=N(0,g'(\theta)^2\tau^2)$.
(2) By Taylor's formula with Peano remainder, $g(t)-g(\theta)=\left(\tfrac12g''(\theta)+k(t)\right)(t-\theta)^2$ with $k(t)\to0$ as $t\to\theta$ (and $k(\theta)=0$). Hence $n(g(T_n)-g(\theta))=\left(\tfrac12g''(\theta)+k(T_n)\right)\left(\sqrt n(T_n-\theta)\right)^2$. The first factor tends in probability to $\frac12g''(\theta)$, and by the continuous mapping theorem the second tends in distribution to $\tau^2Z^2$, $Z\sim N(0,1)$, i.e. to $\tau^2\chi^2_1$. Slutsky's theorem finishes the proof. ∎
- In probability does not imply almost surely. The typewriter sequence converges in probability and in every $L^r$ but at no point $\omega$; independent $\operatorname{Ber}(1/n)$ variables converge in probability but have infinitely many $1$s a.s. (Example 1). A subsequence always converges a.s.
- Almost surely does not imply convergence of means. $n\mathbf{1}\{U\lt1/n\}\to0$ everywhere, yet its mean is $1$. Convergence of $E[X_n]$ needs domination or uniform integrability (Theorem 2.31(7)).
- In distribution does not imply in probability ($X_n=-X$ with $X$ symmetric), except when the limit is a constant (Theorem 2.31(5)).
- Convergence in distribution is a property of the laws only. $X_n\xrightarrow{d}X$ and $Y_n\xrightarrow{d}Y$ do not give $X_n+Y_n\xrightarrow{d}X+Y$ or $X_nY_n\xrightarrow{d}XY$ (Example 6(c)); Slutsky needs one of the limits to be a constant.
- Cdfs need not converge at jumps of the limit. $X_n\sim N(0,\tfrac1n)$ has $F_n(0)=\tfrac12\not\to1$ (Example 2(c)). Nor do $P(X_n\in B)$ converge for all sets $B$: $X_n$ uniform on $\{\frac1n,\frac2n,\dots,1\}$ converges in law to $U(0,1)$, but $P(X_n\in\mathbb{Q})=1$ while $P(U\in\mathbb{Q})=0$.
- Convergence in distribution does not give convergence of moments. $X_n=n$ with probability $\frac1n$ and $0$ otherwise: $X_n\xrightarrow{d}0$, but $E[X_n]=1$ and $E[X_n^2]=n\to\infty$.
- No mean, no law of large numbers. For Cauchy data $\bar X_n\sim C(0,1)$ for every $n$; by Theorem 2.35(2) the SLLN holds for iid data iff $E|X_1|\lt\infty$. (The weak law is subtler: it can hold without a mean, e.g. for the symmetric law $P(X=\pm k)\propto\frac{1}{k^2\log k}$ mentioned in §2.2, while the strong law fails there.)
- Uncorrelated with growing variances can fail the weak law (Example 3(a) with $\alpha\ge\frac12$), and a common random component defeats it altogether (Example 3(b)).
- The CLT needs a finite variance, and it is a statement about the $\sqrt n$-scale. It does not say that $S_n$ is approximately normal far in the tails: for $\operatorname{Bin}(100,\frac12)$, $P(X\ge80)\approx5.6\times10^{-10}$, while the corrected normal value is $1.8\times10^{-9}$, more than three times too large.
- Identically distributed matters in Lindeberg–Lévy. For independent $X_k=\pm2^k$ (probability $\frac12$ each), the last term carries three quarters of the variance and $S_n/s_n$ is not asymptotically normal; Lindeberg's condition fails.
- Continuity correction only for integer-valued variables. Do not apply it to sums of continuous variables (Example 5(c)).
- The delta method with $g'(\theta)=0$ gives the useless limit $0$ at the $\sqrt n$ scale; use the second-order version at scale $n$ (Example 7(c)).
Exercises
- 1 $U\sim U(0,1)$ and $X_n=U^n$. Show that $X_n\to0$ almost surely and in $L^r$ for every $r\gt0$.
- 2 The $X_n$ are independent with $P(X_n=n^a)=n^{-b}$ and $P(X_n=0)=1-n^{-b}$, where $a,b\gt0$. For which $(a,b)$ does $X_n\to0$ (i) in probability, (ii) almost surely, (iii) in $L^1$, (iv) in $L^2$?
- 3 Prove that limits in probability are a.s. unique: if $X_n\xrightarrow{P}X$ and $X_n\xrightarrow{P}Y$, then $P(X=Y)=1$.
- 4 Show that $X_n\xrightarrow{P}X$ and $Y_n\xrightarrow{P}Y$ imply $X_n+Y_n\xrightarrow{P}X+Y$. Give an example with $X_n\xrightarrow{d}X$ and $Y_n\xrightarrow{d}Y$ where $X_n+Y_n$ does not converge in distribution to $X+Y$.
- 5 Show that if $X_n\xrightarrow{P}X$ and $Y_n\xrightarrow{P}Y$, then $X_nY_n\xrightarrow{P}XY$ (use Theorem 2.31(6)).
- 6 $X_n\sim N(\mu_n,\sigma_n^2)$ with $\mu_n\to\mu$ and $\sigma_n\to\sigma\gt0$. Show that $X_n\xrightarrow{d}N(\mu,\sigma^2)$. What is the limit if $\sigma_n\to0$?
- 7 $X_n$ is uniform on $\{\frac1n,\frac2n,\dots,\frac nn\}$. Show that $X_n\xrightarrow{d}U(0,1)$, and that $P(X_n\in B)\to P(U\in B)$ fails for $B=\mathbb{Q}$. Why does this not contradict Theorem 2.32(4)?
- 8 Prove that $X_n\xrightarrow{P}X$ if and only if $E\left[\min(|X_n-X|,1)\right]\to0$.
- 1 $U_1,U_2,\dots$ are iid $U(0,1)$. Find the almost-sure limits of $\dfrac{U_1^2+\dots+U_n^2}{U_1+\dots+U_n}$ and of $\dfrac{1}{n}\sum_{j=1}^n\mathbf{1}\{U_j\le U_j^{\,2}+\tfrac14\}$.
- 2 Show that $\frac1n\sum_{j=1}^n(X_j-\bar X_n)^2\xrightarrow{P}\sigma^2$ for iid $X_j$ with finite variance, using only Khintchine's weak law and the continuous mapping theorem.
- 3 (Monte Carlo) $\frac\pi4=\int_0^1\sqrt{1-x^2}\,dx$ is estimated by $\frac1n\sum\sqrt{1-U_j^2}$. Find the variance of one term, and the $n$ needed for an error of at most $0.01$ with probability at least $0.95$ according to Chebyshev's inequality and according to the CLT.
- 4 The $X_k$ are independent with $X_k\sim U(-k^\beta,k^\beta)$. Does $\bar X_n\xrightarrow{P}0$ when (a) $\beta=\frac14$, (b) $\beta=\frac12$?
- 5 Let $X_j$ be iid with $E|X_1|\lt\infty$. Show that $X_n/n\to0$ a.s. and deduce that $\frac1n\max_{j\le n}|X_j|\to0$ a.s.
- 6 The $X_j$ are iid with density $x^{-2}$ on $(1,\infty)$. Show that $\bar X_n\to\infty$ almost surely. (Hint: truncate at a level $M$.)
- 7 (Borel's normal numbers) Let $U\sim U(0,1)$ have decimal expansion $0.D_1D_2D_3\dots$. Show that the $D_j$ are iid uniform on $\{0,\dots,9\}$ and that, almost surely, each digit appears with limiting frequency $\frac1{10}$.
- 8 (Weierstrass via the weak law) Let $f$ be continuous on $[0,1]$ and $B_n(x)=\sum_{k=0}^nf\left(\frac kn\right)\binom nkx^k(1-x)^{n-k}$, the Bernstein polynomial. Prove that $\sup_{0\le x\le1}|B_n(x)-f(x)|\to0$.
- 1 A fair coin is tossed $400$ times. Approximate the probability that the number of heads is between $190$ and $215$ inclusive.
- 2 $X\sim\operatorname{Poi}(100)$. Approximate $P(X\le85)$ with continuity correction, and compare with the exact value $0.0708$.
- 3 (Airline overbooking) A plane has $200$ seats and each ticket holder turns up independently with probability $0.9$. Using the normal approximation with continuity correction, find the largest number of tickets that can be sold with $P(\text{more than }200\text{ turn up})\le0.05$. The exact binomial answer is $215$; explain the discrepancy.
- 4 Let $Z=U_1+\dots+U_{12}-6$ with $U_j$ iid $U(0,1)$ (an old method for simulating normals). Find $E[Z]$, $\operatorname{Var}(Z)$ and the normal approximation to $P(|Z|\gt2)$; the exact value is $0.04455$. Which feature of the normal law can $Z$ never reproduce?
- 5 Let $Z_n=\sqrt n(\bar X_n-\mu)/\sigma$ for iid data with finite variance. Show that $Z_n$ does not converge in probability, by finding the limit law of $Z_{2n}-Z_n$.
- 6 (Lift safety) Passenger weights are iid with mean $70$ kg and standard deviation $15$ kg. A lift's load limit is $1000$ kg. What is the largest number of passengers for which the approximate probability of overload is at most $0.05$?
- 7 (Records) Let $I_k$ be independent $\operatorname{Ber}(p_k)$ with $\sum p_k(1-p_k)=\infty$. Verify Lyapunov's condition with $\delta=1$, and deduce that the number $R_n$ of records among $n$ iid continuous observations (where $p_k=1/k$) satisfies $\frac{R_n-\log n}{\sqrt{\log n}}\xrightarrow{d}N(0,1)$. (You may use $\sum_{k\le n}\frac1k=\log n+O(1)$.)
- 8 Use the CLT for Poisson variables to prove that $e^{-n}\sum_{k=0}^n\frac{n^k}{k!}\to\frac12$.
Choose $n$, $p$ and $k$. The lab computes $P(X\le k)$ for $X\sim\operatorname{Bin}(n,p)$ exactly and by the normal approximation without and with the continuity correction $k+\frac12$. Try small $n$ or $p$ near $0$ to see when the correction matters and when skewness spoils both.
- 1 $X_j$ iid $U(0,\theta)$. Find the limit laws of $\sqrt n(2\bar X_n-\theta)$ and of $n(\theta-M_n)$ with $M_n=\max_{j\le n}X_j$. Which estimator of $\theta$ converges faster?
- 2 For Bernoulli trials, find the limit law of the estimated log-odds $\sqrt n\left(\log\frac{\hat p}{1-\hat p}-\log\frac{p}{1-p}\right)$.
- 3 $X_j$ iid with mean $\mu$ and variance $\sigma^2$. Find the limit law of $\sqrt n(\bar X_n^2-\mu^2)$ when $\mu\ne0$, and of $n\bar X_n^2$ when $\mu=0$.
- 4 Show that if $E[X_1^4]\lt\infty$, then $\sqrt n(S_n^2-\sigma^2)\xrightarrow{d}N(0,\mu_4-\sigma^4)$, where $\mu_4=E[(X_1-\mu)^4]$. Evaluate the limit variance for normal data.
- 5 $X_j$ iid exponential with mean $\theta$. Find a function $g$ for which the limit variance of $\sqrt n(g(\bar X_n)-g(\theta))$ does not depend on $\theta$, and use it to write an approximate $95\%$ interval for $\theta$.
- 6 Prove or disprove: (a) if $X_n\xrightarrow{d}X$ and $X_n-Y_n\xrightarrow{P}0$, then $Y_n\xrightarrow{d}X$; (b) if $X_n\xrightarrow{d}X$, then $X_n-X\xrightarrow{P}0$; (c) if $X_n\xrightarrow{d}X$ and $Y_n\xrightarrow{P}c$, then $(X_n+Y_n)^2\xrightarrow{d}(X+c)^2$.
- 7 (Clinical trial) In $n=400$ patients, $x=88$ have a side effect. Give the Wald $95\%$ interval for $p$ and the delta-method $95\%$ interval for the log-odds, transformed back to $p$.
- 8 $X_j$ iid with mean $0$ and variance $\sigma^2$. Find the limit law of $n\left(1-\cos\bar X_n\right)$.
E1. CSIR Part B GATE ST $X_1,X_2,\dots$ are iid exponential with mean $2$. With probability one, $\dfrac{X_1^2+\dots+X_n^2}{X_1+\dots+X_n}$ converges to (A) $2$; (B) $4$; (C) $8$; (D) it does not converge.
E2. CSIR Part B $X\sim\operatorname{Bin}(100,\tfrac12)$. The normal approximation with continuity correction to $P(X=50)$ is closest to (A) $0$; (B) $0.040$; (C) $0.080$; (D) $0.159$.
E3. CSIR Part B GATE ST $X_1,X_2,\dots$ are iid exponential with rate $\lambda$. Then $\sqrt n\left(\log\bar X_n+\log\lambda\right)$ converges in distribution to $N(0,v)$ with $v$ equal to (A) $1$; (B) $\lambda^2$; (C) $1/\lambda^2$; (D) $\lambda$.
E4. CSIR Part C Let $X_1,X_2,\dots$ be independent with $P(X_n=1)=\frac1n$ and $P(X_n=0)=1-\frac1n$. Which of the following are true? (One or more options may be correct.) (A) $X_n\to0$ in probability. (B) $X_n\to0$ almost surely. (C) $X_n\to0$ in mean square. (D) $\sum_{n\ge1}X_n\lt\infty$ almost surely.
E5. CSIR Part C Let $X_1,X_2,\dots$ be iid and $\bar X_n=\frac1n\sum_{j\le n}X_j$. Which of the following are true? (A) If $E[X_1^2]\lt\infty$, then $\bar X_n\to E[X_1]$ almost surely. (B) If $X_1$ is standard Cauchy, then $\bar X_n\to0$ in probability. (C) If $X_1\sim U(0,1)$, then $\frac1n\sum_{j\le n}\log X_j\to-1$ almost surely. (D) If $E|X_1|=\infty$, then $\bar X_n$ cannot converge almost surely to a finite constant.
E6. CSIR Part C GATE ST Let $X_1,X_2,\dots$ be iid with mean $0$ and variance $1$, and $S_n=X_1+\dots+X_n$. Which of the following are true? (A) $S_n/\sqrt n\xrightarrow{d}N(0,1)$. (B) $S_n/n\to0$ almost surely. (C) $S_n^2/n\xrightarrow{d}\chi^2_1$. (D) $S_n/\sqrt n$ converges in probability.
Suggested plan: 3 lectures × 60 min.
- Modes of convergence. Definition 2.11 with pictures of sample paths. Theorem 2.30 (prove (1) and (2) via Borel–Cantelli). Theorem 2.31 (1), (2), (4), (5), (6) on the board; (3) and (7) briefly. The implication table, then Example 1, ideally with Interactive 1 projected: typewriter versus $\operatorname{Ber}(1/n)$ versus $\operatorname{Ber}(1/n^2)$ versus $n\mathbf{1}\{U\lt1/n\}$. Theorem 2.32 (prove (1) "if" and (3)), Example 2.
- Laws of large numbers and the CLT. Theorem 2.33 (one line), Theorem 2.34 via cfs, Theorem 2.35 with the fourth-moment proof and the converse. Example 3 as a class discussion, Example 4(b). Then Theorem 2.36: do the four-step proof in full, emphasising where the second moment enters. Interactive 2 with the exponential and Bernoulli(0.1) parents, then the Cauchy parent.
- Using the limit theorems. Theorem 2.37 and the continuity correction with the check lab; Example 5. Theorem 2.38, Definition 2.12, Slutsky (prove (1) and (2)), Example 6(a) (why the $t$-statistic is asymptotically normal), the delta method and Example 7. Finish with E4–E6.
Discussion questions. (1) An insurer with a million independent policies relies on the law of large numbers. Which assumption fails for flood insurance in one river valley, and what does Example 3(b) say about it? (2) Why can convergence in probability be described by a metric (Exercise I.8) but almost-sure convergence cannot? (3) The CLT is often quoted as "$n\ge30$ is enough". Use Interactive 2 and the Berry–Esseen bound to argue for or against this rule for lognormal data.
Misconceptions to watch for. Treating convergence in distribution as a statement about the random variables rather than their laws; expecting $E[X_n]\to E[X]$ from a.s. convergence; forgetting that Slutsky needs a constant limit; applying the CLT to Cauchy or other infinite-variance data; continuity-correcting sums of continuous variables; reading "$\bar X_n\approx N(\mu,\sigma^2/n)$" as convergence of $\bar X_n$ to a normal law (it converges to the constant $\mu$); using the delta method blindly when $g'(\theta)=0$.
Suggested homework. I(2, 4, 7, 8), II(1, 3, 6, 8), III(1, 3, 5, 8), IV(1, 3, 4, 7); Exam corner E2, E4, E6.
Research question. Is "$n\ge30$" enough for the central limit theorem? For realistic skewed data, such as hospital lengths of stay, household incomes or insurance claim sizes, how large must $n$ be before the nominal $95\%$ interval $\bar x\pm1.96\,s/\sqrt n$ really covers the mean $95\%$ of the time?
Why it matters. Health-economics studies report mean costs per patient, and regulators approve budgets based on them; insurers price policies from mean claim sizes. Cost and claim data are strongly right-skewed (close to lognormal or gamma), and for such data the CLT can be slow. An interval that covers only $88\%$ of the time while claiming $95\%$ misleads decision makers.
What to do.
- Choose parent laws: exponential, gamma with shape $0.5$, lognormal with $\sigma=1$ and with $\sigma=1.5$. For each, record the skewness $\gamma_1$.
- For $n\in\{10,30,100,300,1000\}$, simulate $10\,000$ samples with a seeded generator, compute the $t$-interval for each and record its coverage of the true mean, and separately the proportion of misses on each side.
- Plot coverage against $n$, and compare the miss rates with the Berry–Esseen bound $0.4748\rho/(\sigma^3\sqrt n)$ and with the heuristic error $\gamma_1/(6\sqrt n)$ suggested by an Edgeworth expansion.
- Propose and test one remedy: a log transformation followed by the delta method, or a bootstrap percentile interval.
Tools: Python (numpy, scipy.stats) or R; Interactive 2 for intuition about skewed parents.
Hand in: A coverage table and plot for each parent law, a paragraph replacing the "$n\ge30$" rule by one that depends on skewness, and the results of your remedy.
Going further. Read about Edgeworth expansions, which correct the normal approximation by a term proportional to $\gamma_1/\sqrt n$, and about the bootstrap, which estimates the sampling distribution of a statistic by resampling the data. Both are standard tools in modern applied statistics.
🧪 Self-check quiz
Q1. Which implication holds for every sequence of random variables?
Q2. The typewriter sequence of indicators of dyadic intervals sweeping across $(0,1)$ converges to $0$
Q3. A fair coin is tossed $400$ times. The normal approximation with continuity correction to $P(\text{at most }210\text{ heads})$ is closest to
Q4. $X_n\xrightarrow{d}N(0,1)$ and $Y_n\to2$ in probability. Then $Y_nX_n+1$ converges in distribution to
Q5. $X_j$ iid with mean $\mu\ne0$ and variance $\sigma^2$. The limit law of $\sqrt n(\bar X_n^2-\mu^2)$ is