Chapter 5 · Testing of Hypotheses

5.2 Large-Sample Tests and Chi-Square Tests

Tests that work without exact distributions: large-sample $z$-tests for means, proportions and rates, the Wald, score and likelihood ratio trinity, Pearson's chi-square test of goodness of fit with the degrees-of-freedom rule for estimated parameters, contingency tables for independence and homogeneity, the $2\times2$ table with Yates' correction and Fisher's exact test, odds ratios and McNemar's test.

The exact tests of §5.1 need exact sampling distributions, and those exist mainly for normal samples. Much real data is not normal and much of it is not even numerical. An opinion poll records "yes" or "no"; a vaccine trial records infected or not; a geneticist counts offspring of four phenotypes; a hospital cross-classifies patients by treatment and outcome; a quality engineer counts flaws per sheet. For such data two ideas carry almost all the weight. The first is the large-sample $z$-test: by the central limit theorem and Slutsky's theorem (§2.3), almost any reasonable estimator is approximately normal, so (estimate − hypothesised value)/(standard error) is approximately $N(0,1)$ when $H_0$ holds. The second is Karl Pearson's chi-square statistic $X^2=\sum(O-E)^2/E$, which compares observed counts $O$ with the counts $E$ expected under $H_0$ and has an approximate $\chi^2$ law whose degrees of freedom are found by counting parameters, as Wilks' theorem (§5.1) predicts.

This section develops both ideas with proofs. We show why the standard $z$-tests have the right size in large samples and are consistent, and how the Wald, score and likelihood ratio versions are related. We prove Pearson's theorem through the multivariate CLT, explain why each estimated parameter costs one degree of freedom, and derive the tests of independence and homogeneity in contingency tables, with the special identities of the $2\times2$ table. For small samples we give Fisher's exact test, and for paired binary data McNemar's test. CSIR NET Part B questions ask for the value of $X^2$, its degrees of freedom or a $z$ statistic. Part C questions test the assumptions: when the $\chi^2$ approximation is reliable, how estimated parameters change the degrees of freedom, and which statements about $2\times2$ tables are identities. Notation follows §5.1. Upper points are $z_\alpha$ and $\chi^2_{\nu,\alpha}$; for reference, $\chi^2_{\nu,0.05}=3.841,\ 5.991,\ 7.815,\ 9.488,\ 11.070$ for $\nu=1,\dots,5$.

Large-Sample Tests for Means and Proportions

📘 Definition 5.9 — Asymptotic size and consistency

Let $\phi_n$ be a test based on a sample of size $n$, for $H_0:\theta\in\Theta_0$ against $H_1:\theta\in\Theta_1$.

  1. $\{\phi_n\}$ has asymptotic size $\alpha$ if $E_\theta\phi_n\to\alpha$ for every $\theta$ on the boundary of $\Theta_0$ that determines the size (for a simple null, at $\theta_0$), and $\limsup_nE_\theta\phi_n\le\alpha$ for every $\theta\in\Theta_0$.
  2. $\{\phi_n\}$ is consistent if $E_\theta\phi_n\to1$ for every $\theta\in\Theta_1$.
📐 Theorem 5.11 — The large-sample $z$-test

Let $T_n$ be a statistic and $\tau=g(\theta)$ a real parameter. Suppose that for every $\theta$, $$\sqrt n\left(T_n-g(\theta)\right)\xrightarrow dN\left(0,v(\theta)\right)$$ with $v(\theta)\gt0$, and that $\hat v_n$ is a statistic with $\hat v_n\xrightarrow Pv(\theta)$ (or at least $\hat v_n\xrightarrow{P}v(\theta_0)$ when $g(\theta)=\tau_0$). Put $Z_n=\sqrt n(T_n-\tau_0)/\sqrt{\hat v_n}$.

  1. If $g(\theta)=\tau_0$, then $Z_n\xrightarrow dN(0,1)$. Hence the tests "reject $H_0:\tau=\tau_0$ when $|Z_n|\gt z_{\alpha/2}$", "when $Z_n\gt z_\alpha$" and "when $Z_n\lt-z_\alpha$" have asymptotic size $\alpha$ (the one-sided ones also for $H_0:\tau\le\tau_0$, respectively $\tau\ge\tau_0$).
  2. If $g(\theta)\ne\tau_0$ and $\hat v_n$ converges in probability to a finite positive limit, the two-sided test is consistent, and so is each one-sided test against alternatives on its own side.
  3. (Local power.) If $\theta_n$ satisfies $\sqrt n(g(\theta_n)-\tau_0)\to\delta$ and the convergence in (1) holds along $\theta_n$, then the power of the right-sided test tends to $1-\Phi\left(z_\alpha-\delta/\sqrt{v(\theta_0)}\right)$.

Proof. (1) Write $Z_n=\dfrac{\sqrt n(T_n-\tau_0)}{\sqrt{v(\theta)}}\cdot\sqrt{\dfrac{v(\theta)}{\hat v_n}}$. The first factor converges in distribution to $N(0,1)$ and the second in probability to $1$ (continuous mapping), so Slutsky's theorem (§2.3) gives $Z_n\xrightarrow dN(0,1)$. Since $N(0,1)$ is continuous, $P(|Z_n|\gt z_{\alpha/2})\to\alpha$, and similarly for the one-sided tests. For $H_0:\tau\le\tau_0$ and $\theta$ with $g(\theta)\lt\tau_0$, $$Z_n=\frac{\sqrt n(T_n-g(\theta))}{\sqrt{\hat v_n}}+\frac{\sqrt n(g(\theta)-\tau_0)}{\sqrt{\hat v_n}},$$ where the first term is bounded in probability and the second tends to $-\infty$ in probability, so $P(Z_n\gt z_\alpha)\to0$.

(2) The same decomposition with $g(\theta)\ne\tau_0$ shows that $|Z_n|\to\infty$ in probability, with the sign of $g(\theta)-\tau_0$; so $P(|Z_n|\gt z_{\alpha/2})\to1$.

(3) $$Z_n=\frac{\sqrt n(T_n-g(\theta_n))}{\sqrt{\hat v_n}}+\frac{\sqrt n(g(\theta_n)-\tau_0)}{\sqrt{\hat v_n}},$$ where the first term tends to $N(0,1)$ and the second to $\delta/\sqrt{v(\theta_0)}$ in probability. So $Z_n\xrightarrow dN\left(\delta/\sqrt{v(\theta_0)},1\right)$ and $P(Z_n\gt z_\alpha)\to1-\Phi(z_\alpha-\delta/\sqrt{v(\theta_0)})$. ∎

Part (3) gives the sample size formula. To have power $1-\beta$ at the alternative $\tau_1$ with a one-sided level-$\alpha$ test, put $\delta=\sqrt n(\tau_1-\tau_0)$ and solve $z_\alpha-\delta/\sqrt v=-z_\beta$: $$n\approx\left(\frac{(z_\alpha+z_\beta)\sqrt v}{\tau_1-\tau_0}\right)^2.$$ For proportions it is more accurate to use the null variance in the $z_\alpha$ term and the alternative variance in the $z_\beta$ term (Exercise I.7).

📐 Theorem 5.12 — The standard large-sample tests

Each statistic below is asymptotically $N(0,1)$ under the stated $H_0$, so it is used with the critical values $z_\alpha$ or $z_{\alpha/2}$. All samples are independent and iid within themselves.

Problem$H_0$Statistic
one mean, any population with finite variance$\mu=\mu_0$$Z=\dfrac{\bar X-\mu_0}{S/\sqrt n}$
two means$\mu_1-\mu_2=\Delta_0$$Z=\dfrac{\bar X-\bar Y-\Delta_0}{\sqrt{S_1^2/m+S_2^2/n}}$
one proportion, $X\sim\operatorname{Bin}(n,p)$$p=p_0$$Z=\dfrac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}$
two proportions$p_1=p_2$$Z=\dfrac{\hat p_1-\hat p_2}{\sqrt{\hat p(1-\hat p)\left(\frac1m+\frac1n\right)}}$, $\ \hat p=\dfrac{X_1+X_2}{m+n}$
Poisson mean$\lambda=\lambda_0$$Z=\dfrac{\bar X-\lambda_0}{\sqrt{\lambda_0/n}}$
standard deviation (normal population)$\sigma=\sigma_0$$Z=\dfrac{S-\sigma_0}{\sigma_0/\sqrt{2n}}$
correlation (bivariate normal)$\rho=\rho_0$$Z=\sqrt{n-3}\left(\operatorname{artanh}r-\operatorname{artanh}\rho_0\right)$

Proof. Each case is Theorem 5.11 with a specific $T_n$ and $\hat v_n$. One mean: CLT for $\bar X$ with $v=\sigma^2$, and $S^2\xrightarrow P\sigma^2$ by the weak law. Two means: $\bar X-\bar Y$ is a difference of independent asymptotically normal means, with variance $\sigma_1^2/m+\sigma_2^2/n$ estimated consistently; let $m,n\to\infty$ with $m/(m+n)\to\lambda\in(0,1)$. One proportion: $\hat p=X/n$ is a mean of Bernoulli variables, $v=p_0(1-p_0)$ is known under $H_0$, and no estimate is needed. Two proportions: under $H_0$, $p_1=p_2=p$ and the pooled $\hat p$ is consistent for $p$, so $\hat p(1-\hat p)(1/m+1/n)$ estimates the variance of $\hat p_1-\hat p_2$ consistently (in the sense of the ratio tending to $1$). Poisson: $\operatorname{Var}(X_i)=\lambda_0$ under $H_0$. Standard deviation: for normal samples $\sqrt n(S^2-\sigma^2)\xrightarrow dN(0,2\sigma^4)$ (§4.1), and the delta method with $g(t)=\sqrt t$, $g'(\sigma^2)=\frac1{2\sigma}$ gives $\sqrt n(S-\sigma)\xrightarrow dN(0,\sigma^2/2)$. Correlation: for bivariate normal samples $\sqrt n(r-\rho)\xrightarrow dN(0,(1-\rho^2)^2)$, and $\frac{d}{d\rho}\operatorname{artanh}\rho=\frac1{1-\rho^2}$, so Fisher's transformation $\operatorname{artanh}r=\frac12\log\frac{1+r}{1-r}$ is variance stabilising (§2.3), with variance about $1/n$; the refinement $1/(n-3)$ improves small samples. ∎

The one-proportion test uses the null standard error $\sqrt{p_0q_0/n}$ ($q_0=1-p_0$). This is the score test; inverting it gives the Wilson interval of §4.3. Replacing $p_0$ by $\hat p$ in the denominator gives the Wald test, which inverts to the Wald interval. Both are covered by the next theorem.

📐 Theorem 5.13 — The Wald, score and likelihood ratio tests

Let $X_1,\dots,X_n$ be iid from a regular one-parameter family (§4.2) with Fisher information $I(\theta)$, score $U_n(\theta)=\ell_n'(\theta)$ and MLE $\hat\theta_n$. For $H_0:\theta=\theta_0$ define $$W_n=nI(\hat\theta_n)(\hat\theta_n-\theta_0)^2,\qquad S_n=\frac{U_n(\theta_0)^2}{nI(\theta_0)},\qquad\Lambda_n^*=-2\log\Lambda_n.$$ Under $H_0$ all three converge in distribution to $\chi^2_1$, and the differences between any two of them tend to $0$ in probability.

Proof. Under $H_0$, $U_n(\theta_0)/\sqrt{nI(\theta_0)}\xrightarrow dN(0,1)$ by the CLT, since the score has mean $0$ and variance $I(\theta_0)$ per observation (§4.2); so $S_n\xrightarrow d\chi^2_1$. Expanding $0=U_n(\hat\theta_n)=U_n(\theta_0)+\ell_n''(\theta_n^*)(\hat\theta_n-\theta_0)$ gives $$\sqrt n(\hat\theta_n-\theta_0)=\frac{U_n(\theta_0)/\sqrt n}{-\ell_n''(\theta_n^*)/n},$$ which equals $\dfrac{U_n(\theta_0)}{\sqrt n\,I(\theta_0)}+o_P(1)$ because $-\ell_n''(\theta_n^*)/n\xrightarrow PI(\theta_0)$. Squaring and multiplying by $I(\hat\theta_n)\xrightarrow PI(\theta_0)$ shows $W_n=S_n+o_P(1)$. Finally the proof of Wilks' theorem (§5.1) gave $$\Lambda_n^*=\left(-\tfrac1n\ell_n''(\theta_n^{**})\right)n(\hat\theta_n-\theta_0)^2,$$ which is $W_n+o_P(1)$. ∎

For a binomial proportion, $S_n$ is the square of the score $z$ of Theorem 5.12 and $W_n$ the square of the Wald $z$. The score test needs only the null value, the Wald test only the MLE, and the likelihood ratio test both. In small samples they can disagree, and the score and likelihood ratio versions are usually more reliable.

✏️ Example 1 — An opinion poll: score, Wald, likelihood ratio and exact tests
In a random sample of $900$ voters, $477$ support a proposal. (a) Test $H_0:p=0.5$ against $H_1:p\gt0.5$ at $5\%$ with the score $z$-test; find the $p$-value with and without a continuity correction, and compare with the exact binomial $p$-value. (b) Compute the two-sided statistics $W_n$, $S_n$ and $-2\log\Lambda$.
(a) Score test $\hat p=0.53$ and $\sqrt{p_0q_0/n}=\sqrt{0.25/900}=0.016667$, so $z=0.03/0.016667=1.80\gt z_{0.05}=1.645$: reject $H_0$. The $p$-value is $1-\Phi(1.80)=0.0359$.
Continuity correction $P(X\ge477)$ is approximated better by $P(Y\ge476.5)$ for normal $Y$ (§2.3): $z=\frac{476.5-450}{15}=1.767$, $p=0.0386$. The exact binomial value $P_{0.5}(X\ge477)=0.0386$ agrees to three decimals.
(b) Wald: $z_W=\frac{0.03}{\sqrt{0.53\times0.47/900}}=1.803$, so $W_n=3.252$. Score: $S_n=1.80^2=3.240$. Likelihood ratio: $$-2\log\Lambda=2\left[477\log\tfrac{477}{450}+423\log\tfrac{423}{450}\right]=3.242.$$ All three are below $\chi^2_{1,0.05}=3.841$ (two-sided $p\approx0.072$). The two-sided test does not reject although the one-sided test does: the choice of alternative must be made before seeing the data.
✏️ Example 2 — Two proportions: a vaccine trial
In a trial, $60$ of $1500$ participants given a placebo and $24$ of $1500$ given a vaccine became infected. Test whether the vaccine lowers the infection rate, and estimate its efficacy $1-p_V/p_P$.
Set-up $H_0:p_P=p_V$ against $H_1:p_P\gt p_V$. $\hat p_P=0.04$, $\hat p_V=0.016$, pooled $\hat p=84/3000=0.028$.
Statistic $\operatorname{SE}_0=\sqrt{0.028\times0.972\times\frac{2}{1500}}=0.006024$ and $z=\frac{0.024}{0.006024}=3.98$. The one-sided $p$-value is $1-\Phi(3.98)=3.4\times10^{-5}$: very strong evidence that the vaccine helps.
Efficacy $1-0.016/0.04=0.60$, i.e. an estimated $60\%$ reduction in risk.
Check With the unpooled standard error $\sqrt{\hat p_P\hat q_P/1500+\hat p_V\hat q_V/1500}$, $z=3.99$. Under $H_0$ both versions are valid; the pooled one is the square root of Pearson's $X^2$ for the $2\times2$ table (Theorem 5.17).
✏️ Example 3 — Two means without normality
Commuting times (minutes) were recorded for $m=120$ residents of district A ($\bar x=34.2$, $s_1=9.5$) and $n=150$ residents of district B ($\bar y=31.8$, $s_2=11.2$). Test $H_0:\mu_A=\mu_B$ against $\mu_A\ne\mu_B$ at $5\%$. The distributions are skewed, and the variances differ.
Statistic $\operatorname{SE}=\sqrt{\frac{9.5^2}{120}+\frac{11.2^2}{150}}=\sqrt{0.7521+0.8363}=1.2603$ and $z=\frac{2.4}{1.2603}=1.904$.
Decision $|z|\lt1.96$: do not reject at $5\%$ (two-sided $p=0.057$). A one-sided test planned in advance would reject.
Why it is valid Theorem 5.11: the CLT applies to each mean, and $S_1^2,S_2^2$ are consistent. Neither normality nor equal variances is needed, only finite variances and large samples. The pooled $t$-test of §5.1 would be wrong here if the variances differ and the sample sizes are unequal.

Pearson's Chi-Square Test of Goodness of Fit

📘 Definition 5.10 — Multinomial data, Pearson's $X^2$ and the deviance $G^2$

Each of $n$ independent observations falls into exactly one of $k$ cells, cell $j$ having probability $p_j$; the cell counts $(N_1,\dots,N_k)$ have the multinomial law $\operatorname{Mult}(n;p_1,\dots,p_k)$. For a fully specified null hypothesis $H_0:p_j=\pi_j$ ($j=1,\dots,k$, $\pi_j\gt0$, $\sum\pi_j=1$), the expected frequencies are $E_j=n\pi_j$, and

$$X^2=\sum_{j=1}^k\frac{(N_j-E_j)^2}{E_j}=\sum_{j=1}^k\frac{N_j^2}{E_j}-n,$$ $$G^2=2\sum_{j=1}^kN_j\log\frac{N_j}{E_j}\quad(0\log0=0).$$

$X^2$ is Pearson's chi-square statistic and $G^2=-2\log\Lambda$ is the likelihood ratio statistic (the deviance), since the unrestricted multinomial MLE is $\hat p_j=N_j/n$.

📐 Theorem 5.14 — Pearson's theorem

Under $H_0:p=\pi$:

  1. $E[X^2]=k-1$ exactly, for every $n$;
  2. $X^2\xrightarrow d\chi^2_{k-1}$ as $n\to\infty$;
  3. $G^2-X^2\xrightarrow P0$, so $G^2\xrightarrow d\chi^2_{k-1}$ as well.

Under a fixed alternative $p\ne\pi$, $X^2/n\xrightarrow P\sum_j(p_j-\pi_j)^2/\pi_j\gt0$, so the test that rejects when $X^2\gt\chi^2_{k-1,\alpha}$ is consistent.

Proof. (1) $N_j\sim\operatorname{Bin}(n,\pi_j)$, so $E(N_j-n\pi_j)^2=n\pi_j(1-\pi_j)$ and $E[X^2]=\sum_j(1-\pi_j)=k-1$.

(2) Write $N=\sum_{i=1}^n\mathbf e_{(i)}$, where $\mathbf e_{(i)}$ is the indicator vector of the cell of observation $i$: iid vectors with mean $\pi$ and covariance $\Sigma=\operatorname{diag}(\pi)-\pi\pi^{\mathsf T}$ (because $\operatorname{Cov}(N_i,N_j)=-n\pi_i\pi_j$ for $i\ne j$). By the multivariate CLT, $(N-n\pi)/\sqrt n\xrightarrow dN_k(0,\Sigma)$. Put $D=\operatorname{diag}(\pi)^{-1/2}$ and $Y_n=D(N-n\pi)/\sqrt n$, the vector with entries $(N_j-n\pi_j)/\sqrt{n\pi_j}$; then $X^2=\|Y_n\|^2$ and $Y_n\xrightarrow dY\sim N_k(0,D\Sigma D)$ with $$D\Sigma D=I_k-\sqrt\pi\sqrt\pi^{\mathsf T},\qquad\sqrt\pi=(\sqrt{\pi_1},\dots,\sqrt{\pi_k})^{\mathsf T}.$$ Since $\|\sqrt\pi\|^2=\sum\pi_j=1$, the matrix $P=I_k-\sqrt\pi\sqrt\pi^{\mathsf T}$ is the orthogonal projection onto the hyperplane $\sqrt\pi^{\perp}$, of rank $k-1$. Choose an orthonormal basis $q_1,\dots,q_{k-1}$ of that hyperplane and let $Q=[q_1\cdots q_{k-1}]$, so $P=QQ^{\mathsf T}$. If $Z\sim N_{k-1}(0,I)$ then $QZ\sim N_k(0,QQ^{\mathsf T})=N_k(0,P)$, so $Y$ has the law of $QZ$ and $\|Y\|^2$ the law of $\|QZ\|^2=\|Z\|^2\sim\chi^2_{k-1}$. By the continuous mapping theorem, $X^2=\|Y_n\|^2\xrightarrow d\chi^2_{k-1}$.

(3) Put $N_j=E_j+\Delta_j$, where $\Delta_j=O_P(\sqrt n)$ and $\sum\Delta_j=0$. Taylor's formula $(1+u)\log(1+u)=u+\frac{u^2}2-\frac{u^3}{6}+\cdots$ with $u=\Delta_j/E_j=O_P(n^{-1/2})$ gives $$N_j\log\frac{N_j}{E_j}=E_j(1+u)\log(1+u)=\Delta_j+\frac{\Delta_j^2}{2E_j}+O_P(n^{-1/2}).$$ Summing, the $\Delta_j$ cancel and $G^2=X^2+O_P(n^{-1/2})$.

Consistency. By the weak law, $N_j/n\xrightarrow Pp_j$, so $$\frac{X^2}{n}=\sum_j\frac{(N_j/n-\pi_j)^2}{\pi_j}\xrightarrow{P}\sum_j\frac{(p_j-\pi_j)^2}{\pi_j}=c\gt0.$$ Then $P(X^2\gt\chi^2_{k-1,\alpha})=P(X^2/n\gt\chi^2_{k-1,\alpha}/n)\to1$. ∎

For $k=2$ the theorem is the de Moivre–Laplace theorem in disguise: $N_2=n-N_1$ and $X^2=\frac{(N_1-n\pi_1)^2}{n\pi_1}+\frac{(N_1-n\pi_1)^2}{n\pi_2}=\frac{(N_1-n\pi_1)^2}{n\pi_1\pi_2}$, which is the square of the score $z$ for a proportion (Exercise II.2).

🧭 Problem-solving strategy — carrying out a chi-square goodness-of-fit test
  1. Cells. Decide the $k$ categories (for continuous data, class intervals fixed before looking at the data).
  2. Expected counts. Compute $E_j=n\pi_j$, estimating any unknown parameters first (Theorem 5.15).
  3. Pool small cells. The $\chi^2$ approximation is reliable when every $E_j\ge5$ (a conservative rule; most $E_j\ge5$ and none below $1$ is often enough). Merge adjacent cells until this holds.
  4. Degrees of freedom. $\nu=(\text{number of cells after pooling})-1-(\text{number of estimated parameters})$.
  5. Decide. Reject $H_0$ when $X^2\gt\chi^2_{\nu,\alpha}$. The test is one-sided: large $X^2$ means a poor fit. An unusually small $X^2$ is suspicious too ("too good to be true"; Exercise II.6).
✏️ Example 4 — A dihybrid cross
A cross of two plants heterozygous for two independent genes should give four phenotypes in the ratio $9:3:3:1$. In $536$ offspring the counts are $290$, $110$, $95$, $41$. Test the genetic model at $5\%$.
Expected $E=536\times\left(\tfrac9{16},\tfrac3{16},\tfrac3{16},\tfrac1{16}\right)=(301.5,\ 100.5,\ 100.5,\ 33.5)$. All exceed $5$.
Statistic $$X^2=\frac{11.5^2}{301.5}+\frac{9.5^2}{100.5}+\frac{5.5^2}{100.5}+\frac{7.5^2}{33.5}=0.439+0.898+0.301+1.679=3.317.$$
Decision $\nu=4-1=3$ (no parameters estimated) and $\chi^2_{3,0.05}=7.815$. Since $3.317\lt7.815$, the data are consistent with $9:3:3:1$ ($p=0.345$). The likelihood ratio version gives $G^2=3.188$, nearly the same.
Note The largest contribution comes from the rare phenotype ($1.679$ from an excess of $7.5$): a small expected count magnifies a deviation. Looking at the individual contributions $(O-E)^2/E$ shows where a model fits badly.
📐 Theorem 5.15 — Goodness of fit with estimated parameters

Let $H_0:p=\pi(\theta)$, where $\theta\in\Theta\subseteq\mathbb{R}^s$ ($s\lt k-1$) is unknown, $\pi$ is twice continuously differentiable with $\pi_j(\theta)\gt0$, and the $k\times s$ matrix $\partial\pi/\partial\theta$ has rank $s$. Let $\hat\theta$ be the MLE computed from the cell counts $N_1,\dots,N_k$ (or the minimum-$\chi^2$ estimator). Then, under $H_0$, $$X^2=\sum_{j=1}^k\frac{\left(N_j-n\pi_j(\hat\theta)\right)^2}{n\pi_j(\hat\theta)}\xrightarrow d\chi^2_{k-1-s},$$ and the same holds for $G^2$.

Proof (sketch). $G^2=-2\log\Lambda$ for $H_0$ inside the full multinomial model. The full model has $k-1$ free parameters, the null model has $s$, and under the stated smoothness and rank conditions Wilks' theorem (§5.1), in the version for a smoothly parametrised null set, gives $G^2\xrightarrow d\chi^2_{k-1-s}$. Part (3) of Theorem 5.14 extends to show $G^2-X^2\xrightarrow P0$. Geometrically, in the proof of Theorem 5.14 the estimated parameters project $Y$ onto a further $s$-dimensional subspace, which removes $s$ dimensions from the $\chi^2$. ∎

The theorem requires $\hat\theta$ to be computed from the grouped counts. In practice parameters are often estimated from the raw data (for instance $\bar x$ for a Poisson mean computed before pooling). Then the limiting law of $X^2$ lies between $\chi^2_{k-1-s}$ and $\chi^2_{k-1}$ (a result of Chernoff and Lehmann), so the test with $k-1-s$ degrees of freedom is slightly anti-conservative. With moderate $k$ the difference is small, and the $k-1-s$ rule is the one used in examinations.

✏️ Example 5 — Fitting a Poisson law to flaw counts
The numbers of flaws on $200$ glass sheets were: $0$ flaws on $52$ sheets, $1$ on $68$, $2$ on $43$, $3$ on $22$, $4$ on $10$, $5$ on $4$ and $6$ on $1$. Test whether the counts follow a Poisson law.
Estimate $\sum x=68+86+66+40+20+6=286$, so $\hat\lambda=\bar x=1.43$.
Expected $200\,e^{-1.43}1.43^x/x!$ gives $47.86,\ 68.44,\ 48.94,\ 23.33$ for $x=0,1,2,3$, then $8.34$ for $x=4$ and $3.09$ for $x\ge5$. The last cell is below $5$, so pool "$4$ or more": expected $11.43$, observed $15$.
Statistic With the cells $0,1,2,3,\ge4$: $$X^2=\frac{4.14^2}{47.86}+\frac{0.44^2}{68.44}+\frac{5.94^2}{48.94}+\frac{1.33^2}{23.33}+\frac{3.57^2}{11.43}=2.27.$$
Decision $\nu=5-1-1=3$ (five cells after pooling, one estimated parameter), and $\chi^2_{3,0.05}=7.815$: the Poisson model is not rejected ($p=0.52$). The sample variance, $1.633$, is a little larger than the mean $1.43$. The index-of-dispersion test of Theorem 5.19(3) gives $D=227.3$ on $199$ degrees of freedom ($p=0.08$): mild over-dispersion at most.
Interactive 1 — Pearson's $X^2$: Simulation Against the $\chi^2$ Law

Each tick draws fresh multinomial samples of size $n$ (a seeded generator) and computes $X^2$ for the null probabilities $\pi$. The histogram of $X^2$ is compared with the $\chi^2_{k-1}$ density, and the dashed line is the $5\%$ critical value. Under $H_0$, the share of samples beyond the line is the actual size of the test: close to $0.05$ when every $n\pi_j$ is reasonably large, less reliable when some $n\pi_j$ are tiny (try the skewed model with $n=10$). Switch the truth to "$H_0$ false" to see the histogram move right: the share beyond the line is then the power.

30
samples—
mean of X² (k − 1 under H₀)—
share with X² beyond the line—
smallest nπⱼ—

Contingency Tables: Independence and Homogeneity

When each individual is classified by two categorical attributes, such as treatment and outcome or education and opinion, the counts form a contingency table. Two questions lead to the same arithmetic but come from different sampling schemes.

📘 Definition 5.11 — Contingency tables; independence and homogeneity

An $r\times c$ table has counts $N_{ij}$, row totals $R_i=\sum_jN_{ij}$, column totals $C_j=\sum_iN_{ij}$ and grand total $n$.

  1. Independence (one sample, two attributes): $n$ individuals are drawn at random and classified by both attributes, so $(N_{ij})\sim\operatorname{Mult}(n;(p_{ij}))$. The hypothesis is $H_0:p_{ij}=p_{i\cdot}\,p_{\cdot j}$ for all $i,j$, where $p_{i\cdot}=\sum_jp_{ij}$ and $p_{\cdot j}=\sum_ip_{ij}$.
  2. Homogeneity ($r$ samples, one attribute): the row totals $R_i$ are fixed by design, row $i$ is a sample from population $i$, and $(N_{i1},\dots,N_{ic})\sim\operatorname{Mult}(R_i;p_{1|i},\dots,p_{c|i})$ independently over $i$. The hypothesis is $H_0:p_{j|1}=\cdots=p_{j|r}$ for every $j$: the populations have the same distribution over the columns.
📐 Theorem 5.16 — The chi-square test for contingency tables

In both models of Definition 5.11, the maximum likelihood estimates of the expected counts under $H_0$ are $$\hat E_{ij}=\frac{R_iC_j}{n},$$ and under $H_0$, as $n\to\infty$ (with all cell probabilities positive, and in the homogeneity model with every $R_i\to\infty$), $$X^2=\sum_{i=1}^r\sum_{j=1}^c\frac{(N_{ij}-\hat E_{ij})^2}{\hat E_{ij}}\xrightarrow d\chi^2_{(r-1)(c-1)},$$ and the same holds for $G^2=2\sum N_{ij}\log(N_{ij}/\hat E_{ij})$.

Proof. Independence. Under $H_0$ write $p_{ij}=a_ib_j$ with $\sum a_i=\sum b_j=1$. The log-likelihood is $\sum_{i,j}N_{ij}\log(a_ib_j)=\sum_iR_i\log a_i+\sum_jC_j\log b_j$. Maximising $\sum R_i\log a_i$ subject to $\sum a_i=1$ (a Lagrange multiplier, or Gibbs' inequality) gives $\hat a_i=R_i/n$; similarly $\hat b_j=C_j/n$. So $\hat E_{ij}=n\hat a_i\hat b_j=R_iC_j/n$. The full model has $rc-1$ free parameters and the null model $(r-1)+(c-1)$, so by Theorem 5.15 the limit is $\chi^2$ with $$rc-1-(r-1)-(c-1)=(r-1)(c-1)$$ degrees of freedom.

Homogeneity. Under $H_0$ the common column distribution $(q_j)$ has log-likelihood $\sum_i\sum_jN_{ij}\log q_j=\sum_jC_j\log q_j$, maximised at $\hat q_j=C_j/n$; hence $\hat E_{ij}=R_i\hat q_j=R_iC_j/n$. The full model has $r(c-1)$ free parameters (one probability vector per row) and the null model $c-1$, and $r(c-1)-(c-1)=(r-1)(c-1)$. Wilks' theorem for independent multinomial samples gives the limit. ∎

So the two hypotheses, although different in meaning, lead to the same statistic, the same expected counts and the same degrees of freedom. What differs is the conclusion: "the attributes are associated" against "the populations differ".

✏️ Example 6 — Education and opinion: a $3\times3$ table
A survey of $300$ adults cross-classifies education by opinion on a proposed policy:
AgreeNeutralDisagreeTotal
School453025100
Graduate403545120
Postgraduate20154580
Total10580115300
Test at $1\%$ whether opinion is independent of education, and describe the association.
Expected $\hat E_{ij}=R_iC_j/300$: row School $(35,\ 26.67,\ 38.33)$; Graduate $(42,\ 32,\ 46)$; Postgraduate $(28,\ 21.33,\ 30.67)$. All exceed $5$.
Contributions $(O-E)^2/E$: School $(2.857,\ 0.417,\ 4.638)$; Graduate $(0.095,\ 0.281,\ 0.022)$; Postgraduate $(2.286,\ 1.880,\ 6.699)$. Sum: $X^2=19.17$.
Decision $\nu=(3-1)(3-1)=4$ and $\chi^2_{4,0.01}=13.277$. Since $19.17\gt13.28$, reject independence ($p=0.0007$). ($G^2=19.19$.)
Interpretation The largest contributions are in the "Disagree" column: postgraduates disagree far more often than independence predicts ($45$ against $30.7$) and school leavers less often ($25$ against $38.3$). The test says that the attributes are associated; the contributions say how.
📐 Theorem 5.17 — The $2\times2$ table

For a $2\times2$ table with cells $a,b$ (row 1), $c,d$ (row 2), row totals $R_1,R_2$, column totals $C_1,C_2$ and total $n$:

  1. every cell has $|O-E|=|ad-bc|/n$, and $$X^2=\frac{n(ad-bc)^2}{R_1R_2C_1C_2};$$
  2. $X^2=Z^2$, where $Z$ is the pooled two-proportion statistic of Theorem 5.12 for $\hat p_1=a/R_1$ against $\hat p_2=c/R_2$;
  3. $X^2\xrightarrow d\chi^2_1$ under $H_0$. Yates' continuity correction replaces $|ad-bc|$ by $|ad-bc|-n/2$: $$X^2_{\mathrm{Y}}=\frac{n\left(|ad-bc|-n/2\right)^2}{R_1R_2C_1C_2},$$ which brings the $\chi^2_1$ tail closer to the exact (Fisher) $p$-value when counts are small.

Proof. (1) $a-\frac{R_1C_1}{n}=\frac{a(a+b+c+d)-(a+b)(a+c)}{n}=\frac{ad-bc}{n}$, and since the margins of $O$ and $E$ agree, the other three deviations are $\pm\frac{ad-bc}{n}$. Hence $$X^2=\frac{(ad-bc)^2}{n^2}\sum_{i,j}\frac{n}{R_iC_j}=\frac{(ad-bc)^2}{n}\left(\frac1{R_1}+\frac1{R_2}\right)\left(\frac1{C_1}+\frac1{C_2}\right),$$ and $\left(\frac1{R_1}+\frac1{R_2}\right)\left(\frac1{C_1}+\frac1{C_2}\right)=\frac{n}{R_1R_2}\cdot\frac{n}{C_1C_2}$.

(2) $\hat p_1-\hat p_2=\frac{a}{R_1}-\frac{c}{R_2}=\frac{a(c+d)-c(a+b)}{R_1R_2}=\frac{ad-bc}{R_1R_2}$, the pooled proportion is $\hat p=C_1/n$ and $1-\hat p=C_2/n$. So $$Z^2=\frac{(ad-bc)^2/(R_1R_2)^2}{\frac{C_1C_2}{n^2}\cdot\frac{n}{R_1R_2}}=\frac{n(ad-bc)^2}{R_1R_2C_1C_2}=X^2.$$

(3) is Theorem 5.16 with $r=c=2$. The correction subtracts half a unit from each $|O-E|$, the analogue of the continuity correction for the binomial (§2.3); we do not prove its accuracy. ∎

✏️ Example 7 — A small trial: four ways to analyse a $2\times2$ table
Fifty patients are randomised, $25$ to a new treatment and $25$ to the standard one. Improvement: new $18$ of $25$, standard $11$ of $25$. (a) Compute $X^2$ and $Z$. (b) Compute Yates' $X^2_{\mathrm Y}$. (c) Compare with Fisher's exact test (Theorem 5.18 below). (d) Estimate the odds ratio with a $95\%$ interval.
(a) $a=18,b=7,c=11,d=14$; $ad-bc=252-77=175$; $R_1=R_2=25$, $C_1=29$, $C_2=21$. $$X^2=\frac{50\times175^2}{25\times25\times29\times21}=4.023,$$ $p=0.045$. Equivalently $\hat p_1=0.72$, $\hat p_2=0.44$, $\hat p=0.58$, $Z=\frac{0.28}{\sqrt{0.58\times0.42\times0.08}}=2.006$ and $Z^2=4.023$.
(b) $X^2_{\mathrm Y}=\frac{50\,(175-25)^2}{380625}=2.956$, $p=0.086$. The decision at $5\%$ changes.
(c) Conditional on the margins, $a$ is hypergeometric. The one-sided exact $p$-value is $P(A\ge18)=0.042$ and the two-sided one is $0.085$. The two-sided Yates $p$-value ($0.086$) is close to the exact two-sided value; the uncorrected $0.045$ is too small. With such small samples the evidence is borderline.
(d) $\hat\theta=\frac{ad}{bc}=\frac{252}{77}=3.27$; $\log\hat\theta=1.186$ with standard error $\sqrt{\frac1{18}+\frac17+\frac1{11}+\frac1{14}}=0.601$ (Theorem 5.19). The interval $\exp(1.186\pm1.96\times0.601)=[1.01,\ 10.6]$ only just excludes $1$: compatible with anything from no effect to a tenfold increase in the odds.
Interactive 2 — A $2\times2$ Table With Fixed Margins: $X^2$, Yates and Fisher

Choose the row totals $R_1,R_2$ and the first column total $C_1$; then the single count $a$ determines the whole table. On the left: the table with expected counts in brackets. On the right: the hypergeometric law of $a$ given the margins, which is the exact null distribution used by Fisher's test (bar heights on a square-root scale, so that the tails are visible). The red bars are the tables at least as unlikely as the observed one, and their total is the two-sided exact $p$-value. Compare it with the $\chi^2_1$ $p$-values with and without Yates' correction, especially for small margins. Play sweeps $a$ through its range.

25252918
X² (p)—
Yates X² (p)—
Fisher exact p (two-sided)—
odds ratio ad/bc—

Exact and Paired Tests for Discrete Data

📐 Theorem 5.18 — Fisher's exact test

In a $2\times2$ table under either the homogeneity model with $H_0:p_1=p_2$ or the independence model with $H_0$ of independence, the conditional distribution of $A$ (the count $a$) given all the margins is hypergeometric: $$P(A=a\mid R_1,R_2,C_1)=\frac{\binom{R_1}{a}\binom{R_2}{C_1-a}}{\binom{n}{C_1}},\qquad\max(0,C_1-R_2)\le a\le\min(R_1,C_1).$$ It does not depend on the unknown common probability. Fisher's exact test computes $p$-values from this law: one-sided $P(A\ge a_{\mathrm{obs}})$, and two-sided the total probability of all $a$ with $P(A=a)\le P(A=a_{\mathrm{obs}})$.

Proof. In the homogeneity model, under $H_0$ the two rows are independent $\operatorname{Bin}(R_1,p)$ and $\operatorname{Bin}(R_2,p)$ in their first column. So $$P(A=a,\ C=C_1-a)=\binom{R_1}{a}\binom{R_2}{C_1-a}p^{C_1}(1-p)^{n-C_1}.$$ Summing over $a$ and using Vandermonde's identity $\sum_a\binom{R_1}{a}\binom{R_2}{C_1-a}=\binom n{C_1}$ gives $P(A+C=C_1)=\binom n{C_1}p^{C_1}(1-p)^{n-C_1}$, and dividing gives the hypergeometric law: $p$ cancels. In the independence model, conditioning first on the row totals reduces to the homogeneity model (given $R_1,R_2$, the rows are independent binomials with success probability $p_{\cdot1}$ under independence). ∎

The column total $C_1$ is sufficient for the nuisance parameter $p$ under $H_0$, and conditioning on a sufficient statistic removes it. This is the general idea behind exact conditional tests. Because the conditional law is discrete, the test is conservative: its actual size is usually below the nominal $\alpha$.

📘 Definition 5.12 — Odds and the odds ratio
The odds of an event of probability $p$ are $p/(1-p)$. For a $2\times2$ table of probabilities $(p_{ij})$, or for two groups with success probabilities $p_1,p_2$, the odds ratio is $$\theta=\frac{p_{11}p_{22}}{p_{12}p_{21}}=\frac{p_1/(1-p_1)}{p_2/(1-p_2)},$$ estimated by $\hat\theta=\frac{ad}{bc}$. With positive cell probabilities, independence holds iff $\theta=1$, and $\log\theta$ is antisymmetric under swapping the rows or the columns.
📐 Theorem 5.19 — Three further large-sample tests for discrete data
  1. Log odds ratio (Woolf). For two independent binomial samples with all cell counts large, $\log\hat\theta$ is approximately normal with mean $\log\theta$ and estimated standard error $\sqrt{\frac1a+\frac1b+\frac1c+\frac1d}$; so $H_0:\theta=1$ can be tested by $Z=\log\hat\theta\big/\sqrt{\frac1a+\frac1b+\frac1c+\frac1d}$.
  2. McNemar's test. $n$ pairs are each classified (yes/no) twice, for instance before and after a treatment. Let $b$ be the number of pairs (yes, no) and $c$ the number (no, yes). Under $H_0$ of marginal homogeneity, $P(\text{yes first})=P(\text{yes second})$, conditionally on $b+c=m$ we have $b\sim\operatorname{Bin}(m,\frac12)$, and $$X^2_{\mathrm M}=\frac{(b-c)^2}{b+c}\xrightarrow d\chi^2_1\quad(m\to\infty).$$
  3. Index of dispersion. If $X_1,\dots,X_n$ are iid $\operatorname{Poi}(\lambda)$, then given $T=\sum X_i=t$ the vector $(X_1,\dots,X_n)$ is $\operatorname{Mult}(t;\frac1n,\dots,\frac1n)$, and the Pearson statistic of this multinomial is $$D=\frac{\sum_{i=1}^n(X_i-\bar X)^2}{\bar X}\approx\chi^2_{n-1}.$$ Large $D$ indicates over-dispersion (variance larger than the mean), small $D$ under-dispersion.

Proof. (1) For row 1, $\hat p_1=a/R_1$ and $g(p)=\log\frac p{1-p}$ has $g'(p)=\frac1{p(1-p)}$; by the delta method $\log\frac{a}{b}=g(\hat p_1)$ has approximate variance $\frac{p_1(1-p_1)}{R_1}\cdot\frac1{p_1^2(1-p_1)^2}=\frac1{R_1p_1}+\frac1{R_1(1-p_1)}$, estimated by $\frac1a+\frac1b$. Row 2 is independent and gives $\frac1c+\frac1d$, and $\log\hat\theta=\log\frac ab-\log\frac cd$.

(2) Let $p_{10},p_{01}$ be the probabilities of the discordant outcomes (yes, no) and (no, yes). Marginal homogeneity is $p_{11}+p_{10}=p_{11}+p_{01}$, i.e. $p_{10}=p_{01}$. Given that $m$ pairs are discordant, each is of type (yes, no) independently with probability $\frac{p_{10}}{p_{10}+p_{01}}=\frac12$. So $b\sim\operatorname{Bin}(m,\frac12)$, $\frac{b-m/2}{\sqrt{m/4}}=\frac{b-c}{\sqrt{b+c}}\xrightarrow dN(0,1)$, and squaring gives $X^2_{\mathrm M}$. The concordant pairs carry no information about $H_0$.

(3) For $\sum x_i=t$, $$P(X=x\mid T=t)=\frac{\prod e^{-\lambda}\lambda^{x_i}/x_i!}{e^{-n\lambda}(n\lambda)^t/t!}=\frac{t!}{\prod x_i!}\left(\frac1n\right)^t,$$ the uniform multinomial. Its expected counts are $t/n=\bar x$, so Pearson's statistic is $\sum(X_i-\bar X)^2/\bar X=D$, and Theorem 5.14 (with $k=n$ cells, as $t\to\infty$) gives the $\chi^2_{n-1}$ approximation. ∎

⚠️ Counterexamples and pitfalls
  1. Large expected counts, not large observed counts. The $\chi^2$ approximation needs the $E_j$ to be reasonably large (rule of thumb $\ge5$). Pool cells, and count the degrees of freedom after pooling (Example 5).
  2. Degrees of freedom. Subtract one for each parameter estimated from the data, not for parameters fixed by $H_0$. For a contingency table it is $(r-1)(c-1)$, not $rc-1$.
  3. The test is upper-tailed. Only large $X^2$ leads to rejection, even though the alternative is "any departure". A tiny $X^2$ (for example $0.08$ on $3$ df, left-tail probability $0.006$) suggests data that fit too well (Exercise II.6).
  4. Percentages are not counts. $X^2$ is proportional to $n$ for fixed proportions: computing it from percentages, or from counts doubled, gives a wrong value.
  5. Large $n$ finds trivial departures; small $n$ finds nothing. Non-rejection is not proof of fit, and rejection is not proof of an important departure.
  6. Paired data are not independent samples. Before/after proportions on the same subjects must be compared with McNemar's test, not the two-proportion $z$-test.
  7. Pooled, not unpooled. $X^2$ for a $2\times2$ table equals $Z^2$ with the pooled standard error (Theorem 5.17), not the unpooled one.
  8. Collapsing tables can reverse an association (Simpson's paradox). A treatment can be better in every stratum and worse in the combined table (Exercise III.6).
  9. Yates' correction is for $1$ degree of freedom. It applies to $2\times2$ tables; it makes the test conservative, and it is not used for larger tables.
  10. Wald fails at the boundary. With $\hat p=0$ the Wald standard error is $0$ and the Wald statistic is infinite or undefined; the score test, which uses $p_0$, is fine.
  11. Independence versus homogeneity. They have the same statistic and degrees of freedom but different sampling schemes and different conclusions; with fixed row totals one cannot estimate the row proportions $p_{i\cdot}$.
  12. Two-sided exact $p$-values are not always double the one-sided ones. With unequal margins the hypergeometric law is skewed (Exercise IV.7).

Exercises

I
Large-sample tests.
  • 1 In a sample of $1000$ voters, $540$ favour a candidate. Test $H_0:p=0.5$ against $H_1:p\gt0.5$ at $1\%$.
  • 2 A coin is tossed $400$ times and shows $220$ heads. Test fairness (two-sided, $5\%$) with and without the continuity correction. Comment.
  • 3 Smokers: $140$ of $400$ men and $140$ of $500$ women. Test $p_{\text{men}}=p_{\text{women}}$ at $5\%$.
  • 4 A sample of $64$ observations from a skewed population gives $\bar x=52.3$, $s=6.4$. Test $\mu=50$ (two-sided).
  • 5 A road averaged $2$ accidents per month. In the last $12$ months it had $36$. Assuming Poisson counts, test for an increase with the large-sample test, and compare with the exact Poisson $p$-value.
  • 6 Site A recorded $45$ events in $100$ days and site B $30$ events in $100$ days (Poisson). Test equality of the rates (two-sided). Show that the test based on $A$ given $A+B=75$ leads to the same $z$.
  • 7 How many voters must be polled so that the one-sided $5\%$ test of $p=0.5$ has power $0.9$ when $p=0.55$? Use the null variance for $z_\alpha$ and the alternative variance for $z_\beta$.
  • 8 Prove that the one-proportion score test is consistent: if $p\ne p_0$, then $P(|Z_n|\gt z_{\alpha/2})\to1$.
  • 9 From $n=39$ pairs (bivariate normal), $r=0.42$. Test $H_0:\rho=0.25$ with Fisher's $z$.
1 $z=\frac{0.54-0.5}{\sqrt{0.25/1000}}=2.530\gt z_{0.01}=2.326$: reject ($p=0.0057$).
2 Without correction $z=\frac{220-200}{10}=2.0$, $p=0.0455$: reject. With correction $z=\frac{219.5-200}{10}=1.95$, $p=0.0512$: do not reject. The exact binomial two-sided $p$-value is $0.0510$. A borderline result should be reported as borderline.
3 $\hat p_1=0.35$, $\hat p_2=0.28$, $\hat p=280/900=0.3111$; $\operatorname{SE}_0=\sqrt{0.3111\times0.6889\times(1/400+1/500)}=0.03106$; $z=0.07/0.03106=2.254$, $p=0.024$: reject at $5\%$.
4 $z=\frac{2.3}{6.4/8}=2.875$, two-sided $p=0.004$: reject. Validity rests on the CLT with $n=64$, not on normality.
5 $T=36\sim\operatorname{Poi}(24)$ under $H_0$: $z=\frac{36-24}{\sqrt{24}}=2.449$, $p=0.0072$. Exact $P(T\ge36)=0.0132$. The normal approximation understates the $p$-value because the Poisson is right-skewed; both reject at $5\%$.
6 $z=\frac{0.45-0.30}{\sqrt{0.45/100+0.30/100}}=\frac{0.15}{0.0866}=1.732$, $p=0.083$: do not reject. Given $A+B=75$, $A\sim\operatorname{Bin}(75,\frac12)$ under equal rates (§3.3, Poisson splitting), and $\frac{45-37.5}{\sqrt{18.75}}=1.732$ as well. (Exact conditional two-sided $p=0.105$.)
7 $n=\left(\frac{z_{0.05}\sqrt{0.25}+z_{0.10}\sqrt{0.55\times0.45}}{0.05}\right)^2=\left(\frac{0.8224+0.6376}{0.05}\right)^2=852.6$, so $853$ voters.
8 $Z_n=\sqrt n\frac{\hat p-p}{\sqrt{p_0q_0}}+\sqrt n\frac{p-p_0}{\sqrt{p_0q_0}}$. The first term is bounded in probability (CLT), the second tends to $\pm\infty$. Hence $|Z_n|\to\infty$ in probability (Theorem 5.11(2)).
9 $\operatorname{artanh}0.42=0.4477$, $\operatorname{artanh}0.25=0.2554$, $z=\sqrt{36}\,(0.1923)=1.154$, $p=0.249$: no evidence that $\rho\ne0.25$.
II
Goodness of fit.
  • 1 A die rolled $120$ times gives the counts $25,17,15,23,24,16$. Test fairness at $5\%$ with $X^2$ and with $G^2$.
  • 2 Show that for $k=2$ cells $X^2$ is the square of the score statistic $\frac{N_1-n\pi_1}{\sqrt{n\pi_1(1-\pi_1)}}$.
  • 3 Show that $X^2=\sum N_j^2/E_j-n$ when $\sum E_j=n$.
  • 4 The numbers of girls in $100$ families with four children are: $0$ girls in $8$ families, $1$ in $28$, $2$ in $35$, $3$ in $22$, $4$ in $7$. Fit a binomial law with $p$ estimated and test the fit.
  • 5 A normal distribution with both parameters estimated is fitted to grouped data in $8$ classes; the two classes at each end have expected counts below $5$ and are merged with their neighbours. How many degrees of freedom does the $X^2$ test have?
  • 6 A student reports $25,24,26,25$ in four equally likely categories. Compute $X^2$ and $P(\chi^2_3\le X^2)$. What might you suspect?
  • 7 Show that multiplying every count by $m$ multiplies $X^2$ by $m$. If the proportions of part 1 were observed in $120m$ rolls, what is the smallest $m$ for which fairness is rejected at $5\%$?
  • 8 A random digit generator produced $1000$ digits with frequencies $92,108,97,115,89,102,95,110,87,105$ for $0,\dots,9$. Test uniformity at $5\%$.
1 $E_j=20$; $X^2=\frac{25+9+25+9+16+16}{20}=5.0$ on $5$ df, below $11.070$ ($p=0.416$). $G^2=2\sum O\log(O/20)=5.04$. Do not reject.
2 $N_2-n\pi_2=-(N_1-n\pi_1)$, so $X^2=(N_1-n\pi_1)^2\left(\frac1{n\pi_1}+\frac1{n\pi_2}\right)=\frac{(N_1-n\pi_1)^2}{n\pi_1\pi_2}$, the square of the score statistic.
3 $\sum\frac{(N_j-E_j)^2}{E_j}=\sum\frac{N_j^2}{E_j}-2\sum N_j+\sum E_j=\sum\frac{N_j^2}{E_j}-2n+n$.
4 $\hat p=\frac{0\cdot8+28+70+66+28}{400}=0.48$. Expected $100\binom4kp^k(1-p)^{4-k}$: $7.31,\ 27.00,\ 37.38,\ 23.00,\ 5.31$. $X^2=0.836$ on $5-1-1=3$ df ($p=0.84$): the binomial model fits well.
5 After merging, $8-2=6$ classes remain; two parameters are estimated: $\nu=6-1-2=3$.
6 $X^2=\frac{0+1+1+0}{25}=0.08$ and $P(\chi^2_3\le0.08)=0.006$. Data this close to expectation occur only $6$ times in $1000$ under honest random sampling: the counts may have been "adjusted". (Fisher raised exactly this objection against some historical genetic data.)
7 Each term $(mN_j-mE_j)^2/(mE_j)=m(N_j-E_j)^2/E_j$. Here $X^2=5m\gt11.070$ first for $m=3$ ($X^2=15$): the same percentages become significant with $360$ rolls.
8 $E_j=100$; $X^2=\frac{64+64+9+225+121+4+25+100+169+25}{100}=8.06$ on $9$ df, below $\chi^2_{9,0.05}=16.919$ ($p=0.53$): consistent with uniformity.
III
Contingency tables.
  • 1 For the table with rows $(30,20)$ and $(15,35)$ compute $X^2$ by the formula of Theorem 5.17, the pooled $Z$, and Yates' $X^2_{\mathrm Y}$.
  • 2 Using Lagrange multipliers, show that $\hat a_i=R_i/n$ maximises $\sum_iR_i\log a_i$ subject to $\sum a_i=1$.
  • 3 Three treatments are given to $50$ patients each. Recovered: $40,35,25$; not recovered: $10,15,25$. Test homogeneity at $5\%$.
  • 4 State the degrees of freedom for testing independence in a $3\times4$ table and homogeneity of $5$ binomial populations.
  • 5 Explain why a researcher who computes $X^2$ for a $2\times3$ table from percentages of row totals, instead of counts, gets a wrong answer.
  • 6 Construct an example of Simpson's paradox: in each of two strata a treatment has the higher success rate, but in the combined table it has the lower rate.
  • 7 Prove that in a $2\times2$ table $X^2=0$ iff $ad=bc$ iff $\hat p_1=\hat p_2$.
  • 8 A $2\times2$ table with $n=20$ has smallest expected count $2.4$. Which test should be used, and why?
1 $ad-bc=1050-300=750$; $R_1=R_2=50$, $C_1=45$, $C_2=55$, $n=100$. $X^2=\frac{100\times750^2}{50\times50\times45\times55}=9.091$ ($p=0.0026$); $Z=\sqrt{9.091}=3.015$; $X^2_{\mathrm Y}=\frac{100\times700^2}{6187500}=7.919$ ($p=0.0049$). All reject at $1\%$.
2 Maximise $\sum R_i\log a_i-\mu(\sum a_i-1)$: $R_i/a_i=\mu$, so $a_i=R_i/\mu$, and $\sum a_i=1$ gives $\mu=n$. The function is strictly concave, so this is the maximum.
3 $\hat E$: recovered $33.33$ in each column, not recovered $16.67$. $X^2=10.5$ on $(2-1)(3-1)=2$ df, above $5.991$ ($p=0.0052$): the recovery rates differ.
4 $(3-1)(4-1)=6$; for $5$ binomial populations the table is $5\times2$, so $(5-1)(2-1)=4$.
5 $X^2$ is proportional to the sample size (Exercise II.7); percentages correspond to $100$ observations per row, whatever the real row totals, so the statistic is rescaled arbitrarily, and the $\chi^2$ reference law no longer applies.
6 Mild cases: treatment $90/100=0.90$, control $170/200=0.85$. Severe cases: treatment $120/200=0.60$, control $25/50=0.50$. Combined: treatment $210/300=0.70$, control $195/250=0.78$. The treatment was given mostly to severe cases, and severity is a confounder.
7 By Theorem 5.17(1), $X^2=0\iff ad=bc$, and by the proof of (2), $\hat p_1-\hat p_2=\frac{ad-bc}{R_1R_2}$.
8 Fisher's exact test: an expected count of $2.4$ makes the $\chi^2_1$ approximation unreliable, while the hypergeometric law (Theorem 5.18) is exact for every sample size.
IV
Exact and paired tests.
  • 1 A table has rows $(7,1)$ and $(2,6)$. Compute the one- and two-sided Fisher exact $p$-values.
  • 2 $200$ voters were asked before and after a debate. $25$ switched from "yes" to "no" and $10$ from "no" to "yes". Test for a change with McNemar's test, with and without the continuity correction $(|b-c|-1)^2/(b+c)$, and exactly.
  • 3 For the table of Exercise III.1 estimate the odds ratio and give a $95\%$ confidence interval.
  • 4 Ten Poisson counts are $3,7,2,9,4,6,1,8,5,5$. Test for over-dispersion with the index of dispersion.
  • 5 In McNemar's set-up, show that marginal homogeneity holds iff $p_{10}=p_{01}$, and that the concordant pairs do not enter the test.
  • 6 Prove that, given $\sum X_i=t$, iid Poisson counts $X_1,\dots,X_n$ are $\operatorname{Mult}(t;\frac1n,\dots,\frac1n)$.
  • 7 For the table with rows $(5,0)$ and $(1,14)$, compute the one- and two-sided Fisher $p$-values. Why are they equal?
1 Margins $R_1=R_2=8$, $C_1=9$, $n=16$; $P(A=a)=\binom8a\binom8{9-a}/\binom{16}9$, so $P(A=7)=0.01958$, $P(A=8)=0.00070$. One-sided $p=P(A\ge7)=0.0203$. The law is symmetric about $4.5$, so the two-sided $p=0.0406$.
2 $X^2_{\mathrm M}=\frac{(25-10)^2}{35}=6.43$ ($p=0.011$); corrected $\frac{14^2}{35}=5.60$ ($p=0.018$); exact $2P(\operatorname{Bin}(35,\frac12)\le10)=0.017$. Opinion shifted towards "no". The $165$ voters who did not change are irrelevant.
3 $\hat\theta=\frac{30\times35}{20\times15}=3.5$, $\log\hat\theta=1.253$, $\operatorname{SE}=\sqrt{\frac1{30}+\frac1{20}+\frac1{15}+\frac1{35}}=0.4226$, interval $\exp(1.253\pm0.828)=[1.53,\ 8.01]$.
4 $\bar x=5$, $\sum(x_i-5)^2=60$, $D=12$ on $9$ df; $P(\chi^2_9\gt12)=0.213$: no evidence of over-dispersion.
5 $P(\text{yes first})=p_{11}+p_{10}$ and $P(\text{yes second})=p_{11}+p_{01}$; they are equal iff $p_{10}=p_{01}$. Given the number $m$ of discordant pairs, $b$ is $\operatorname{Bin}\left(m,\frac{p_{10}}{p_{10}+p_{01}}\right)$, a law not involving $p_{11},p_{00}$; the test conditions on $m$ and uses only $b$.
6 $T\sim\operatorname{Poi}(n\lambda)$, and for $\sum x_i=t$: $$\frac{\prod_ie^{-\lambda}\lambda^{x_i}/x_i!}{e^{-n\lambda}(n\lambda)^t/t!}=\frac{t!}{x_1!\cdots x_n!}n^{-t},$$ the $\operatorname{Mult}(t;\frac1n,\dots,\frac1n)$ pmf, free of $\lambda$.
7 $R_1=5$, $R_2=15$, $C_1=6$, $n=20$: $P(A=5)=\binom55\binom{15}1/\binom{20}6=15/38760=0.00039$. Every other value of $a$ is more probable (the next smallest is $P(A=4)=0.0135$), so both $p$-values equal $0.00039$. The hypergeometric law here is skewed, and the opposite tail contains no table as extreme.
🎯 Exam corner CSIR Part BCSIR Part CGATE ST

E1. CSIR Part B GATE ST In a $2\times2$ contingency table the cells are $20,30$ (first row) and $30,20$ (second row). Pearson's $X^2$ statistic for independence (without continuity correction) equals (A) $1.0$; (B) $2.0$; (C) $4.0$; (D) $8.0$.

Idea Theorem 5.17: $X^2=\frac{n(ad-bc)^2}{R_1R_2C_1C_2}$.
Compute $ad-bc=400-900=-500$, all margins $50$, $n=100$: $X^2=\frac{100\times250000}{50^4}=4.0$. (Check: every $E=25$, every $|O-E|=5$, $X^2=4\times25/25=4$.)
Answer (C).

E2. CSIR Part B A Poisson distribution, with its mean estimated from the grouped data, is fitted to frequency data. After pooling, $7$ classes remain. The degrees of freedom of the chi-square goodness-of-fit statistic are (A) $7$; (B) $6$; (C) $5$; (D) $4$.

Idea $\nu=k-1-s$ with $k$ counted after pooling (Theorem 5.15).
Compute $7-1-1=5$. (B) forgets the estimated parameter; (D) subtracts it twice.
Answer (C).

E3. CSIR Part B A coin is tossed $400$ times and shows $220$ heads. The large-sample statistic $Z=(\hat p-p_0)/\sqrt{p_0(1-p_0)/n}$ for $H_0:p=\frac12$ equals (A) $1.0$; (B) $2.0$; (C) $2.2$; (D) $4.0$.

Compute $\hat p=0.55$ and $\sqrt{0.25/400}=0.025$, so $Z=0.05/0.025=2.0$. Equivalently $(220-200)/\sqrt{400\times\frac14}=20/10$.
Answer (B).

E4. CSIR Part C Let $(N_1,\dots,N_k)\sim\operatorname{Mult}(n;p_1,\dots,p_k)$ and let $X^2=\sum(N_j-n\pi_j)^2/(n\pi_j)$ for a fully specified $\pi$ with all $\pi_j\gt0$. Which of the following are true? (A) Under $H_0:p=\pi$, $E[X^2]=k-1$ for every $n$. (B) $X^2=0$ if and only if $N_j=n\pi_j$ for every $j$. (C) Under $H_0$, $X^2$ has exactly the $\chi^2_{k-1}$ distribution for every $n$. (D) If $p\ne\pi$ is fixed, $P(X^2\gt\chi^2_{k-1,\alpha})\to1$ as $n\to\infty$.

(A) True $E(N_j-n\pi_j)^2=n\pi_j(1-\pi_j)$, so $E[X^2]=\sum(1-\pi_j)=k-1$ (Theorem 5.14(1)).
(B) True A sum of non-negative terms vanishes iff each term does.
(C) False $X^2$ takes finitely many values for fixed $n$, so it cannot have a continuous law; $\chi^2_{k-1}$ is only the limit.
(D) True $X^2/n\to\sum(p_j-\pi_j)^2/\pi_j\gt0$ in probability (consistency, Theorem 5.14).
Answer (A), (B) and (D).

E5. CSIR Part C For a $2\times2$ table with cells $a,b,c,d$ and all margins positive, which of the following are true? (A) Pearson's $X^2$ equals $Z^2$, where $Z$ is the two-proportion statistic with the pooled standard error. (B) The test of independence (one multinomial sample) and the test of homogeneity (fixed row totals) use the same statistic and the same degrees of freedom. (C) If $X^2$ is significant, then $ad/(bc)\gt1$. (D) Fisher's exact test uses the binomial distribution of $a$ given the first row total only.

(A) True Theorem 5.17(2).
(B) True Theorem 5.16: $\hat E_{ij}=R_iC_j/n$ and $(r-1)(c-1)=1$ in both models.
(C) False $X^2$ depends on $(ad-bc)^2$ and ignores the direction: the table with rows $(5,20)$, $(20,5)$ has $X^2=18$ and $ad/(bc)=1/16$.
(D) False It conditions on all margins, and the law of $a$ is hypergeometric (Theorem 5.18). The binomial law given the row total still depends on the unknown $p$.
Answer (A) and (B).

E6. CSIR Part C GATE ST Let $X_1\sim\operatorname{Bin}(m,p_1)$ and $X_2\sim\operatorname{Bin}(n,p_2)$ be independent, $m,n\to\infty$ with $m/(m+n)\to\lambda\in(0,1)$, and let $Z$ be the pooled two-proportion statistic. Which of the following are true? (A) Under $H_0:p_1=p_2$, $Z\xrightarrow dN(0,1)$. (B) $Z$ remains asymptotically $N(0,1)$ under $H_0$ if the two "samples" are the same subjects measured twice (paired data). (C) For fixed $p_1\ne p_2$, the power of the test that rejects when $|Z|\gt z_{\alpha/2}$ tends to $1$. (D) Under $H_0$, $Z$ has the $t_{m+n-2}$ distribution.

(A) True Theorem 5.12 (CLT and Slutsky).
(B) False With paired data $\hat p_1$ and $\hat p_2$ are dependent, so the variance of $\hat p_1-\hat p_2$ is not $p(1-p)(1/m+1/n)$; McNemar's test is the correct tool (Theorem 5.19(2)).
(C) True Theorem 5.11(2): $|Z|\to\infty$ in probability.
(D) False $Z$ is discrete; the $t$ law belongs to normal samples with an estimated variance.
Answer (A) and (C).
🧑‍🏫 For the teacher

Suggested plan — 3 lectures × 60 min.

  1. Large-sample tests. Theorem 5.11 with the Slutsky argument on the board, then the table of Theorem 5.12, proving the two-proportion and Poisson cases. Examples 1–3; in Example 1 contrast the one- and two-sided decisions. Theorem 5.13 (trinity) in ten minutes, with the poll numbers.
  2. Goodness of fit. Definition 5.10, Theorem 5.14 (prove (1) and the case $k=2$ in full; give the projection argument for general $k$). Project Interactive 1: first the fair die with $n=30$, then the skewed model with $n=10$, so that the class sees when the approximation fails. Examples 4 and 5; state Theorem 5.15 and practise counting degrees of freedom.
  3. Contingency tables and discrete data. Theorem 5.16 with the parameter count, Example 6, Theorem 5.17 and Example 7; project Interactive 2 with the margins of Example 7 and then with small margins ($R_1=R_2=6$). Fisher's test, odds ratio and McNemar (Theorems 5.18–5.19). Finish with E4 and E5 live.

Discussion questions. (1) Why is the chi-square test upper-tailed although the alternative is two-sided in spirit? What would a lower-tail rejection mean? (2) In Example 7 the uncorrected test rejects and the corrected and exact tests do not. Which would you report, and why? (3) A table shows that hospitals with more specialists have higher death rates. What third variable could explain this, and how would you check?

Misconceptions to watch for. Using observed instead of expected counts in the denominator; counting degrees of freedom before pooling; subtracting degrees of freedom for parameters fixed by $H_0$; computing $X^2$ from percentages; applying the two-proportion test to paired data; believing that a significant $X^2$ shows the direction or the size of an association.

Suggested homework. I(2, 3, 5, 7), II(1, 4, 6, 7), III(1, 3, 6, 7), IV(1, 2, 4); Exam corner E2, E4, E6.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. Do the leading digits of real-world numbers follow Benford's law, $P(D=d)=\log_{10}\left(1+\frac1d\right)$, $d=1,\dots,9$? And can a chi-square test of this law help to detect fabricated figures?

Why it matters. Tax authorities, auditors and election observers in several countries use departures from Benford's law as a screening signal for invented numbers, because people who make up figures tend to spread the first digits too evenly. The same law appears in physical constants, river lengths, populations and stock prices. Testing it is a direct application of Pearson's goodness-of-fit test, and it shows the difficulties of very large samples.

What to do.

  1. Collect at least $500$ positive numbers spanning several orders of magnitude from a public source (for example, the populations of all towns of a country, or the areas of lakes), and record the leading digits.
  2. Compute the expected counts under Benford's law and Pearson's $X^2$ on $8$ degrees of freedom; report the $p$-value and the cells with the largest contributions.
  3. Ask ten classmates to write down $50$ "random-looking" invoice amounts each, and test these made-up data in the same way.
  4. Take random subsamples of your real data of sizes $100$, $1000$ and (if available) $10\,000$. Show that $X^2$ grows roughly in proportion to $n$ for a fixed small departure, and compute an effect size such as the mean absolute deviation of the digit proportions, which does not grow with $n$.

Tools: a spreadsheet or Python (scipy.stats.chisquare); Interactive 1 for intuition about the null distribution.

Hand in: A one-page report: the observed and expected digit table, $X^2$ and $p$ for real and invented data, and the effect of sample size on the decision.

Going further. Read why scale invariance implies Benford's law, and why data confined to a narrow range (adult heights, exam marks) do not follow it.

Instructor note: Benford's proportions are $0.301,\ 0.176,\ 0.125,\ 0.097,\ 0.079,\ 0.067,\ 0.058,\ 0.051,\ 0.046$, and $\chi^2_{8,0.05}=15.507$. Invented data usually fail badly (too few 1s, too many 5–9s). Real data with tens of thousands of records often "fail" the test too, because $X^2$ detects tiny departures; this is a good moment to discuss statistical versus practical significance.

🧪 Self-check quiz

0/5 correct

Q1. The chi-square test of independence in an $r\times c$ contingency table has degrees of freedom

Q2. A goodness-of-fit test uses $9$ classes after pooling and a normal model with mean and variance estimated from the grouped data. The degrees of freedom are

Q3. For a $2\times2$ table, Pearson's $X^2$ (no correction) equals

Q4. Under a fully specified null hypothesis with $k$ cells, the exact mean of Pearson's $X^2$ is

Q5. McNemar's test for paired yes/no data uses