5.2 Large-Sample Tests and Chi-Square Tests
Tests that work without exact distributions: large-sample $z$-tests for means, proportions and rates, the Wald, score and likelihood ratio trinity, Pearson's chi-square test of goodness of fit with the degrees-of-freedom rule for estimated parameters, contingency tables for independence and homogeneity, the $2\times2$ table with Yates' correction and Fisher's exact test, odds ratios and McNemar's test.
The exact tests of §5.1 need exact sampling distributions, and those exist mainly for normal samples. Much real data is not normal and much of it is not even numerical. An opinion poll records "yes" or "no"; a vaccine trial records infected or not; a geneticist counts offspring of four phenotypes; a hospital cross-classifies patients by treatment and outcome; a quality engineer counts flaws per sheet. For such data two ideas carry almost all the weight. The first is the large-sample $z$-test: by the central limit theorem and Slutsky's theorem (§2.3), almost any reasonable estimator is approximately normal, so (estimate − hypothesised value)/(standard error) is approximately $N(0,1)$ when $H_0$ holds. The second is Karl Pearson's chi-square statistic $X^2=\sum(O-E)^2/E$, which compares observed counts $O$ with the counts $E$ expected under $H_0$ and has an approximate $\chi^2$ law whose degrees of freedom are found by counting parameters, as Wilks' theorem (§5.1) predicts.
This section develops both ideas with proofs. We show why the standard $z$-tests have the right size in large samples and are consistent, and how the Wald, score and likelihood ratio versions are related. We prove Pearson's theorem through the multivariate CLT, explain why each estimated parameter costs one degree of freedom, and derive the tests of independence and homogeneity in contingency tables, with the special identities of the $2\times2$ table. For small samples we give Fisher's exact test, and for paired binary data McNemar's test. CSIR NET Part B questions ask for the value of $X^2$, its degrees of freedom or a $z$ statistic. Part C questions test the assumptions: when the $\chi^2$ approximation is reliable, how estimated parameters change the degrees of freedom, and which statements about $2\times2$ tables are identities. Notation follows §5.1. Upper points are $z_\alpha$ and $\chi^2_{\nu,\alpha}$; for reference, $\chi^2_{\nu,0.05}=3.841,\ 5.991,\ 7.815,\ 9.488,\ 11.070$ for $\nu=1,\dots,5$.
Large-Sample Tests for Means and Proportions
Let $\phi_n$ be a test based on a sample of size $n$, for $H_0:\theta\in\Theta_0$ against $H_1:\theta\in\Theta_1$.
- $\{\phi_n\}$ has asymptotic size $\alpha$ if $E_\theta\phi_n\to\alpha$ for every $\theta$ on the boundary of $\Theta_0$ that determines the size (for a simple null, at $\theta_0$), and $\limsup_nE_\theta\phi_n\le\alpha$ for every $\theta\in\Theta_0$.
- $\{\phi_n\}$ is consistent if $E_\theta\phi_n\to1$ for every $\theta\in\Theta_1$.
Let $T_n$ be a statistic and $\tau=g(\theta)$ a real parameter. Suppose that for every $\theta$, $$\sqrt n\left(T_n-g(\theta)\right)\xrightarrow dN\left(0,v(\theta)\right)$$ with $v(\theta)\gt0$, and that $\hat v_n$ is a statistic with $\hat v_n\xrightarrow Pv(\theta)$ (or at least $\hat v_n\xrightarrow{P}v(\theta_0)$ when $g(\theta)=\tau_0$). Put $Z_n=\sqrt n(T_n-\tau_0)/\sqrt{\hat v_n}$.
- If $g(\theta)=\tau_0$, then $Z_n\xrightarrow dN(0,1)$. Hence the tests "reject $H_0:\tau=\tau_0$ when $|Z_n|\gt z_{\alpha/2}$", "when $Z_n\gt z_\alpha$" and "when $Z_n\lt-z_\alpha$" have asymptotic size $\alpha$ (the one-sided ones also for $H_0:\tau\le\tau_0$, respectively $\tau\ge\tau_0$).
- If $g(\theta)\ne\tau_0$ and $\hat v_n$ converges in probability to a finite positive limit, the two-sided test is consistent, and so is each one-sided test against alternatives on its own side.
- (Local power.) If $\theta_n$ satisfies $\sqrt n(g(\theta_n)-\tau_0)\to\delta$ and the convergence in (1) holds along $\theta_n$, then the power of the right-sided test tends to $1-\Phi\left(z_\alpha-\delta/\sqrt{v(\theta_0)}\right)$.
Proof. (1) Write $Z_n=\dfrac{\sqrt n(T_n-\tau_0)}{\sqrt{v(\theta)}}\cdot\sqrt{\dfrac{v(\theta)}{\hat v_n}}$. The first factor converges in distribution to $N(0,1)$ and the second in probability to $1$ (continuous mapping), so Slutsky's theorem (§2.3) gives $Z_n\xrightarrow dN(0,1)$. Since $N(0,1)$ is continuous, $P(|Z_n|\gt z_{\alpha/2})\to\alpha$, and similarly for the one-sided tests. For $H_0:\tau\le\tau_0$ and $\theta$ with $g(\theta)\lt\tau_0$, $$Z_n=\frac{\sqrt n(T_n-g(\theta))}{\sqrt{\hat v_n}}+\frac{\sqrt n(g(\theta)-\tau_0)}{\sqrt{\hat v_n}},$$ where the first term is bounded in probability and the second tends to $-\infty$ in probability, so $P(Z_n\gt z_\alpha)\to0$.
(2) The same decomposition with $g(\theta)\ne\tau_0$ shows that $|Z_n|\to\infty$ in probability, with the sign of $g(\theta)-\tau_0$; so $P(|Z_n|\gt z_{\alpha/2})\to1$.
(3) $$Z_n=\frac{\sqrt n(T_n-g(\theta_n))}{\sqrt{\hat v_n}}+\frac{\sqrt n(g(\theta_n)-\tau_0)}{\sqrt{\hat v_n}},$$ where the first term tends to $N(0,1)$ and the second to $\delta/\sqrt{v(\theta_0)}$ in probability. So $Z_n\xrightarrow dN\left(\delta/\sqrt{v(\theta_0)},1\right)$ and $P(Z_n\gt z_\alpha)\to1-\Phi(z_\alpha-\delta/\sqrt{v(\theta_0)})$. ∎
Part (3) gives the sample size formula. To have power $1-\beta$ at the alternative $\tau_1$ with a one-sided level-$\alpha$ test, put $\delta=\sqrt n(\tau_1-\tau_0)$ and solve $z_\alpha-\delta/\sqrt v=-z_\beta$: $$n\approx\left(\frac{(z_\alpha+z_\beta)\sqrt v}{\tau_1-\tau_0}\right)^2.$$ For proportions it is more accurate to use the null variance in the $z_\alpha$ term and the alternative variance in the $z_\beta$ term (Exercise I.7).
Each statistic below is asymptotically $N(0,1)$ under the stated $H_0$, so it is used with the critical values $z_\alpha$ or $z_{\alpha/2}$. All samples are independent and iid within themselves.
| Problem | $H_0$ | Statistic |
|---|---|---|
| one mean, any population with finite variance | $\mu=\mu_0$ | $Z=\dfrac{\bar X-\mu_0}{S/\sqrt n}$ |
| two means | $\mu_1-\mu_2=\Delta_0$ | $Z=\dfrac{\bar X-\bar Y-\Delta_0}{\sqrt{S_1^2/m+S_2^2/n}}$ |
| one proportion, $X\sim\operatorname{Bin}(n,p)$ | $p=p_0$ | $Z=\dfrac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}$ |
| two proportions | $p_1=p_2$ | $Z=\dfrac{\hat p_1-\hat p_2}{\sqrt{\hat p(1-\hat p)\left(\frac1m+\frac1n\right)}}$, $\ \hat p=\dfrac{X_1+X_2}{m+n}$ |
| Poisson mean | $\lambda=\lambda_0$ | $Z=\dfrac{\bar X-\lambda_0}{\sqrt{\lambda_0/n}}$ |
| standard deviation (normal population) | $\sigma=\sigma_0$ | $Z=\dfrac{S-\sigma_0}{\sigma_0/\sqrt{2n}}$ |
| correlation (bivariate normal) | $\rho=\rho_0$ | $Z=\sqrt{n-3}\left(\operatorname{artanh}r-\operatorname{artanh}\rho_0\right)$ |
Proof. Each case is Theorem 5.11 with a specific $T_n$ and $\hat v_n$. One mean: CLT for $\bar X$ with $v=\sigma^2$, and $S^2\xrightarrow P\sigma^2$ by the weak law. Two means: $\bar X-\bar Y$ is a difference of independent asymptotically normal means, with variance $\sigma_1^2/m+\sigma_2^2/n$ estimated consistently; let $m,n\to\infty$ with $m/(m+n)\to\lambda\in(0,1)$. One proportion: $\hat p=X/n$ is a mean of Bernoulli variables, $v=p_0(1-p_0)$ is known under $H_0$, and no estimate is needed. Two proportions: under $H_0$, $p_1=p_2=p$ and the pooled $\hat p$ is consistent for $p$, so $\hat p(1-\hat p)(1/m+1/n)$ estimates the variance of $\hat p_1-\hat p_2$ consistently (in the sense of the ratio tending to $1$). Poisson: $\operatorname{Var}(X_i)=\lambda_0$ under $H_0$. Standard deviation: for normal samples $\sqrt n(S^2-\sigma^2)\xrightarrow dN(0,2\sigma^4)$ (§4.1), and the delta method with $g(t)=\sqrt t$, $g'(\sigma^2)=\frac1{2\sigma}$ gives $\sqrt n(S-\sigma)\xrightarrow dN(0,\sigma^2/2)$. Correlation: for bivariate normal samples $\sqrt n(r-\rho)\xrightarrow dN(0,(1-\rho^2)^2)$, and $\frac{d}{d\rho}\operatorname{artanh}\rho=\frac1{1-\rho^2}$, so Fisher's transformation $\operatorname{artanh}r=\frac12\log\frac{1+r}{1-r}$ is variance stabilising (§2.3), with variance about $1/n$; the refinement $1/(n-3)$ improves small samples. ∎
The one-proportion test uses the null standard error $\sqrt{p_0q_0/n}$ ($q_0=1-p_0$). This is the score test; inverting it gives the Wilson interval of §4.3. Replacing $p_0$ by $\hat p$ in the denominator gives the Wald test, which inverts to the Wald interval. Both are covered by the next theorem.
Let $X_1,\dots,X_n$ be iid from a regular one-parameter family (§4.2) with Fisher information $I(\theta)$, score $U_n(\theta)=\ell_n'(\theta)$ and MLE $\hat\theta_n$. For $H_0:\theta=\theta_0$ define $$W_n=nI(\hat\theta_n)(\hat\theta_n-\theta_0)^2,\qquad S_n=\frac{U_n(\theta_0)^2}{nI(\theta_0)},\qquad\Lambda_n^*=-2\log\Lambda_n.$$ Under $H_0$ all three converge in distribution to $\chi^2_1$, and the differences between any two of them tend to $0$ in probability.
Proof. Under $H_0$, $U_n(\theta_0)/\sqrt{nI(\theta_0)}\xrightarrow dN(0,1)$ by the CLT, since the score has mean $0$ and variance $I(\theta_0)$ per observation (§4.2); so $S_n\xrightarrow d\chi^2_1$. Expanding $0=U_n(\hat\theta_n)=U_n(\theta_0)+\ell_n''(\theta_n^*)(\hat\theta_n-\theta_0)$ gives $$\sqrt n(\hat\theta_n-\theta_0)=\frac{U_n(\theta_0)/\sqrt n}{-\ell_n''(\theta_n^*)/n},$$ which equals $\dfrac{U_n(\theta_0)}{\sqrt n\,I(\theta_0)}+o_P(1)$ because $-\ell_n''(\theta_n^*)/n\xrightarrow PI(\theta_0)$. Squaring and multiplying by $I(\hat\theta_n)\xrightarrow PI(\theta_0)$ shows $W_n=S_n+o_P(1)$. Finally the proof of Wilks' theorem (§5.1) gave $$\Lambda_n^*=\left(-\tfrac1n\ell_n''(\theta_n^{**})\right)n(\hat\theta_n-\theta_0)^2,$$ which is $W_n+o_P(1)$. ∎
For a binomial proportion, $S_n$ is the square of the score $z$ of Theorem 5.12 and $W_n$ the square of the Wald $z$. The score test needs only the null value, the Wald test only the MLE, and the likelihood ratio test both. In small samples they can disagree, and the score and likelihood ratio versions are usually more reliable.
Pearson's Chi-Square Test of Goodness of Fit
Each of $n$ independent observations falls into exactly one of $k$ cells, cell $j$ having probability $p_j$; the cell counts $(N_1,\dots,N_k)$ have the multinomial law $\operatorname{Mult}(n;p_1,\dots,p_k)$. For a fully specified null hypothesis $H_0:p_j=\pi_j$ ($j=1,\dots,k$, $\pi_j\gt0$, $\sum\pi_j=1$), the expected frequencies are $E_j=n\pi_j$, and
$$X^2=\sum_{j=1}^k\frac{(N_j-E_j)^2}{E_j}=\sum_{j=1}^k\frac{N_j^2}{E_j}-n,$$ $$G^2=2\sum_{j=1}^kN_j\log\frac{N_j}{E_j}\quad(0\log0=0).$$
$X^2$ is Pearson's chi-square statistic and $G^2=-2\log\Lambda$ is the likelihood ratio statistic (the deviance), since the unrestricted multinomial MLE is $\hat p_j=N_j/n$.
Under $H_0:p=\pi$:
- $E[X^2]=k-1$ exactly, for every $n$;
- $X^2\xrightarrow d\chi^2_{k-1}$ as $n\to\infty$;
- $G^2-X^2\xrightarrow P0$, so $G^2\xrightarrow d\chi^2_{k-1}$ as well.
Under a fixed alternative $p\ne\pi$, $X^2/n\xrightarrow P\sum_j(p_j-\pi_j)^2/\pi_j\gt0$, so the test that rejects when $X^2\gt\chi^2_{k-1,\alpha}$ is consistent.
Proof. (1) $N_j\sim\operatorname{Bin}(n,\pi_j)$, so $E(N_j-n\pi_j)^2=n\pi_j(1-\pi_j)$ and $E[X^2]=\sum_j(1-\pi_j)=k-1$.
(2) Write $N=\sum_{i=1}^n\mathbf e_{(i)}$, where $\mathbf e_{(i)}$ is the indicator vector of the cell of observation $i$: iid vectors with mean $\pi$ and covariance $\Sigma=\operatorname{diag}(\pi)-\pi\pi^{\mathsf T}$ (because $\operatorname{Cov}(N_i,N_j)=-n\pi_i\pi_j$ for $i\ne j$). By the multivariate CLT, $(N-n\pi)/\sqrt n\xrightarrow dN_k(0,\Sigma)$. Put $D=\operatorname{diag}(\pi)^{-1/2}$ and $Y_n=D(N-n\pi)/\sqrt n$, the vector with entries $(N_j-n\pi_j)/\sqrt{n\pi_j}$; then $X^2=\|Y_n\|^2$ and $Y_n\xrightarrow dY\sim N_k(0,D\Sigma D)$ with $$D\Sigma D=I_k-\sqrt\pi\sqrt\pi^{\mathsf T},\qquad\sqrt\pi=(\sqrt{\pi_1},\dots,\sqrt{\pi_k})^{\mathsf T}.$$ Since $\|\sqrt\pi\|^2=\sum\pi_j=1$, the matrix $P=I_k-\sqrt\pi\sqrt\pi^{\mathsf T}$ is the orthogonal projection onto the hyperplane $\sqrt\pi^{\perp}$, of rank $k-1$. Choose an orthonormal basis $q_1,\dots,q_{k-1}$ of that hyperplane and let $Q=[q_1\cdots q_{k-1}]$, so $P=QQ^{\mathsf T}$. If $Z\sim N_{k-1}(0,I)$ then $QZ\sim N_k(0,QQ^{\mathsf T})=N_k(0,P)$, so $Y$ has the law of $QZ$ and $\|Y\|^2$ the law of $\|QZ\|^2=\|Z\|^2\sim\chi^2_{k-1}$. By the continuous mapping theorem, $X^2=\|Y_n\|^2\xrightarrow d\chi^2_{k-1}$.
(3) Put $N_j=E_j+\Delta_j$, where $\Delta_j=O_P(\sqrt n)$ and $\sum\Delta_j=0$. Taylor's formula $(1+u)\log(1+u)=u+\frac{u^2}2-\frac{u^3}{6}+\cdots$ with $u=\Delta_j/E_j=O_P(n^{-1/2})$ gives $$N_j\log\frac{N_j}{E_j}=E_j(1+u)\log(1+u)=\Delta_j+\frac{\Delta_j^2}{2E_j}+O_P(n^{-1/2}).$$ Summing, the $\Delta_j$ cancel and $G^2=X^2+O_P(n^{-1/2})$.
Consistency. By the weak law, $N_j/n\xrightarrow Pp_j$, so $$\frac{X^2}{n}=\sum_j\frac{(N_j/n-\pi_j)^2}{\pi_j}\xrightarrow{P}\sum_j\frac{(p_j-\pi_j)^2}{\pi_j}=c\gt0.$$ Then $P(X^2\gt\chi^2_{k-1,\alpha})=P(X^2/n\gt\chi^2_{k-1,\alpha}/n)\to1$. ∎
For $k=2$ the theorem is the de Moivre–Laplace theorem in disguise: $N_2=n-N_1$ and $X^2=\frac{(N_1-n\pi_1)^2}{n\pi_1}+\frac{(N_1-n\pi_1)^2}{n\pi_2}=\frac{(N_1-n\pi_1)^2}{n\pi_1\pi_2}$, which is the square of the score $z$ for a proportion (Exercise II.2).
- Cells. Decide the $k$ categories (for continuous data, class intervals fixed before looking at the data).
- Expected counts. Compute $E_j=n\pi_j$, estimating any unknown parameters first (Theorem 5.15).
- Pool small cells. The $\chi^2$ approximation is reliable when every $E_j\ge5$ (a conservative rule; most $E_j\ge5$ and none below $1$ is often enough). Merge adjacent cells until this holds.
- Degrees of freedom. $\nu=(\text{number of cells after pooling})-1-(\text{number of estimated parameters})$.
- Decide. Reject $H_0$ when $X^2\gt\chi^2_{\nu,\alpha}$. The test is one-sided: large $X^2$ means a poor fit. An unusually small $X^2$ is suspicious too ("too good to be true"; Exercise II.6).
Let $H_0:p=\pi(\theta)$, where $\theta\in\Theta\subseteq\mathbb{R}^s$ ($s\lt k-1$) is unknown, $\pi$ is twice continuously differentiable with $\pi_j(\theta)\gt0$, and the $k\times s$ matrix $\partial\pi/\partial\theta$ has rank $s$. Let $\hat\theta$ be the MLE computed from the cell counts $N_1,\dots,N_k$ (or the minimum-$\chi^2$ estimator). Then, under $H_0$, $$X^2=\sum_{j=1}^k\frac{\left(N_j-n\pi_j(\hat\theta)\right)^2}{n\pi_j(\hat\theta)}\xrightarrow d\chi^2_{k-1-s},$$ and the same holds for $G^2$.
Proof (sketch). $G^2=-2\log\Lambda$ for $H_0$ inside the full multinomial model. The full model has $k-1$ free parameters, the null model has $s$, and under the stated smoothness and rank conditions Wilks' theorem (§5.1), in the version for a smoothly parametrised null set, gives $G^2\xrightarrow d\chi^2_{k-1-s}$. Part (3) of Theorem 5.14 extends to show $G^2-X^2\xrightarrow P0$. Geometrically, in the proof of Theorem 5.14 the estimated parameters project $Y$ onto a further $s$-dimensional subspace, which removes $s$ dimensions from the $\chi^2$. ∎
The theorem requires $\hat\theta$ to be computed from the grouped counts. In practice parameters are often estimated from the raw data (for instance $\bar x$ for a Poisson mean computed before pooling). Then the limiting law of $X^2$ lies between $\chi^2_{k-1-s}$ and $\chi^2_{k-1}$ (a result of Chernoff and Lehmann), so the test with $k-1-s$ degrees of freedom is slightly anti-conservative. With moderate $k$ the difference is small, and the $k-1-s$ rule is the one used in examinations.
Contingency Tables: Independence and Homogeneity
When each individual is classified by two categorical attributes, such as treatment and outcome or education and opinion, the counts form a contingency table. Two questions lead to the same arithmetic but come from different sampling schemes.
An $r\times c$ table has counts $N_{ij}$, row totals $R_i=\sum_jN_{ij}$, column totals $C_j=\sum_iN_{ij}$ and grand total $n$.
- Independence (one sample, two attributes): $n$ individuals are drawn at random and classified by both attributes, so $(N_{ij})\sim\operatorname{Mult}(n;(p_{ij}))$. The hypothesis is $H_0:p_{ij}=p_{i\cdot}\,p_{\cdot j}$ for all $i,j$, where $p_{i\cdot}=\sum_jp_{ij}$ and $p_{\cdot j}=\sum_ip_{ij}$.
- Homogeneity ($r$ samples, one attribute): the row totals $R_i$ are fixed by design, row $i$ is a sample from population $i$, and $(N_{i1},\dots,N_{ic})\sim\operatorname{Mult}(R_i;p_{1|i},\dots,p_{c|i})$ independently over $i$. The hypothesis is $H_0:p_{j|1}=\cdots=p_{j|r}$ for every $j$: the populations have the same distribution over the columns.
In both models of Definition 5.11, the maximum likelihood estimates of the expected counts under $H_0$ are $$\hat E_{ij}=\frac{R_iC_j}{n},$$ and under $H_0$, as $n\to\infty$ (with all cell probabilities positive, and in the homogeneity model with every $R_i\to\infty$), $$X^2=\sum_{i=1}^r\sum_{j=1}^c\frac{(N_{ij}-\hat E_{ij})^2}{\hat E_{ij}}\xrightarrow d\chi^2_{(r-1)(c-1)},$$ and the same holds for $G^2=2\sum N_{ij}\log(N_{ij}/\hat E_{ij})$.
Proof. Independence. Under $H_0$ write $p_{ij}=a_ib_j$ with $\sum a_i=\sum b_j=1$. The log-likelihood is $\sum_{i,j}N_{ij}\log(a_ib_j)=\sum_iR_i\log a_i+\sum_jC_j\log b_j$. Maximising $\sum R_i\log a_i$ subject to $\sum a_i=1$ (a Lagrange multiplier, or Gibbs' inequality) gives $\hat a_i=R_i/n$; similarly $\hat b_j=C_j/n$. So $\hat E_{ij}=n\hat a_i\hat b_j=R_iC_j/n$. The full model has $rc-1$ free parameters and the null model $(r-1)+(c-1)$, so by Theorem 5.15 the limit is $\chi^2$ with $$rc-1-(r-1)-(c-1)=(r-1)(c-1)$$ degrees of freedom.
Homogeneity. Under $H_0$ the common column distribution $(q_j)$ has log-likelihood $\sum_i\sum_jN_{ij}\log q_j=\sum_jC_j\log q_j$, maximised at $\hat q_j=C_j/n$; hence $\hat E_{ij}=R_i\hat q_j=R_iC_j/n$. The full model has $r(c-1)$ free parameters (one probability vector per row) and the null model $c-1$, and $r(c-1)-(c-1)=(r-1)(c-1)$. Wilks' theorem for independent multinomial samples gives the limit. ∎
So the two hypotheses, although different in meaning, lead to the same statistic, the same expected counts and the same degrees of freedom. What differs is the conclusion: "the attributes are associated" against "the populations differ".
| Agree | Neutral | Disagree | Total | |
|---|---|---|---|---|
| School | 45 | 30 | 25 | 100 |
| Graduate | 40 | 35 | 45 | 120 |
| Postgraduate | 20 | 15 | 45 | 80 |
| Total | 105 | 80 | 115 | 300 |
For a $2\times2$ table with cells $a,b$ (row 1), $c,d$ (row 2), row totals $R_1,R_2$, column totals $C_1,C_2$ and total $n$:
- every cell has $|O-E|=|ad-bc|/n$, and $$X^2=\frac{n(ad-bc)^2}{R_1R_2C_1C_2};$$
- $X^2=Z^2$, where $Z$ is the pooled two-proportion statistic of Theorem 5.12 for $\hat p_1=a/R_1$ against $\hat p_2=c/R_2$;
- $X^2\xrightarrow d\chi^2_1$ under $H_0$. Yates' continuity correction replaces $|ad-bc|$ by $|ad-bc|-n/2$: $$X^2_{\mathrm{Y}}=\frac{n\left(|ad-bc|-n/2\right)^2}{R_1R_2C_1C_2},$$ which brings the $\chi^2_1$ tail closer to the exact (Fisher) $p$-value when counts are small.
Proof. (1) $a-\frac{R_1C_1}{n}=\frac{a(a+b+c+d)-(a+b)(a+c)}{n}=\frac{ad-bc}{n}$, and since the margins of $O$ and $E$ agree, the other three deviations are $\pm\frac{ad-bc}{n}$. Hence $$X^2=\frac{(ad-bc)^2}{n^2}\sum_{i,j}\frac{n}{R_iC_j}=\frac{(ad-bc)^2}{n}\left(\frac1{R_1}+\frac1{R_2}\right)\left(\frac1{C_1}+\frac1{C_2}\right),$$ and $\left(\frac1{R_1}+\frac1{R_2}\right)\left(\frac1{C_1}+\frac1{C_2}\right)=\frac{n}{R_1R_2}\cdot\frac{n}{C_1C_2}$.
(2) $\hat p_1-\hat p_2=\frac{a}{R_1}-\frac{c}{R_2}=\frac{a(c+d)-c(a+b)}{R_1R_2}=\frac{ad-bc}{R_1R_2}$, the pooled proportion is $\hat p=C_1/n$ and $1-\hat p=C_2/n$. So $$Z^2=\frac{(ad-bc)^2/(R_1R_2)^2}{\frac{C_1C_2}{n^2}\cdot\frac{n}{R_1R_2}}=\frac{n(ad-bc)^2}{R_1R_2C_1C_2}=X^2.$$
(3) is Theorem 5.16 with $r=c=2$. The correction subtracts half a unit from each $|O-E|$, the analogue of the continuity correction for the binomial (§2.3); we do not prove its accuracy. ∎
Exact and Paired Tests for Discrete Data
In a $2\times2$ table under either the homogeneity model with $H_0:p_1=p_2$ or the independence model with $H_0$ of independence, the conditional distribution of $A$ (the count $a$) given all the margins is hypergeometric: $$P(A=a\mid R_1,R_2,C_1)=\frac{\binom{R_1}{a}\binom{R_2}{C_1-a}}{\binom{n}{C_1}},\qquad\max(0,C_1-R_2)\le a\le\min(R_1,C_1).$$ It does not depend on the unknown common probability. Fisher's exact test computes $p$-values from this law: one-sided $P(A\ge a_{\mathrm{obs}})$, and two-sided the total probability of all $a$ with $P(A=a)\le P(A=a_{\mathrm{obs}})$.
Proof. In the homogeneity model, under $H_0$ the two rows are independent $\operatorname{Bin}(R_1,p)$ and $\operatorname{Bin}(R_2,p)$ in their first column. So $$P(A=a,\ C=C_1-a)=\binom{R_1}{a}\binom{R_2}{C_1-a}p^{C_1}(1-p)^{n-C_1}.$$ Summing over $a$ and using Vandermonde's identity $\sum_a\binom{R_1}{a}\binom{R_2}{C_1-a}=\binom n{C_1}$ gives $P(A+C=C_1)=\binom n{C_1}p^{C_1}(1-p)^{n-C_1}$, and dividing gives the hypergeometric law: $p$ cancels. In the independence model, conditioning first on the row totals reduces to the homogeneity model (given $R_1,R_2$, the rows are independent binomials with success probability $p_{\cdot1}$ under independence). ∎
The column total $C_1$ is sufficient for the nuisance parameter $p$ under $H_0$, and conditioning on a sufficient statistic removes it. This is the general idea behind exact conditional tests. Because the conditional law is discrete, the test is conservative: its actual size is usually below the nominal $\alpha$.
- Log odds ratio (Woolf). For two independent binomial samples with all cell counts large, $\log\hat\theta$ is approximately normal with mean $\log\theta$ and estimated standard error $\sqrt{\frac1a+\frac1b+\frac1c+\frac1d}$; so $H_0:\theta=1$ can be tested by $Z=\log\hat\theta\big/\sqrt{\frac1a+\frac1b+\frac1c+\frac1d}$.
- McNemar's test. $n$ pairs are each classified (yes/no) twice, for instance before and after a treatment. Let $b$ be the number of pairs (yes, no) and $c$ the number (no, yes). Under $H_0$ of marginal homogeneity, $P(\text{yes first})=P(\text{yes second})$, conditionally on $b+c=m$ we have $b\sim\operatorname{Bin}(m,\frac12)$, and $$X^2_{\mathrm M}=\frac{(b-c)^2}{b+c}\xrightarrow d\chi^2_1\quad(m\to\infty).$$
- Index of dispersion. If $X_1,\dots,X_n$ are iid $\operatorname{Poi}(\lambda)$, then given $T=\sum X_i=t$ the vector $(X_1,\dots,X_n)$ is $\operatorname{Mult}(t;\frac1n,\dots,\frac1n)$, and the Pearson statistic of this multinomial is $$D=\frac{\sum_{i=1}^n(X_i-\bar X)^2}{\bar X}\approx\chi^2_{n-1}.$$ Large $D$ indicates over-dispersion (variance larger than the mean), small $D$ under-dispersion.
Proof. (1) For row 1, $\hat p_1=a/R_1$ and $g(p)=\log\frac p{1-p}$ has $g'(p)=\frac1{p(1-p)}$; by the delta method $\log\frac{a}{b}=g(\hat p_1)$ has approximate variance $\frac{p_1(1-p_1)}{R_1}\cdot\frac1{p_1^2(1-p_1)^2}=\frac1{R_1p_1}+\frac1{R_1(1-p_1)}$, estimated by $\frac1a+\frac1b$. Row 2 is independent and gives $\frac1c+\frac1d$, and $\log\hat\theta=\log\frac ab-\log\frac cd$.
(2) Let $p_{10},p_{01}$ be the probabilities of the discordant outcomes (yes, no) and (no, yes). Marginal homogeneity is $p_{11}+p_{10}=p_{11}+p_{01}$, i.e. $p_{10}=p_{01}$. Given that $m$ pairs are discordant, each is of type (yes, no) independently with probability $\frac{p_{10}}{p_{10}+p_{01}}=\frac12$. So $b\sim\operatorname{Bin}(m,\frac12)$, $\frac{b-m/2}{\sqrt{m/4}}=\frac{b-c}{\sqrt{b+c}}\xrightarrow dN(0,1)$, and squaring gives $X^2_{\mathrm M}$. The concordant pairs carry no information about $H_0$.
(3) For $\sum x_i=t$, $$P(X=x\mid T=t)=\frac{\prod e^{-\lambda}\lambda^{x_i}/x_i!}{e^{-n\lambda}(n\lambda)^t/t!}=\frac{t!}{\prod x_i!}\left(\frac1n\right)^t,$$ the uniform multinomial. Its expected counts are $t/n=\bar x$, so Pearson's statistic is $\sum(X_i-\bar X)^2/\bar X=D$, and Theorem 5.14 (with $k=n$ cells, as $t\to\infty$) gives the $\chi^2_{n-1}$ approximation. ∎
- Large expected counts, not large observed counts. The $\chi^2$ approximation needs the $E_j$ to be reasonably large (rule of thumb $\ge5$). Pool cells, and count the degrees of freedom after pooling (Example 5).
- Degrees of freedom. Subtract one for each parameter estimated from the data, not for parameters fixed by $H_0$. For a contingency table it is $(r-1)(c-1)$, not $rc-1$.
- The test is upper-tailed. Only large $X^2$ leads to rejection, even though the alternative is "any departure". A tiny $X^2$ (for example $0.08$ on $3$ df, left-tail probability $0.006$) suggests data that fit too well (Exercise II.6).
- Percentages are not counts. $X^2$ is proportional to $n$ for fixed proportions: computing it from percentages, or from counts doubled, gives a wrong value.
- Large $n$ finds trivial departures; small $n$ finds nothing. Non-rejection is not proof of fit, and rejection is not proof of an important departure.
- Paired data are not independent samples. Before/after proportions on the same subjects must be compared with McNemar's test, not the two-proportion $z$-test.
- Pooled, not unpooled. $X^2$ for a $2\times2$ table equals $Z^2$ with the pooled standard error (Theorem 5.17), not the unpooled one.
- Collapsing tables can reverse an association (Simpson's paradox). A treatment can be better in every stratum and worse in the combined table (Exercise III.6).
- Yates' correction is for $1$ degree of freedom. It applies to $2\times2$ tables; it makes the test conservative, and it is not used for larger tables.
- Wald fails at the boundary. With $\hat p=0$ the Wald standard error is $0$ and the Wald statistic is infinite or undefined; the score test, which uses $p_0$, is fine.
- Independence versus homogeneity. They have the same statistic and degrees of freedom but different sampling schemes and different conclusions; with fixed row totals one cannot estimate the row proportions $p_{i\cdot}$.
- Two-sided exact $p$-values are not always double the one-sided ones. With unequal margins the hypergeometric law is skewed (Exercise IV.7).
Exercises
- 1 In a sample of $1000$ voters, $540$ favour a candidate. Test $H_0:p=0.5$ against $H_1:p\gt0.5$ at $1\%$.
- 2 A coin is tossed $400$ times and shows $220$ heads. Test fairness (two-sided, $5\%$) with and without the continuity correction. Comment.
- 3 Smokers: $140$ of $400$ men and $140$ of $500$ women. Test $p_{\text{men}}=p_{\text{women}}$ at $5\%$.
- 4 A sample of $64$ observations from a skewed population gives $\bar x=52.3$, $s=6.4$. Test $\mu=50$ (two-sided).
- 5 A road averaged $2$ accidents per month. In the last $12$ months it had $36$. Assuming Poisson counts, test for an increase with the large-sample test, and compare with the exact Poisson $p$-value.
- 6 Site A recorded $45$ events in $100$ days and site B $30$ events in $100$ days (Poisson). Test equality of the rates (two-sided). Show that the test based on $A$ given $A+B=75$ leads to the same $z$.
- 7 How many voters must be polled so that the one-sided $5\%$ test of $p=0.5$ has power $0.9$ when $p=0.55$? Use the null variance for $z_\alpha$ and the alternative variance for $z_\beta$.
- 8 Prove that the one-proportion score test is consistent: if $p\ne p_0$, then $P(|Z_n|\gt z_{\alpha/2})\to1$.
- 9 From $n=39$ pairs (bivariate normal), $r=0.42$. Test $H_0:\rho=0.25$ with Fisher's $z$.
- 1 A die rolled $120$ times gives the counts $25,17,15,23,24,16$. Test fairness at $5\%$ with $X^2$ and with $G^2$.
- 2 Show that for $k=2$ cells $X^2$ is the square of the score statistic $\frac{N_1-n\pi_1}{\sqrt{n\pi_1(1-\pi_1)}}$.
- 3 Show that $X^2=\sum N_j^2/E_j-n$ when $\sum E_j=n$.
- 4 The numbers of girls in $100$ families with four children are: $0$ girls in $8$ families, $1$ in $28$, $2$ in $35$, $3$ in $22$, $4$ in $7$. Fit a binomial law with $p$ estimated and test the fit.
- 5 A normal distribution with both parameters estimated is fitted to grouped data in $8$ classes; the two classes at each end have expected counts below $5$ and are merged with their neighbours. How many degrees of freedom does the $X^2$ test have?
- 6 A student reports $25,24,26,25$ in four equally likely categories. Compute $X^2$ and $P(\chi^2_3\le X^2)$. What might you suspect?
- 7 Show that multiplying every count by $m$ multiplies $X^2$ by $m$. If the proportions of part 1 were observed in $120m$ rolls, what is the smallest $m$ for which fairness is rejected at $5\%$?
- 8 A random digit generator produced $1000$ digits with frequencies $92,108,97,115,89,102,95,110,87,105$ for $0,\dots,9$. Test uniformity at $5\%$.
- 1 For the table with rows $(30,20)$ and $(15,35)$ compute $X^2$ by the formula of Theorem 5.17, the pooled $Z$, and Yates' $X^2_{\mathrm Y}$.
- 2 Using Lagrange multipliers, show that $\hat a_i=R_i/n$ maximises $\sum_iR_i\log a_i$ subject to $\sum a_i=1$.
- 3 Three treatments are given to $50$ patients each. Recovered: $40,35,25$; not recovered: $10,15,25$. Test homogeneity at $5\%$.
- 4 State the degrees of freedom for testing independence in a $3\times4$ table and homogeneity of $5$ binomial populations.
- 5 Explain why a researcher who computes $X^2$ for a $2\times3$ table from percentages of row totals, instead of counts, gets a wrong answer.
- 6 Construct an example of Simpson's paradox: in each of two strata a treatment has the higher success rate, but in the combined table it has the lower rate.
- 7 Prove that in a $2\times2$ table $X^2=0$ iff $ad=bc$ iff $\hat p_1=\hat p_2$.
- 8 A $2\times2$ table with $n=20$ has smallest expected count $2.4$. Which test should be used, and why?
- 1 A table has rows $(7,1)$ and $(2,6)$. Compute the one- and two-sided Fisher exact $p$-values.
- 2 $200$ voters were asked before and after a debate. $25$ switched from "yes" to "no" and $10$ from "no" to "yes". Test for a change with McNemar's test, with and without the continuity correction $(|b-c|-1)^2/(b+c)$, and exactly.
- 3 For the table of Exercise III.1 estimate the odds ratio and give a $95\%$ confidence interval.
- 4 Ten Poisson counts are $3,7,2,9,4,6,1,8,5,5$. Test for over-dispersion with the index of dispersion.
- 5 In McNemar's set-up, show that marginal homogeneity holds iff $p_{10}=p_{01}$, and that the concordant pairs do not enter the test.
- 6 Prove that, given $\sum X_i=t$, iid Poisson counts $X_1,\dots,X_n$ are $\operatorname{Mult}(t;\frac1n,\dots,\frac1n)$.
- 7 For the table with rows $(5,0)$ and $(1,14)$, compute the one- and two-sided Fisher $p$-values. Why are they equal?
E1. CSIR Part B GATE ST In a $2\times2$ contingency table the cells are $20,30$ (first row) and $30,20$ (second row). Pearson's $X^2$ statistic for independence (without continuity correction) equals (A) $1.0$; (B) $2.0$; (C) $4.0$; (D) $8.0$.
E2. CSIR Part B A Poisson distribution, with its mean estimated from the grouped data, is fitted to frequency data. After pooling, $7$ classes remain. The degrees of freedom of the chi-square goodness-of-fit statistic are (A) $7$; (B) $6$; (C) $5$; (D) $4$.
E3. CSIR Part B A coin is tossed $400$ times and shows $220$ heads. The large-sample statistic $Z=(\hat p-p_0)/\sqrt{p_0(1-p_0)/n}$ for $H_0:p=\frac12$ equals (A) $1.0$; (B) $2.0$; (C) $2.2$; (D) $4.0$.
E4. CSIR Part C Let $(N_1,\dots,N_k)\sim\operatorname{Mult}(n;p_1,\dots,p_k)$ and let $X^2=\sum(N_j-n\pi_j)^2/(n\pi_j)$ for a fully specified $\pi$ with all $\pi_j\gt0$. Which of the following are true? (A) Under $H_0:p=\pi$, $E[X^2]=k-1$ for every $n$. (B) $X^2=0$ if and only if $N_j=n\pi_j$ for every $j$. (C) Under $H_0$, $X^2$ has exactly the $\chi^2_{k-1}$ distribution for every $n$. (D) If $p\ne\pi$ is fixed, $P(X^2\gt\chi^2_{k-1,\alpha})\to1$ as $n\to\infty$.
E5. CSIR Part C For a $2\times2$ table with cells $a,b,c,d$ and all margins positive, which of the following are true? (A) Pearson's $X^2$ equals $Z^2$, where $Z$ is the two-proportion statistic with the pooled standard error. (B) The test of independence (one multinomial sample) and the test of homogeneity (fixed row totals) use the same statistic and the same degrees of freedom. (C) If $X^2$ is significant, then $ad/(bc)\gt1$. (D) Fisher's exact test uses the binomial distribution of $a$ given the first row total only.
E6. CSIR Part C GATE ST Let $X_1\sim\operatorname{Bin}(m,p_1)$ and $X_2\sim\operatorname{Bin}(n,p_2)$ be independent, $m,n\to\infty$ with $m/(m+n)\to\lambda\in(0,1)$, and let $Z$ be the pooled two-proportion statistic. Which of the following are true? (A) Under $H_0:p_1=p_2$, $Z\xrightarrow dN(0,1)$. (B) $Z$ remains asymptotically $N(0,1)$ under $H_0$ if the two "samples" are the same subjects measured twice (paired data). (C) For fixed $p_1\ne p_2$, the power of the test that rejects when $|Z|\gt z_{\alpha/2}$ tends to $1$. (D) Under $H_0$, $Z$ has the $t_{m+n-2}$ distribution.
Suggested plan — 3 lectures × 60 min.
- Large-sample tests. Theorem 5.11 with the Slutsky argument on the board, then the table of Theorem 5.12, proving the two-proportion and Poisson cases. Examples 1–3; in Example 1 contrast the one- and two-sided decisions. Theorem 5.13 (trinity) in ten minutes, with the poll numbers.
- Goodness of fit. Definition 5.10, Theorem 5.14 (prove (1) and the case $k=2$ in full; give the projection argument for general $k$). Project Interactive 1: first the fair die with $n=30$, then the skewed model with $n=10$, so that the class sees when the approximation fails. Examples 4 and 5; state Theorem 5.15 and practise counting degrees of freedom.
- Contingency tables and discrete data. Theorem 5.16 with the parameter count, Example 6, Theorem 5.17 and Example 7; project Interactive 2 with the margins of Example 7 and then with small margins ($R_1=R_2=6$). Fisher's test, odds ratio and McNemar (Theorems 5.18–5.19). Finish with E4 and E5 live.
Discussion questions. (1) Why is the chi-square test upper-tailed although the alternative is two-sided in spirit? What would a lower-tail rejection mean? (2) In Example 7 the uncorrected test rejects and the corrected and exact tests do not. Which would you report, and why? (3) A table shows that hospitals with more specialists have higher death rates. What third variable could explain this, and how would you check?
Misconceptions to watch for. Using observed instead of expected counts in the denominator; counting degrees of freedom before pooling; subtracting degrees of freedom for parameters fixed by $H_0$; computing $X^2$ from percentages; applying the two-proportion test to paired data; believing that a significant $X^2$ shows the direction or the size of an association.
Suggested homework. I(2, 3, 5, 7), II(1, 4, 6, 7), III(1, 3, 6, 7), IV(1, 2, 4); Exam corner E2, E4, E6.
Research question. Do the leading digits of real-world numbers follow Benford's law, $P(D=d)=\log_{10}\left(1+\frac1d\right)$, $d=1,\dots,9$? And can a chi-square test of this law help to detect fabricated figures?
Why it matters. Tax authorities, auditors and election observers in several countries use departures from Benford's law as a screening signal for invented numbers, because people who make up figures tend to spread the first digits too evenly. The same law appears in physical constants, river lengths, populations and stock prices. Testing it is a direct application of Pearson's goodness-of-fit test, and it shows the difficulties of very large samples.
What to do.
- Collect at least $500$ positive numbers spanning several orders of magnitude from a public source (for example, the populations of all towns of a country, or the areas of lakes), and record the leading digits.
- Compute the expected counts under Benford's law and Pearson's $X^2$ on $8$ degrees of freedom; report the $p$-value and the cells with the largest contributions.
- Ask ten classmates to write down $50$ "random-looking" invoice amounts each, and test these made-up data in the same way.
- Take random subsamples of your real data of sizes $100$, $1000$ and (if available) $10\,000$. Show that $X^2$ grows roughly in proportion to $n$ for a fixed small departure, and compute an effect size such as the mean absolute deviation of the digit proportions, which does not grow with $n$.
Tools: a spreadsheet or Python (scipy.stats.chisquare); Interactive 1 for intuition about the null distribution.
Hand in: A one-page report: the observed and expected digit table, $X^2$ and $p$ for real and invented data, and the effect of sample size on the decision.
Going further. Read why scale invariance implies Benford's law, and why data confined to a narrow range (adult heights, exam marks) do not follow it.
🧪 Self-check quiz
Q1. The chi-square test of independence in an $r\times c$ contingency table has degrees of freedom
Q2. A goodness-of-fit test uses $9$ classes after pooling and a normal model with mean and variance estimated from the grouped data. The degrees of freedom are
Q3. For a $2\times2$ table, Pearson's $X^2$ (no correction) equals
Q4. Under a fully specified null hypothesis with $k$ cells, the exact mean of Pearson's $X^2$ is
Q5. McNemar's test for paired yes/no data uses