Chapter 5 · Introduction

Chapter 5 — Testing of Hypotheses

From estimates to decisions: how to test a claim about a population with a controlled risk of error, which tests are optimal and why, and how to test counts, proportions, tables and whole distributions when exact theory is not available.

Chapter 4 ended with intervals: ranges of plausible values for an unknown parameter. Very often, though, the question asked is a yes-or-no one. Does the new tablet line meet the declared mean weight of $500$ mg? Does the vaccine reduce infections, or was the difference between the arms of the trial a fluke? Do offspring counts agree with a genetic model? Are opinion and education independent? Each question concerns a claim about the population, the null hypothesis, and the data can only ever make the claim more or less believable. Any rule that turns data into a verdict will sometimes reject a true claim and sometimes accept a false one. The theory of testing accepts this and asks for the rule that keeps the first kind of error below a fixed level $\alpha$ and makes the second kind as rare as possible.

Section 5.1 builds that theory. It defines size, power and $p$-values, proves the Neyman–Pearson lemma, which says that the best test of one distribution against another rejects when the likelihood ratio is large, and extends the lemma to uniformly most powerful one-sided tests in families with a monotone likelihood ratio. It shows why two-sided UMP tests usually do not exist, and introduces the likelihood ratio test, which yields the familiar $z$-, $t$-, $\chi^2$- and $F$-tests and has the approximate $\chi^2$ law of Wilks' theorem. Section 5.2 handles large samples and counted data: $z$-tests for means, proportions and rates, Pearson's chi-square test of goodness of fit, contingency tables, and exact and paired tests for $2\times2$ tables. Section 5.3 drops distributional assumptions altogether and tests with signs and ranks.

Hypotheses & errors→Size, power, p-value→Neyman–Pearson lemma→MLR & UMP tests→Unbiased tests→Likelihood ratio & Wilks→Large-sample z-tests→Chi-square goodness of fit→Contingency tables→Sign & rank tests
🎯 By the end of this chapter you can…
  • set up null and alternative hypotheses, and compute the size, the power function and the $p$-value of a test, including randomised tests for discrete data;
  • prove the Neyman–Pearson lemma and use it to find most powerful tests, including cases where the critical region is not a tail;
  • recognise families with a monotone likelihood ratio, write down UMP one-sided tests by the Karlin–Rubin theorem, and explain why a two-sided UMP test fails to exist for a normal mean but exists for $U(0,\theta)$;
  • derive the $t$-, $\chi^2$- and $F$-tests as likelihood ratio tests and apply Wilks' theorem, knowing when it fails;
  • carry out large-sample tests for means, proportions, Poisson rates and correlations, and relate the Wald, score and likelihood ratio versions;
  • apply Pearson's chi-square test of goodness of fit with the correct degrees of freedom after pooling and estimation;
  • test independence and homogeneity in contingency tables, use the $2\times2$ identities, Yates' correction, Fisher's exact test, odds ratios and McNemar's test;
  • use sign, rank and Kolmogorov–Smirnov tests and rank correlations when no parametric model is assumed.
Teaser: What a $p$-Value Looks Like When $H_0$ Is True and When It Is False

Each tick simulates many studies (a seeded generator). In every study $n$ observations are drawn from $N(\delta,1)$ and $H_0:\mu=0$ is tested against $\mu\gt0$ with the $z$-test, $p=1-\Phi(\sqrt n\,\bar x)$. The histogram shows the $p$-values of all studies so far. With $\delta=0$, so that $H_0$ is true, it is flat: under $H_0$ a $p$-value is uniformly distributed, and $5\%$ of studies fall below $0.05$ by chance alone. With $\delta\gt0$ the $p$-values pile up near $0$, and the red share below $0.05$ is the power of the test. Section 5.1 proves both facts and finds the test with the largest such share.

0.0016
studies simulated—
share with p < 0.05—
theory: 1 − Φ(1.645 − √n δ)—

How this is examined

Testing is one of the most heavily examined topics of Unit 4. Part B questions usually ask for a single number: the size or the power of a given critical region, a randomisation probability, a $p$-value, the critical value of a most powerful test, the value of Pearson's $X^2$ for a small table, or the degrees of freedom of a chi-square test after parameters have been estimated. Part C questions, where several options may be correct, test the structure of the theory. Does a UMP test exist for this pair of hypotheses? Does the family have a monotone likelihood ratio in the given statistic? Is the test unbiased? Is the likelihood ratio statistic asymptotically $\chi^2$, and with how many degrees of freedom, or does the support depend on the parameter? Which statements about the $2\times2$ table are identities? Is a given nonparametric statistic distribution-free under $H_0$? The wrong options come from the ⚠️ boxes: tail regions that are not most powerful, the uniform family where a two-sided UMP test does exist, Wilks applied where it fails, degrees of freedom counted before pooling, and paired data treated as independent samples. GATE ST asks the same material, often in numerical-answer form.

Sections in this chapter

✅ Before you start

This chapter uses Chapter 4 throughout: sampling distributions, the $\chi^2$, $t$ and $F$ laws and order statistics (§4.1); likelihood, maximum likelihood, sufficiency and the factorisation theorem, exponential families, Fisher information and the asymptotic normality of the MLE (§4.2); and confidence intervals, which are dual to tests (§4.3). From the earlier chapters you need the probability integral transform (§1.3); the binomial, Poisson and hypergeometric laws (§2.1); and the CLT, Slutsky's theorem, the continuous mapping theorem and the delta method (§2.3). From linear algebra, orthogonal projections and their ranks are used in the proof of Pearson's theorem; the sister course CSIR NET Unit 1: Analysis and Linear Algebra covers them.