Chapter 4 — Sampling Distributions and Estimation
From probability to inference: the laws of statistics computed from random samples, the methods that turn data into estimates, the criteria that rank estimators, and the intervals and posterior laws that report how uncertain an estimate is.
Chapters 1–3 worked forwards. A probability model was given, with all its parameters, and we computed what the data would look like: the chance of $k$ defectives, the distribution of a waiting time, the long-run share of rainy days. Statistics works backwards. A pharmaceutical company weighs $25$ tablets from a production run of a million and must say something about the mean weight of all of them. An epidemiologist counts infections in $40$ villages and must estimate the infection rate. A reliability engineer stops a life test after the third of five components fails and must estimate the mean lifetime. In each case the model is known only up to an unknown parameter $\theta$, and the data are one random draw from it. Every number we compute from the data, whether a mean, a variance, a maximum or a median, is itself a random variable, and it would come out differently on another sample.
This chapter builds the theory that makes such reasoning precise. Section 4.1 studies sampling distributions: the exact laws of $\bar X$ and $S^2$ for normal samples, which lead to the $\chi^2$, $t$ and $F$ laws, the approximate laws of other statistics given by the central limit theorem and the delta method, and the laws of order statistics, the sample minimum, maximum, median and range. Section 4.2 turns to estimation. It covers two general methods (moments and maximum likelihood) and the criteria for judging an estimator (bias, mean squared error, consistency, efficiency). Then come the ideas that lead to optimal estimators: sufficiency, completeness, the Rao–Blackwell and Lehmann–Scheffé theorems, and the Cramér–Rao lower bound. Section 4.3 reports uncertainty, through confidence intervals built from pivotal quantities and through the posterior distribution of a Bayesian analysis. The tools of Chapter 2, namely gamma and beta laws, characteristic functions, Slutsky's theorem and the CLT, are used throughout.
- compute the mean, variance and standard error of $\bar X$, $S^2$ and other statistics, and use the CLT and the delta method to find their approximate laws;
- prove that $\bar X$ and $S^2$ are independent for normal samples, with $(n-1)S^2/\sigma^2\sim\chi^2_{n-1}$, and use the $t$ and $F$ laws and the relations between them;
- write down the density of any order statistic, the joint density of two of them, and the law of the sample range, and use the beta law of uniform order statistics and the spacings of exponential samples;
- find method-of-moments and maximum likelihood estimators, including the boundary cases such as $U(0,\theta)$, and use the invariance of the MLE;
- compare estimators by bias, MSE, consistency and efficiency, and recognise when an unbiased estimator does not exist or is absurd;
- find sufficient, minimal sufficient and complete statistics with the factorisation theorem, the ratio criterion and exponential families;
- construct UMVUEs with the Rao–Blackwell and Lehmann–Scheffé theorems, compute Fisher information and the Cramér–Rao bound, and state the asymptotic normality of the MLE;
- build confidence intervals from pivots for normal, exponential, uniform and binomial models, and carry out conjugate Bayesian updating with Bayes estimators and credible intervals.
How this is examined
Every CSIR NET Unit 4 paper has several questions from this chapter. Part B questions usually ask for one quantity: the distribution of a combination of normal variables (is it $\chi^2$, $t$ or $F$, and with how many degrees of freedom?), the mean of an order statistic, the probability that a sample range exceeds a given value, an MLE or method-of-moments estimate for a stated density, the UMVUE of a function such as $e^{-\lambda}$ or $p^2$, a Fisher information or a Cramér–Rao bound, or the length or confidence level of an interval. Part C questions, where several options may be correct, test the boundaries of the theory. Is the MLE unbiased? Is it unique? Is this statistic sufficient, minimal sufficient, complete? Does the estimator attain the Cramér–Rao bound, and does the bound even apply when the support depends on $\theta$? Are two given statistics independent? Is the posterior mean a weighted average of the prior mean and the MLE? The wrong options are taken from the ⚠️ boxes: $U(0,\theta)$ and its non-regular MLE, the curved family $N(\theta,\theta^2)$, unbiased estimators that do not exist, and statements that are true for normal samples only. GATE ST asks the same material in numerical-answer form.
Sections in this chapter
Sampling Distributions and Order Statistics
Statistics and standard errors, $\bar X$ and $S^2$ for normal samples, the $\chi^2$, $t$ and $F$ laws and their relations, asymptotic laws by the CLT and the delta method, order statistics, the range and the median, and uniform and exponential order statistics.
Methods of Estimation and Properties of Estimators
Bias, MSE and consistency, the method of moments, maximum likelihood and its invariance, sufficiency and the factorisation theorem, minimal sufficiency, completeness, exponential families, Rao–Blackwell, Lehmann–Scheffé, Fisher information, the Cramér–Rao bound and the asymptotics of the MLE.
Confidence Intervals and Elementary Bayesian Inference
Pivotal quantities, intervals for normal means and variances, for differences of means and for proportions, intervals in exponential and uniform models, and the Bayesian alternative: priors, posteriors, conjugate families, Bayes estimators and credible intervals.
This chapter relies on Chapters 1 and 2 throughout: expectation, variance and moment generating functions (§1.3); the quantile transform and the change-of-variables formula (§1.3); joint densities, conditional expectation, the law of total variance, the Jacobian formula and the laws of maxima and minima (§1.4); the gamma, beta, exponential, normal and Poisson laws, the chi-square law as a gamma law, and the beta–binomial tail identity (§2.1); Chebyshev's and Jensen's inequalities and the Cauchy–Schwarz inequality (§2.2); and convergence in probability and in distribution, the laws of large numbers, the CLT, Slutsky's theorem and the delta method (§2.3). From analysis you need differentiation under the integral sign, Taylor's formula and the uniqueness of power series and Laplace transforms; from linear algebra, orthogonal matrices. The sister course CSIR NET Unit 1: Analysis and Linear Algebra covers these.