Chapter 2 — Distributions and Limit Theorems
The standard distributions every statistician uses, the characteristic functions and inequalities that control them, and the limit theorems — laws of large numbers and the central limit theorem — that explain why averages settle down and why the bell curve is everywhere.
Count the defective chips in a batch, the claims reaching an insurer in a month, the minutes until the next call at a helpline, or the share of voters who favour a candidate, and you meet the same few distributions again and again: binomial, Poisson, exponential, gamma, beta, normal. They are not an arbitrary collection. Each comes from a simple mechanism (independent trials, rare events, waiting, mixing, adding many small effects), and the mechanisms tie the families together. Sums of exponential waiting times are gamma, Poisson counts and gamma waiting times describe the same process, and the binomial turns into the Poisson when successes are rare and into the normal when the number of trials is large. Knowing these laws and their relationships turns most probability calculations into a matter of recognition.
The second half of the chapter asks what happens to averages. Toss a fair coin a thousand times and the proportion of heads is very likely to be close to $\tfrac12$. Add many independent errors and their sum is close to normal, whatever the individual errors look like. Making these statements precise needs new tools. Characteristic functions $E[e^{itX}]$ exist for every random variable and turn sums of independent variables into products. Inequalities such as Markov's, Chebyshev's and Jensen's bound probabilities using moments alone. Four different modes of convergence are needed to say what "a sequence of random variables converges" means. With them we prove the weak law of large numbers and the central limit theorem, the two results on which almost all of statistical inference rests.
- state and derive the pmf or density, mean, variance and generating function of every standard discrete and continuous law, and recognise a law from its mechanism, its kernel or its generating function;
- prove the memoryless characterisations of the geometric and exponential laws, the Poisson limit of the binomial, and the relations between Poisson counts and gamma waiting times, gamma ratios and beta laws, beta integrals and binomial tails;
- use reproductive properties correctly, and give counterexamples when their hypotheses fail;
- compute and use characteristic functions, including the inversion and uniqueness theorems and Lévy's continuity theorem;
- apply the Markov, Chebyshev, one-sided Chebyshev, Jensen, Cauchy–Schwarz and Hölder inequalities;
- distinguish almost-sure convergence, convergence in probability, in $r$-th mean and in distribution, prove the implications between them and give the standard counterexamples;
- state and apply the weak and strong laws of large numbers, the Lindeberg–Lévy central limit theorem, Slutsky's theorem and the delta method, including normal approximations with continuity correction.
How this is examined
This chapter supplies a large share of the Unit 4 questions. Part B questions usually ask for one number. Typical requests are a Poisson or binomial probability, the parameter of a law from a condition such as $P(X=1)=P(X=2)$, a moment read off an mgf or a characteristic function, a gamma or beta integral, a probability for a sum of independent exponentials, a Chebyshev bound, or a normal approximation with continuity correction. Part C questions, where more than one option may be correct, prefer statements to be judged. Is the uniform law memoryless? Is a sum of exponentials with different rates gamma? Is a given function a characteristic function? Does convergence in probability imply almost-sure convergence, or convergence of means? Does the sample mean of Cauchy observations converge? Which sequences satisfy the weak law of large numbers? The wrong options are built from the counterexamples collected in the ⚠️ boxes of the three sections, so learn those boxes well. GATE ST asks similar questions in numerical-answer and multiple-select formats.
Sections in this chapter
Standard Discrete and Continuous Distributions
Bernoulli, binomial, Poisson, geometric, negative binomial, hypergeometric; uniform, exponential, gamma, beta, normal, lognormal, Cauchy, Weibull, Laplace; moments, generating functions, memorylessness, relationships and reproductive properties.
Characteristic Functions and Probability Inequalities
Characteristic functions and their properties, inversion and uniqueness, cfs of the standard laws, sums of independent variables, Lévy's continuity theorem; Markov, Chebyshev, Cantelli, Jensen, Cauchy–Schwarz and Hölder inequalities.
Modes of Convergence, Laws of Large Numbers and the CLT
Almost-sure convergence, convergence in probability, in $r$-th mean and in distribution; implications and counterexamples; Slutsky; weak and strong laws; Lindeberg–Lévy CLT, normal approximations and the delta method.
This chapter uses all of Chapter 1. You need distribution functions, densities, the change-of-variables formula, expectations and moment and probability generating functions from §1.3, and joint densities, independence, the Jacobian formula, convolutions and the law of total variance from §1.4. From calculus you need improper integrals, integration by parts, double integrals in polar and other coordinates, and power series such as $\sum x^k/k!$ and $(1-x)^{-r}=\sum_k\binom{k+r-1}{k}x^k$. Sections 2.2 and 2.3 also use complex exponentials $e^{it}=\cos t+i\sin t$, the dominated convergence theorem and the elementary limit $(1+x_n/n)^n\to e^x$ when $x_n\to x$. If these are rusty, revise them from the sister course CSIR NET Unit 1: Analysis and Linear Algebra.