2.1 Standard Discrete and Continuous Distributions
The catalogue every statistician carries: Bernoulli trials and the binomial, geometric, negative binomial and hypergeometric laws; Poisson as the law of rare events; exponential, gamma and beta waiting times; the normal family and its heavy-tailed relatives — with their moments, generating functions and the relationships that tie them together.
Chapter 1 built the general machinery: distribution functions, densities, expectations, generating functions, joint laws and transformations. In practice, though, a small number of distributions do almost all the work. The number of defective chips in a sample is binomial or hypergeometric. The number of insurance claims in a month, of radioactive decays in a second, or of typing errors on a page is close to Poisson. Waiting times between such events are exponential, and the waiting time for the $r$-th event is gamma. Proportions are often given a beta law, measurement errors a normal law, incomes and particle sizes a lognormal law, and lifetimes of components a Weibull law. These families are not an arbitrary zoo. Each arises from a simple mechanism, such as independent trials, rare events, waiting or adding many small effects, and the mechanisms connect the families to one another.
This section develops each standard law from its mechanism and proves its basic properties: the pmf or density, mean, variance and generating function, and the characterising properties (memorylessness of the geometric and exponential laws, the Poisson limit of the binomial). Above all, it proves the relationships: sums of exponentials are gamma, Poisson counts and gamma waiting times describe the same process, gamma ratios are beta, beta integrals are binomial tails, and many families are closed under independent sums (the reproductive property). CSIR Part B questions regularly ask for one number from these facts, such as a Poisson probability, a gamma moment or a beta integral. Part C questions test the boundaries: is a sum of exponentials with different rates still gamma? Is the uniform law memoryless? Which laws have a mean at all? A summary table of all the laws is in the last teaching subsection.
Bernoulli Trials: Binomial, Geometric, Negative Binomial and Hypergeometric Laws
The simplest random experiment has two outcomes, conventionally "success" and "failure". Repeating it independently under identical conditions gives a sequence of Bernoulli trials. Four questions about such a sequence lead to four distributions: how many successes in $n$ trials (binomial), how many trials until the first success (geometric), how many failures before the $r$-th success (negative binomial), and, when the trials are draws without replacement from a finite population, how many successes in the sample (hypergeometric).
Let $0\lt p\lt1$ and $q=1-p$. A sequence $I_1,I_2,\dots$ of independent random variables with $P(I_j=1)=p$, $P(I_j=0)=q$ is a sequence of Bernoulli trials with success probability $p$. A single $I_j$ has the Bernoulli law $\operatorname{Ber}(p)$.
- The number of successes $X=I_1+\dots+I_n$ in $n$ trials has the binomial law $\operatorname{Bin}(n,p)$: $$P(X=k)=\binom nkp^kq^{n-k},\qquad k=0,1,\dots,n.$$
- The number $T$ of trials up to and including the first success has the geometric law $\operatorname{Geo}(p)$ on $\mathbb{N}$: $P(T=k)=q^{k-1}p$, $k=1,2,\dots$ The number of failures before the first success, $T-1$, is geometric on $\{0,1,2,\dots\}$ with $P(T-1=k)=q^kp$.
- For $r\in\mathbb{N}$, the number $Y$ of failures before the $r$-th success has the negative binomial law $\operatorname{NB}(r,p)$: $$P(Y=k)=\binom{k+r-1}{k}p^rq^k,\qquad k=0,1,2,\dots$$ The number of trials needed for $r$ successes is $Y+r$. More generally, for real $r\gt0$, $\operatorname{NB}(r,p)$ is the law with $P(Y=k)=\dfrac{\Gamma(k+r)}{k!\,\Gamma(r)}p^rq^k$.
The discrete uniform law on $\{1,\dots,n\}$ gives each value probability $\tfrac1n$. It has mean $\tfrac{n+1}{2}$ and variance $\tfrac{n^2-1}{12}$ (Exercise IV.1 of §1.3 finds the mean of the uniform law on $\{0,\dots,n\}$ with a pgf). Watch the conventions: some books count trials and some count failures for the geometric and negative binomial laws. Always check the support before using a formula.
Let $X\sim\operatorname{Bin}(n,p)$. Then:
- $E[X]=np$, $\operatorname{Var}(X)=npq$, $G_X(s)=(q+ps)^n$ and $M_X(t)=(q+pe^t)^n$;
- (Reproductive property) if $X\sim\operatorname{Bin}(n,p)$ and $Y\sim\operatorname{Bin}(m,p)$ are independent, then $X+Y\sim\operatorname{Bin}(n+m,p)$;
- (Mode) with $m=(n+1)p$, the pmf increases strictly for $k\lt m$ and decreases strictly for $k\gt m$. So the mode is $\lfloor m\rfloor$ when $m$ is not an integer, and both $m-1$ and $m$ are modes when $m$ is an integer.
Proof. (1) By the binomial theorem the probabilities add up to $(q+p)^n=1$. Write $X=\sum_{j=1}^nI_j$ with independent Bernoulli trials. Each $I_j$ has mean $p$, variance $E[I_j^2]-p^2=p-p^2=pq$ and pgf $q+ps$. Expectations add, variances of independent variables add, and pgfs of independent variables multiply (Theorem 1.40(3) of §1.4), which gives $np$, $npq$ and $(q+ps)^n$. Put $s=e^t$ for the mgf.
(2) $G_{X+Y}(s)=(q+ps)^n(q+ps)^m=(q+ps)^{n+m}$, the pgf of $\operatorname{Bin}(n+m,p)$, and a pgf determines the law (Theorem 1.31 of §1.3). More directly: $X+Y$ counts the successes in $n+m$ Bernoulli trials.
(3) Write $b(k)=P(X=k)$. For $1\le k\le n$, $$\frac{b(k)}{b(k-1)}=\frac{n-k+1}{k}\cdot\frac pq.$$ This is greater than $1$ iff $(n-k+1)p\gt kq$, i.e. iff $(n+1)p\gt k(p+q)=k$. Likewise it equals $1$ iff $k=m$ and is less than $1$ iff $k\gt m$. So $b$ increases up to the largest integer below $m$ and decreases after it, with a tie $b(m-1)=b(m)$ when $m$ is an integer. ∎
(a) If $T\sim\operatorname{Geo}(p)$ on $\mathbb{N}$, then $P(T\gt n)=q^n$ for $n\ge0$, $E[T]=\tfrac1p$, $\operatorname{Var}(T)=\tfrac{q}{p^2}$, $G_T(s)=\dfrac{ps}{1-qs}$ for $|s|\lt\tfrac1q$, and $T$ is memoryless: $$P(T\gt m+n\mid T\gt m)=P(T\gt n)\qquad\text{for all integers }m,n\ge0.$$
(b) Conversely, let $T$ take values in $\mathbb{N}$ with $P(T\gt m+n)=P(T\gt m)\,P(T\gt n)$ for all integers $m,n\ge0$. Then $T$ is geometric with $p=P(T=1)$ (where $p=1$ means $T\equiv1$).
Proof. (a) $T\gt n$ iff the first $n$ trials are failures, so $P(T\gt n)=q^n$ by independence. Then $P(T\gt m+n\mid T\gt m)=q^{m+n}/q^m=q^n$. The pgf is $\sum_{k\ge1}q^{k-1}ps^k=ps\sum_{j\ge0}(qs)^j=\frac{ps}{1-qs}$. Differentiating, $G'(s)=\frac{p}{(1-qs)^2}$ and $G''(s)=\frac{2pq}{(1-qs)^3}$, so $E[T]=G'(1)=\tfrac1p$ and $$\operatorname{Var}(T)=G''(1)+G'(1)-G'(1)^2=\frac{2q}{p^2}+\frac1p-\frac1{p^2}=\frac{q}{p^2}.$$
(b) Let $g(n)=P(T\gt n)$, so $g(0)=1$ because $T\ge1$. The hypothesis with $m=1$ gives $g(n+1)=g(1)g(n)$, hence $g(n)=g(1)^n$ by induction. Put $q=g(1)=P(T\gt1)=1-P(T=1)$. Then $q\lt1$, because otherwise $g(n)=1$ for all $n$, which contradicts $g(n)\to0$. So $P(T=k)=g(k-1)-g(k)=q^{k-1}(1-q)$: $T$ is geometric with $p=1-q$. ∎
Memorylessness says that past failures carry no information about the future. After $m$ failed trials, the number of further trials needed is again $\operatorname{Geo}(p)$. This is the discrete twin of the exponential law (Theorem 2.9), and Exercise III.2 shows how the two are linked: the integer part of an exponential variable is geometric.
Let $Y\sim\operatorname{NB}(r,p)$ with $r\gt0$.
- The probabilities add up to $1$ and $G_Y(s)=\left(\dfrac{p}{1-qs}\right)^r$ for $|s|\lt\tfrac1q$. Hence $E[Y]=\dfrac{rq}{p}$ and $\operatorname{Var}(Y)=\dfrac{rq}{p^2}$.
- For $r\in\mathbb{N}$, $Y$ has the law of $G_1+\dots+G_r$, where the $G_i$ are independent geometric variables on $\{0,1,\dots\}$ (failures before the first success). In particular, the pmf in Definition 2.1(3) is the law of the number of failures before the $r$-th success.
- (Reproductive property) If $Y_1\sim\operatorname{NB}(r_1,p)$ and $Y_2\sim\operatorname{NB}(r_2,p)$ are independent, then $Y_1+Y_2\sim\operatorname{NB}(r_1+r_2,p)$.
Proof. (1) The negative binomial series is $(1-x)^{-r}=\sum_{k\ge0}\frac{\Gamma(k+r)}{k!\,\Gamma(r)}x^k$ for $|x|\lt1$ and real $r\gt0$. It is the Taylor series of $(1-x)^{-r}$, whose $k$-th derivative at $0$ is $r(r+1)\cdots(r+k-1)=\frac{\Gamma(k+r)}{\Gamma(r)}$, and it converges for $|x|\lt1$ by the ratio test. Hence $$G_Y(s)=p^r\sum_k\frac{\Gamma(k+r)}{k!\,\Gamma(r)}(qs)^k=p^r(1-qs)^{-r},$$ and $s=1$ gives total mass $p^rp^{-r}=1$. With $K(s)=\log G_Y(s)=r\log p-r\log(1-qs)$, we get $G'=G\,K'$ and $G''=G(K''+K'^2)$, where $K'(s)=\frac{rq}{1-qs}$ and $K''(s)=\frac{rq^2}{(1-qs)^2}$. At $s=1$: $E[Y]=\frac{rq}{p}$, and $$\operatorname{Var}(Y)=G''(1)+G'(1)-G'(1)^2=\frac{rq^2}{p^2}+\frac{rq}{p}=\frac{rq(q+p)}{p^2}=\frac{rq}{p^2}.$$
(2) A geometric variable on $\{0,1,\dots\}$ has pgf $\sum_kq^kps^k=\frac{p}{1-qs}$. The sum of $r$ independent copies has pgf $\left(\frac{p}{1-qs}\right)^r=G_Y(s)$, so it has the law of $Y$. Directly: the $r$-th success happens on trial $k+r$ iff trial $k+r$ is a success and the first $k+r-1$ trials contain exactly $r-1$ successes and $k$ failures. That has probability $\binom{k+r-1}{k}p^{r-1}q^k\cdot p$. Also $\binom{k+r-1}{k}=\frac{\Gamma(k+r)}{k!\,\Gamma(r)}$ for integer $r$, so the two formulas in Definition 2.1(3) agree.
(3) The pgfs multiply to $\left(\frac{p}{1-qs}\right)^{r_1+r_2}$. ∎
Note that $\operatorname{Var}(Y)=\frac{E[Y]}{p}\gt E[Y]$: the negative binomial is overdispersed compared with a Poisson law of the same mean. This is why it is the standard model for counts that vary more than a Poisson model allows (the Research–Teaching Nexus below).
Let $X\sim\operatorname{Hyp}(N,K,n)$ and $p=K/N$. Then $$E[X]=np,\qquad \operatorname{Var}(X)=np(1-p)\,\frac{N-n}{N-1}.$$ The factor $\frac{N-n}{N-1}$ is the finite population correction. If $N\to\infty$ and $K=K_N$ with $K_N/N\to p\in(0,1)$, while $n$ and $k$ stay fixed, then $P(X=k)\to\binom nkp^k(1-p)^{n-k}$.
Proof. Imagine the sample drawn one item at a time and let $I_j=1$ if the $j$-th draw is a success. Every ordering of the population is equally likely, so the $j$-th item drawn is equally likely to be any of the $N$ items. Hence $P(I_j=1)=\frac KN=p$ for each $j$. Likewise, the ordered pair (draw $i$, draw $j$) for $i\ne j$ is equally likely to be any ordered pair of distinct items, so $$P(I_i=I_j=1)=\frac{K(K-1)}{N(N-1)},\qquad \operatorname{Cov}(I_i,I_j)=\frac{K(K-1)}{N(N-1)}-\frac{K^2}{N^2}=-\frac{p(1-p)}{N-1}.$$ Now $X=\sum_{j=1}^nI_j$, so $E[X]=np$ and $$\operatorname{Var}(X)=\sum_j\operatorname{Var}(I_j)+\sum_{i\ne j}\operatorname{Cov}(I_i,I_j)=np(1-p)-n(n-1)\frac{p(1-p)}{N-1}=np(1-p)\frac{N-n}{N-1}.$$ For the limit, write the binomial coefficients with falling factorials $[a]_j=a(a-1)\cdots(a-j+1)$: $$P(X=k)=\binom nk\frac{[K]_k\,[N-K]_{n-k}}{[N]_n}=\binom nk\cdot\frac{[K]_k}{N^k}\cdot\frac{[N-K]_{n-k}}{N^{n-k}}\cdot\frac{N^n}{[N]_n}.$$ Since $[K]_k/N^k=\prod_{i\lt k}\left(\frac KN-\frac iN\right)\to p^k$, and similarly the third factor tends to $(1-p)^{n-k}$ and the fourth to $1$, the limit is the binomial probability. ∎
The indicators $I_j$ are exchangeable but not independent: drawing a success makes later successes less likely, which is why the covariance is negative and the variance is smaller than the binomial $np(1-p)$. When the sampling fraction $n/N$ is small, the correction is close to $1$, and sampling without replacement behaves like sampling with replacement. This is the basis of the "finite population correction" in survey sampling (§8.1).
The Poisson Distribution and the Law of Rare Events
When the number of trials is large and the success probability small, the binomial law depends, to a good approximation, only on the mean number of successes $\lambda=np$. The limiting law is the Poisson distribution, the universal model for counts of rare events: accidents at a junction, calls to a helpline, mutations in a DNA strand, claims on an insurance portfolio.
Let $X\sim\operatorname{Poi}(\lambda)$.
- $G_X(s)=e^{\lambda(s-1)}$ and $M_X(t)=\exp\left(\lambda(e^t-1)\right)$ for all $s,t$. The $r$-th factorial moment is $\lambda^r$, so $E[X]=\operatorname{Var}(X)=\lambda$.
- $P(X=k)=\dfrac{\lambda}{k}P(X=k-1)$ for $k\ge1$. Hence the mode is $\lfloor\lambda\rfloor$ if $\lambda\notin\mathbb{N}$, and both $\lambda-1$ and $\lambda$ are modes if $\lambda\in\mathbb{N}$.
- (Reproductive property) If $X_i\sim\operatorname{Poi}(\lambda_i)$, $i=1,\dots,m$, are independent, then $\sum_iX_i\sim\operatorname{Poi}\left(\sum_i\lambda_i\right)$.
Proof. (1) $G_X(s)=e^{-\lambda}\sum_k\frac{(\lambda s)^k}{k!}=e^{\lambda(s-1)}$, finite for every $s$; putting $s=e^t$ gives $M_X$. The $r$-th derivative of $G_X$ at $1$ is $\lambda^r$, which is the factorial moment by Theorem 1.31 of §1.3. So $E[X]=\lambda$ and $\operatorname{Var}(X)=\lambda^2+\lambda-\lambda^2=\lambda$. (2) Divide consecutive probabilities: the ratio $\lambda/k$ exceeds $1$ iff $k\lt\lambda$ and equals $1$ iff $k=\lambda$. (3) The pgfs multiply to $e^{(\sum\lambda_i)(s-1)}$. ∎
(a) If $p_n\in(0,1)$ and $np_n\to\lambda\gt0$ as $n\to\infty$, then for every fixed $k\ge0$, $$\binom nkp_n^k(1-p_n)^{n-k}\to e^{-\lambda}\frac{\lambda^k}{k!}.$$
(b) (Le Cam) Let $S=I_1+\dots+I_n$ with independent $I_i\sim\operatorname{Ber}(p_i)$, and let $\lambda=\sum_ip_i$. Then for every set $A\subseteq\{0,1,2,\dots\}$, $$\left|P(S\in A)-\operatorname{Poi}(\lambda)(A)\right|\le\sum_{i=1}^np_i^2.$$ In particular, the total variation distance between $\operatorname{Bin}(n,\lambda/n)$ and $\operatorname{Poi}(\lambda)$ is at most $\lambda^2/n$.
Proof. (a) Write $\lambda_n=np_n$ and split the binomial probability into four factors: $$\binom nkp_n^k(1-p_n)^{n-k}=\frac{[n]_k}{n^k}\cdot\frac{\lambda_n^k}{k!}\cdot\left(1-\frac{\lambda_n}{n}\right)^n\cdot(1-p_n)^{-k}.$$ The first factor is $\prod_{i\lt k}(1-\tfrac in)\to1$; the second tends to $\frac{\lambda^k}{k!}$; the fourth tends to $1$ because $p_n=\lambda_n/n\to0$. For the third, $\log(1-x)=-x+O(x^2)$ as $x\to0$, so $n\log(1-\lambda_n/n)=-\lambda_n+O(\lambda_n^2/n)\to-\lambda$, and the third factor tends to $e^{-\lambda}$.
(b) We couple each Bernoulli variable with a Poisson variable on the same probability space. Let $U_1,\dots,U_n$ be independent uniform $(0,1)$ variables. Put $I_i=\mathbf{1}\{U_i\gt1-p_i\}$, so $I_i\sim\operatorname{Ber}(p_i)$. Put $Y_i=0$ if $U_i\le e^{-p_i}$, and otherwise $Y_i=k$ when $U_i$ lies in the $k$-th of the consecutive intervals of lengths $e^{-p_i}p_i^k/k!$, $k=1,2,\dots$, that fill $(e^{-p_i},1)$. Then $Y_i\sim\operatorname{Poi}(p_i)$, and the $Y_i$ are independent. Because $e^{-p}\ge1-p$, the event $\{I_i=0\}=\{U_i\le1-p_i\}$ lies inside $\{Y_i=0\}$, and the event $\{Y_i=1\}$ lies inside $\{U_i\gt e^{-p_i}\}\subseteq\{I_i=1\}$. So $I_i$ and $Y_i$ differ only on $\{I_i=1,\ Y_i\ne1\}$, and $$P(I_i\ne Y_i)=P(I_i=1)-P(Y_i=1)=p_i-p_ie^{-p_i}=p_i(1-e^{-p_i})\le p_i^2.$$ Let $T=\sum Y_i$, which is $\operatorname{Poi}(\lambda)$ by Theorem 2.5(3). For any $A$, the events $\{S\in A\}$ and $\{T\in A\}$ can differ only where $S\ne T$, so $$|P(S\in A)-P(T\in A)|\le P(S\ne T)\le\sum_iP(I_i\ne Y_i)\le\sum_ip_i^2.$$ For $\operatorname{Bin}(n,\lambda/n)$, all $p_i=\lambda/n$ and the bound is $n\cdot\lambda^2/n^2=\lambda^2/n$. ∎
Part (b) is stronger than (a) in two ways: it gives an explicit error, and it does not need equal success probabilities. The number of rare events among many different independent sources (households filing claims, each with its own small probability) is approximately Poisson with mean $\sum p_i$. The bound also shows what can go wrong: with $p=\tfrac12$ the error bound $n/4$ is useless, and indeed $\operatorname{Bin}(n,\tfrac12)$ (variance $\tfrac n4$) is not close to $\operatorname{Poi}(\tfrac n2)$ (variance $\tfrac n2$). Interactive 1 shows the total variation distance shrinking like $1/n$.
(a) (Thinning) Let $N\sim\operatorname{Poi}(\lambda)$, and suppose that, given $N=n$, each of the $n$ events is independently of type 1 with probability $p$ and of type 2 with probability $q=1-p$. Let $N_1$, $N_2$ be the numbers of events of each type. Then $N_1\sim\operatorname{Poi}(\lambda p)$ and $N_2\sim\operatorname{Poi}(\lambda q)$, and $N_1$, $N_2$ are independent.
(b) (Conditioning) If $X\sim\operatorname{Poi}(\lambda)$ and $Y\sim\operatorname{Poi}(\mu)$ are independent, then given $X+Y=n$, $X\sim\operatorname{Bin}\left(n,\tfrac{\lambda}{\lambda+\mu}\right)$.
Proof. (a) For $i,j\ge0$, the event $\{N_1=i,N_2=j\}$ requires $N=i+j$ and then $i$ type-1 events among the $i+j$: $$P(N_1=i,N_2=j)=e^{-\lambda}\frac{\lambda^{i+j}}{(i+j)!}\binom{i+j}{i}p^iq^j=e^{-\lambda p}\frac{(\lambda p)^i}{i!}\cdot e^{-\lambda q}\frac{(\lambda q)^j}{j!},$$ using $e^{-\lambda}=e^{-\lambda p}e^{-\lambda q}$. The joint pmf factorises into the two Poisson pmfs, which proves both claims. (b) $X+Y\sim\operatorname{Poi}(\lambda+\mu)$ by Theorem 2.5(3), so for $0\le k\le n$, $$P(X=k\mid X+Y=n)=\frac{e^{-\lambda}\frac{\lambda^k}{k!}\,e^{-\mu}\frac{\mu^{n-k}}{(n-k)!}}{e^{-(\lambda+\mu)}\frac{(\lambda+\mu)^n}{n!}}=\binom nk\left(\frac{\lambda}{\lambda+\mu}\right)^k\left(\frac{\mu}{\lambda+\mu}\right)^{n-k}.\ ∎$$
Part (a) is surprising at first: for a fixed number $n$ of events, the counts $N_1$ and $n-N_1$ are perfectly negatively dependent. It is the Poisson randomness of the total that makes them independent. Independence of the two parts holds for no other law of $N$: it forces the pgf identity $G(ps+qt)=G(ps+q)\,G(p+qt)$, whose only solutions are $G(s)=e^{\lambda(s-1)}$ (statement only). The result is the key to thinning and superposing Poisson processes (§3.3).
Waiting Times: Exponential, Gamma and Beta Laws
Continuous waiting times are modelled by the exponential law and its sums, the gamma laws. Proportions and probabilities themselves are modelled by the beta laws on $(0,1)$. Both families are built on two classical integrals.
- $\Gamma(\alpha+1)=\alpha\,\Gamma(\alpha)$ and $\Gamma(1)=1$; hence $\Gamma(n)=(n-1)!$ for $n\in\mathbb{N}$.
- $B(a,b)=\dfrac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}$, and $B(a,b)=B(b,a)$.
- $\Gamma(\tfrac12)=\sqrt\pi$, and consequently $\displaystyle\int_{-\infty}^\infty e^{-z^2/2}\,dz=\sqrt{2\pi}$.
Proof. (1) Integrate by parts: $\int_0^Rx^\alpha e^{-x}dx=\left[-x^\alpha e^{-x}\right]_0^R+\alpha\int_0^Rx^{\alpha-1}e^{-x}dx$. The boundary term is $-R^\alpha e^{-R}\to0$ (and it vanishes at $0$ because $\alpha\gt0$). Also $\Gamma(1)=\int_0^\infty e^{-x}dx=1$, and induction gives $\Gamma(n)=(n-1)!$.
(2) By Tonelli's theorem, $\Gamma(a)\Gamma(b)=\iint_{(0,\infty)^2}x^{a-1}y^{b-1}e^{-(x+y)}\,dx\,dy$. Substitute $x=uv$, $y=u(1-v)$ with $u\gt0$, $0\lt v\lt1$. This is the bijection of Example 5 of §1.4, with Jacobian of absolute value $u$. So $$\Gamma(a)\Gamma(b)=\int_0^\infty\!\!\int_0^1(uv)^{a-1}(u(1-v))^{b-1}e^{-u}\,u\,dv\,du=\int_0^\infty u^{a+b-1}e^{-u}du\int_0^1v^{a-1}(1-v)^{b-1}dv,$$ which is $\Gamma(a+b)B(a,b)$. Symmetry follows from the substitution $x\mapsto1-x$.
(3) By (1) and (2), $\Gamma(\tfrac12)^2=\Gamma(1)B(\tfrac12,\tfrac12)=\int_0^1\frac{dv}{\sqrt{v(1-v)}}$. With $v=\sin^2\theta$, $dv=2\sin\theta\cos\theta\,d\theta$, the integral becomes $\int_0^{\pi/2}2\,d\theta=\pi$. Since $\Gamma(\tfrac12)\gt0$, $\Gamma(\tfrac12)=\sqrt\pi$. Finally, with $z=\sqrt{2u}$ on $(0,\infty)$, $dz=\frac{du}{\sqrt{2u}}$ and $$\int_{-\infty}^\infty e^{-z^2/2}dz=2\int_0^\infty e^{-u}\frac{du}{\sqrt{2u}}=\sqrt2\,\Gamma(\tfrac12)=\sqrt{2\pi}.\ ∎$$
- $U(a,b)$, $a\lt b$: density $\frac{1}{b-a}$ on $(a,b)$. Mean $\frac{a+b}{2}$, variance $\frac{(b-a)^2}{12}$, mgf $\frac{e^{tb}-e^{ta}}{t(b-a)}$ (§1.3, Exercise IV.6).
- $\operatorname{Exp}(\lambda)$, rate $\lambda\gt0$: density $\lambda e^{-\lambda x}$, $x\gt0$, and $P(X\gt x)=e^{-\lambda x}$. Its mean is $\theta=1/\lambda$ (the scale); many texts parametrise by $\theta$ instead.
- $\operatorname{Gamma}(\alpha,\lambda)$, shape $\alpha\gt0$ and rate $\lambda\gt0$: density $$f(x)=\frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x},\qquad x\gt0.$$ $\operatorname{Gamma}(1,\lambda)=\operatorname{Exp}(\lambda)$; for integer $\alpha$ it is also called the Erlang law. The chi-square law with $n$ degrees of freedom is $\chi^2_n=\operatorname{Gamma}(\tfrac n2,\tfrac12)$.
- $\operatorname{Beta}(a,b)$, $a,b\gt0$: density $\dfrac{x^{a-1}(1-x)^{b-1}}{B(a,b)}$, $0\lt x\lt1$. $\operatorname{Beta}(1,1)=U(0,1)$.
(a) If $X\sim\operatorname{Exp}(\lambda)$, then $P(X\gt s+t\mid X\gt s)=P(X\gt t)$ for all $s,t\ge0$. The conditional density of $X$ at $s$ given survival to $s$, $f(s)/P(X\gt s)$, is the constant $\lambda$.
(b) Conversely, let $X$ be a random variable with $P(X\gt0)=1$ and $P(X\gt s+t)=P(X\gt s)P(X\gt t)$ for all $s,t\ge0$. Then $X\sim\operatorname{Exp}(\lambda)$ for some $\lambda\gt0$.
Proof. (a) $e^{-\lambda(s+t)}/e^{-\lambda s}=e^{-\lambda t}$, and $\lambda e^{-\lambda s}/e^{-\lambda s}=\lambda$.
(b) Let $G(t)=P(X\gt t)$. It is nonincreasing and right-continuous, with $G(0)=1$ and $G(t)\to0$ as $t\to\infty$, and it satisfies $G(s+t)=G(s)G(t)$. Put $a=G(1)$. Applying the equation $n$ times gives $G(\tfrac1n)^n=G(1)=a$ and $G(\tfrac mn)=G(\tfrac1n)^m$. If $a=0$, then $G(\tfrac1n)=0$ for all $n$, contradicting $G(\tfrac1n)\to G(0)=1$ by right-continuity. If $a=1$, then $G(m)=1$ for all $m$, contradicting $G(t)\to0$. So $0\lt a\lt1$; put $\lambda=-\log a\gt0$. Then $G(\tfrac1n)=a^{1/n}$ (the unique nonnegative $n$-th root) and $G(r)=a^r=e^{-\lambda r}$ for every positive rational $r$. For real $t\ge0$, choose rationals $r_k\downarrow t$; right-continuity gives $G(t)=\lim e^{-\lambda r_k}=e^{-\lambda t}$. ∎
In reliability language (§9.1), the ratio $h(s)=f(s)/P(X\gt s)$ is the hazard rate: the rate of failure of a unit that has survived to age $s$. The exponential law is the law of a unit that does not age. Interactive 2 shows how the gamma and Weibull hazards rise or fall as the shape parameter crosses $1$.
Let $X\sim\operatorname{Gamma}(\alpha,\lambda)$.
- For real $r\gt-\alpha$, $E[X^r]=\dfrac{\Gamma(\alpha+r)}{\Gamma(\alpha)\lambda^r}$, and $E[X^r]=\infty$ for $r\le-\alpha$. In particular $E[X]=\frac\alpha\lambda$, $\operatorname{Var}(X)=\frac{\alpha}{\lambda^2}$, and $E[1/X]=\frac{\lambda}{\alpha-1}$ for $\alpha\gt1$.
- $M_X(t)=\left(\dfrac{\lambda}{\lambda-t}\right)^\alpha$ for $t\lt\lambda$.
- For $c\gt0$, $cX\sim\operatorname{Gamma}(\alpha,\lambda/c)$. In particular $2\lambda X\sim\chi^2_{2\alpha}$.
- (Reproductive property) If $X_i\sim\operatorname{Gamma}(\alpha_i,\lambda)$ are independent with a common rate, then $\sum_iX_i\sim\operatorname{Gamma}(\sum_i\alpha_i,\lambda)$.
- Consequently, a sum of $n$ iid $\operatorname{Exp}(\lambda)$ variables is $\operatorname{Gamma}(n,\lambda)$, and if $Z_1,\dots,Z_n$ are iid $N(0,1)$, then $Z_1^2+\dots+Z_n^2\sim\chi^2_n$.
- For $\alpha\ge1$ the mode is $\frac{\alpha-1}{\lambda}$. For $\alpha\lt1$ the density is unbounded at $0$ and decreasing.
Proof. (1) $E[X^r]=\frac{\lambda^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha+r-1}e^{-\lambda x}dx$. Near $0$ this converges iff $\alpha+r\gt0$. Then the substitution $y=\lambda x$ gives $\frac{\lambda^\alpha}{\Gamma(\alpha)}\cdot\frac{\Gamma(\alpha+r)}{\lambda^{\alpha+r}}$. With $r=1,2$ and Theorem 2.8(1), $E[X]=\frac\alpha\lambda$ and $E[X^2]=\frac{\alpha(\alpha+1)}{\lambda^2}$, so $\operatorname{Var}(X)=\frac{\alpha}{\lambda^2}$. With $r=-1$, $\frac{\Gamma(\alpha-1)}{\Gamma(\alpha)}\lambda=\frac{\lambda}{\alpha-1}$.
(2) For $t\lt\lambda$, $M_X(t)=\frac{\lambda^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha-1}e^{-(\lambda-t)x}dx=\frac{\lambda^\alpha}{(\lambda-t)^\alpha}$, by the same substitution with rate $\lambda-t$. For $t\ge\lambda$ the integral diverges.
(3) By Theorem 1.26 of §1.3, $cX$ has density $\frac1cf(y/c)=\frac{(\lambda/c)^\alpha}{\Gamma(\alpha)}y^{\alpha-1}e^{-(\lambda/c)y}$. With $c=2\lambda$ the rate becomes $\tfrac12$ and the shape is $\alpha=\tfrac{2\alpha}{2}$.
(4) The mgfs multiply to $\left(\frac{\lambda}{\lambda-t}\right)^{\sum\alpha_i}$ on $t\lt\lambda$, and an mgf determines the law (Theorem 1.32(5) of §1.3).
(5) $\operatorname{Exp}(\lambda)=\operatorname{Gamma}(1,\lambda)$. By Example 3(b) of §1.3, $Z^2$ has density $\frac{1}{\sqrt{2\pi y}}e^{-y/2}=\frac{(1/2)^{1/2}}{\Gamma(1/2)}y^{-1/2}e^{-y/2}$ (using $\Gamma(\frac12)=\sqrt\pi$), which is $\operatorname{Gamma}(\tfrac12,\tfrac12)=\chi^2_1$. Now apply (4).
(6) $\frac{d}{dx}\log f(x)=\frac{\alpha-1}{x}-\lambda$, which is positive for $x\lt\frac{\alpha-1}{\lambda}$ and negative after when $\alpha\gt1$, and negative everywhere when $\alpha\le1$. ∎
Let $E_1,E_2,\dots$ be iid $\operatorname{Exp}(\lambda)$, let $S_n=E_1+\dots+E_n$ (so $S_n\sim\operatorname{Gamma}(n,\lambda)$), and for $t\gt0$ let $N(t)=\#\{n\ge1:S_n\le t\}$ be the number of "arrivals" by time $t$. Then for every $n\in\mathbb{N}$, $$P(S_n\le t)=P(N(t)\ge n)=1-\sum_{k=0}^{n-1}e^{-\lambda t}\frac{(\lambda t)^k}{k!},$$ and $N(t)\sim\operatorname{Poi}(\lambda t)$.
Proof. The $E_i$ are positive, so $S_1\lt S_2\lt\cdots$, and at least $n$ arrivals have occurred by time $t$ iff the $n$-th has: $\{N(t)\ge n\}=\{S_n\le t\}$. Let $H(t)$ be the right-hand side. Then $H(0)=0$ and, differentiating term by term, $$H'(t)=\sum_{k=0}^{n-1}\left[\lambda e^{-\lambda t}\frac{(\lambda t)^k}{k!}-\lambda e^{-\lambda t}\frac{(\lambda t)^{k-1}}{(k-1)!}\right]=\lambda e^{-\lambda t}\frac{(\lambda t)^{n-1}}{(n-1)!},$$ where the $k=0$ term has no second part and the sum telescopes. This is the $\operatorname{Gamma}(n,\lambda)$ density, so $H(t)=\int_0^tf_{S_n}=P(S_n\le t)$. Finally, for $n\ge1$, $P(N(t)=n)=P(N(t)\ge n)-P(N(t)\ge n+1)$ $=e^{-\lambda t}\frac{(\lambda t)^n}{n!}$, and $P(N(t)=0)=P(E_1\gt t)=e^{-\lambda t}$. ∎
This is the Poisson process of rate $\lambda$ (§3.3): exponential gaps between events, gamma arrival times and Poisson counts are three descriptions of one object. The identity is also a computational tool. A gamma probability with integer shape is a finite Poisson sum, and a Poisson tail is a gamma integral.
- If $X\sim\operatorname{Beta}(a,b)$, then $E[X^r]=\dfrac{B(a+r,b)}{B(a,b)}$ for $r\gt-a$; in particular $$E[X]=\frac{a}{a+b},\qquad\operatorname{Var}(X)=\frac{ab}{(a+b)^2(a+b+1)}.$$ For $a,b\gt1$ the mode is $\frac{a-1}{a+b-2}$. Also $1-X\sim\operatorname{Beta}(b,a)$.
- If $X\sim\operatorname{Gamma}(a,\lambda)$ and $Y\sim\operatorname{Gamma}(b,\lambda)$ are independent, then $U=X+Y$ and $V=\dfrac{X}{X+Y}$ are independent, with $U\sim\operatorname{Gamma}(a+b,\lambda)$ and $V\sim\operatorname{Beta}(a,b)$.
- If $U\sim U(0,1)$ and $a\gt0$, then $U^{1/a}\sim\operatorname{Beta}(a,1)$.
Proof. (1) $E[X^r]=\frac{1}{B(a,b)}\int_0^1x^{a+r-1}(1-x)^{b-1}dx=\frac{B(a+r,b)}{B(a,b)}$. By Theorem 2.8, $\frac{B(a+1,b)}{B(a,b)}=\frac{\Gamma(a+1)\Gamma(a+b)}{\Gamma(a)\Gamma(a+b+1)}=\frac{a}{a+b}$ and similarly $E[X^2]=\frac{a(a+1)}{(a+b)(a+b+1)}$. Then $$\operatorname{Var}(X)=\frac{a(a+1)}{(a+b)(a+b+1)}-\frac{a^2}{(a+b)^2}=\frac{a\left[(a+1)(a+b)-a(a+b+1)\right]}{(a+b)^2(a+b+1)}=\frac{ab}{(a+b)^2(a+b+1)}.$$ The mode solves $\frac{a-1}{x}=\frac{b-1}{1-x}$. The law of $1-X$ follows from the substitution $x\mapsto1-x$.
(2) The joint density of $(X,Y)$ is $\frac{\lambda^{a+b}}{\Gamma(a)\Gamma(b)}x^{a-1}y^{b-1}e^{-\lambda(x+y)}$ on $(0,\infty)^2$. The map $(x,y)\mapsto(u,v)=(x+y,\frac{x}{x+y})$ is a bijection onto $(0,\infty)\times(0,1)$ with inverse $x=uv$, $y=u(1-v)$ and $|J|=u$. By the Jacobian formula (Theorem 1.39 of §1.4) and Theorem 2.8(2), $$f_{U,V}(u,v)=\frac{\lambda^{a+b}}{\Gamma(a)\Gamma(b)}u^{a+b-1}e^{-\lambda u}v^{a-1}(1-v)^{b-1}=\underbrace{\frac{\lambda^{a+b}u^{a+b-1}e^{-\lambda u}}{\Gamma(a+b)}}_{\operatorname{Gamma}(a+b,\lambda)}\cdot\underbrace{\frac{v^{a-1}(1-v)^{b-1}}{B(a,b)}}_{\operatorname{Beta}(a,b)}.$$ A joint density that factorises on a product set gives independence (Theorem 1.34 of §1.4).
(3) $P(U^{1/a}\le x)=P(U\le x^a)=x^a$ on $(0,1)$, with derivative $ax^{a-1}=\frac{x^{a-1}}{B(a,1)}$, because $B(a,1)=\frac1a$. ∎
(a) (Binomial tails are beta integrals) For $1\le k\le n$ and $0\le p\le1$, $$\sum_{j=k}^n\binom njp^j(1-p)^{n-j}=\frac{n!}{(k-1)!\,(n-k)!}\int_0^pt^{k-1}(1-t)^{n-k}\,dt.$$ That is, $P(\operatorname{Bin}(n,p)\ge k)=P(\operatorname{Beta}(k,n-k+1)\le p)$.
(b) (Beta–binomial law) Let $P\sim\operatorname{Beta}(a,b)$ and, given $P=p$, let $X\sim\operatorname{Bin}(n,p)$. Then $$P(X=k)=\binom nk\frac{B(a+k,\,b+n-k)}{B(a,b)},\qquad k=0,\dots,n,$$ $E[X]=\dfrac{na}{a+b}$, $\operatorname{Var}(X)=\dfrac{nab(a+b+n)}{(a+b)^2(a+b+1)}$, and given $X=k$ the conditional law of $P$ is $\operatorname{Beta}(a+k,\,b+n-k)$.
Proof. (a) Both sides vanish at $p=0$, so it suffices to compare derivatives in $p$. Using $j\binom nj=n\binom{n-1}{j-1}$ and $(n-j)\binom nj=n\binom{n-1}{j}$, $$\frac{d}{dp}\sum_{j=k}^n\binom njp^j(1-p)^{n-j}=n\sum_{j=k}^n\left[\binom{n-1}{j-1}p^{j-1}(1-p)^{n-j}-\binom{n-1}{j}p^j(1-p)^{n-1-j}\right].$$ The sum telescopes, because the second term for $j$ is the first term for $j+1$, and $\binom{n-1}{n}=0$. What remains is $n\binom{n-1}{k-1}p^{k-1}(1-p)^{n-k}$ $=\frac{n!}{(k-1)!(n-k)!}p^{k-1}(1-p)^{n-k}$, the derivative of the right side. Finally, $B(k,n-k+1)=\frac{(k-1)!(n-k)!}{n!}$ by Theorem 2.8, so the right side is the $\operatorname{Beta}(k,n-k+1)$ cdf at $p$.
(b) Integrate the conditional pmf against the beta density: $$P(X=k)=\int_0^1\binom nkp^k(1-p)^{n-k}\frac{p^{a-1}(1-p)^{b-1}}{B(a,b)}dp=\binom nk\frac{B(a+k,b+n-k)}{B(a,b)}.$$ By the tower property, $E[X]=E[nP]=\frac{na}{a+b}$. By the law of total variance (Theorem 1.38 of §1.4), $\operatorname{Var}(X)=E[nP(1-P)]+\operatorname{Var}(nP)$. Theorem 2.12(1) gives $E[P(1-P)]=\frac{a}{a+b}-\frac{a(a+1)}{(a+b)(a+b+1)}=\frac{ab}{(a+b)(a+b+1)}$, so $$\operatorname{Var}(X)=\frac{nab}{(a+b)(a+b+1)}+\frac{n^2ab}{(a+b)^2(a+b+1)}=\frac{nab(a+b+n)}{(a+b)^2(a+b+1)}.$$ Bayes' theorem for densities gives a posterior proportional to $p^k(1-p)^{n-k}\cdot p^{a-1}(1-p)^{b-1}$, the $\operatorname{Beta}(a+k,b+n-k)$ kernel. ∎
Part (a) has a probabilistic reading. If $U_1,\dots,U_n$ are iid uniform, then the $k$-th smallest of them is at most $p$ iff at least $k$ of the $U_i$ are at most $p$, a binomial event. So the $k$-th order statistic of a uniform sample is $\operatorname{Beta}(k,n-k+1)$ (§4.1). Part (b) with $a=b=1$ makes $X$ uniform on $\{0,\dots,n\}$, the Bayes–Laplace urn of Example 4 in §1.4. The posterior statement is the first example of a conjugate prior (§4.3).
The Normal Family and Its Heavy-Tailed Relatives
The normal law is the law of sums of many small independent effects; the central limit theorem of §2.3 makes this precise. Its relatives arise by simple transformations. Exponentiating a normal variable gives the lognormal law, dividing two independent normals gives the Cauchy law, the difference of two exponentials gives the Laplace law, and a power of an exponential gives the Weibull law. They differ most in their tails, and the tails decide which moments exist.
- $f$ is a density, and $X\sim N(\mu,\sigma^2)$ iff $Z=\frac{X-\mu}{\sigma}\sim N(0,1)$. Hence $P(X\le x)=\Phi\left(\frac{x-\mu}{\sigma}\right)$, and $\Phi(-z)=1-\Phi(z)$.
- $M_X(t)=\exp\left(\mu t+\tfrac12\sigma^2t^2\right)$ for all $t$.
- $E[Z^{2k+1}]=0$ and $E[Z^{2k}]=\dfrac{(2k)!}{2^kk!}=1\cdot3\cdot5\cdots(2k-1)$. So $E[X]=\mu$, $\operatorname{Var}(X)=\sigma^2$, the skewness is $0$ and the kurtosis is $\beta_2=3$.
- (Closure under linear combinations) If $X_i\sim N(\mu_i,\sigma_i^2)$, $i=1,\dots,n$, are independent and $a_i,b$ are constants, then $$b+\sum_ia_iX_i\sim N\left(b+\sum_ia_i\mu_i,\ \sum_ia_i^2\sigma_i^2\right).$$
- (Mills' upper bound) For $z\gt0$, $1-\Phi(z)\le\dfrac{\varphi(z)}{z}$.
Proof. (1) By Theorem 2.8(3) and the substitution $z=(x-\mu)/\sigma$, $\int f=\frac{1}{\sqrt{2\pi}}\int e^{-z^2/2}dz=1$. The equivalence is Theorem 1.26 of §1.3 for the affine map $x\mapsto(x-\mu)/\sigma$. Symmetry of $\varphi$ gives $\Phi(-z)=1-\Phi(z)$.
(2) Complete the square: $tz-\frac{z^2}{2}=\frac{t^2}{2}-\frac{(z-t)^2}{2}$, so $M_Z(t)=e^{t^2/2}\int\varphi(z-t)dz=e^{t^2/2}$. Then $M_X(t)=e^{\mu t}M_Z(\sigma t)$ by Theorem 1.32(3) of §1.3.
(3) Expand $e^{t^2/2}=\sum_k\frac{t^{2k}}{2^kk!}$ and compare with $M_Z(t)=\sum_r\frac{t^r}{r!}E[Z^r]$ (Theorem 1.32(2)). Odd coefficients vanish, and $\frac{E[Z^{2k}]}{(2k)!}=\frac{1}{2^kk!}$. Since $(2k)!=\left[2\cdot4\cdots(2k)\right]\left[1\cdot3\cdots(2k-1)\right]$ and $2\cdot4\cdots(2k)=2^kk!$, the second form follows. With $k=2$, $E[Z^4]=3$.
(4) The mgf of the combination is $$e^{bt}\prod_iM_{X_i}(a_it)=\exp\left(\left(b+\sum_ia_i\mu_i\right)t+\tfrac12\left(\sum_ia_i^2\sigma_i^2\right)t^2\right),$$ the mgf of the stated normal law, and mgfs determine laws.
(5) For $t\ge z\gt0$, $1\le t/z$, so $1-\Phi(z)=\int_z^\infty\varphi(t)dt\le\frac1z\int_z^\infty t\varphi(t)dt=\frac{\varphi(z)}{z}$, because $\varphi'(t)=-t\varphi(t)$. ∎
Values worth remembering (all checked numerically): $\Phi(1)=0.8413$, $\Phi(1.645)=0.9500$, $\Phi(1.96)=0.9750$, $\Phi(2)=0.9772$, $\Phi(2.576)=0.9950$; $P(|Z|\le1)=0.6827$, $P(|Z|\le2)=0.9545$, $P(|Z|\le3)=0.9973$. Exercise IV.4 gives the matching lower bound $\left(\frac1z-\frac1{z^3}\right)\varphi(z)\le1-\Phi(z)$, so $1-\Phi(z)\sim\varphi(z)/z$ as $z\to\infty$.
- Lognormal $\operatorname{LN}(\mu,\sigma^2)$: the law of $X=e^Y$ with $Y\sim N(\mu,\sigma^2)$. Its density is $\dfrac{1}{x\sigma\sqrt{2\pi}}\exp\left(-\dfrac{(\log x-\mu)^2}{2\sigma^2}\right)$, $x\gt0$.
- Cauchy $C(\theta,\gamma)$, location $\theta\in\mathbb{R}$ and scale $\gamma\gt0$: density $\dfrac{\gamma}{\pi\left(\gamma^2+(x-\theta)^2\right)}$, cdf $\tfrac12+\tfrac1\pi\arctan\frac{x-\theta}{\gamma}$.
- Laplace (double exponential) $\operatorname{Laplace}(\theta,b)$, $b\gt0$: density $\dfrac{1}{2b}e^{-|x-\theta|/b}$, $x\in\mathbb{R}$.
- Weibull $\operatorname{Weib}(k,\lambda)$, shape $k\gt0$ and scale $\lambda\gt0$: $P(X\gt x)=\exp\left(-(x/\lambda)^k\right)$ for $x\ge0$, density $\dfrac k\lambda\left(\dfrac x\lambda\right)^{k-1}e^{-(x/\lambda)^k}$. An equivalent common form is $f(x)=\alpha\beta x^{\beta-1}e^{-\alpha x^\beta}$, with $\beta=k$ and $\alpha=\lambda^{-k}$. $\operatorname{Weib}(1,\lambda)=\operatorname{Exp}(1/\lambda)$, and $k=2$ is the Rayleigh law.
- (Lognormal) If $X\sim\operatorname{LN}(\mu,\sigma^2)$, then $E[X^r]=e^{r\mu+r^2\sigma^2/2}$ for every real $r$. So $E[X]=e^{\mu+\sigma^2/2}$, $\operatorname{Var}(X)=e^{2\mu+\sigma^2}\left(e^{\sigma^2}-1\right)$, the median is $e^\mu$ and the mode is $e^{\mu-\sigma^2}$. Products of independent lognormal variables are lognormal: $\operatorname{LN}(\mu_1,\sigma_1^2)\cdot\operatorname{LN}(\mu_2,\sigma_2^2)=\operatorname{LN}(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$. Yet $E[e^{tX}]=\infty$ for every $t\gt0$.
- (Cauchy) If $X\sim C(0,1)$, then $\theta+\gamma X\sim C(\theta,\gamma)$ and $1/X\sim C(0,1)$; $E[X]$ does not exist; the median is $0$ and the quartiles are $\pm1$. If $Z_1,Z_2$ are iid $N(0,1)$, then $Z_1/Z_2\sim C(0,1)$. If $X_1,\dots,X_n$ are iid $C(\theta,\gamma)$, then $\bar X_n=\frac1n\sum X_i\sim C(\theta,\gamma)$ (statement here; the proof via characteristic functions is in §2.2).
- (Laplace) If $E_1,E_2$ are iid $\operatorname{Exp}(1)$, then $\theta+b(E_1-E_2)\sim\operatorname{Laplace}(\theta,b)$. Its mgf is $\dfrac{e^{\theta t}}{1-b^2t^2}$ for $|t|\lt\tfrac1b$, its mean is $\theta$ and its variance $2b^2$, and $|X-\theta|\sim\operatorname{Exp}(1/b)$.
- (Weibull) If $E\sim\operatorname{Exp}(1)$, then $\lambda E^{1/k}\sim\operatorname{Weib}(k,\lambda)$, and $E[X^r]=\lambda^r\Gamma\left(1+\tfrac rk\right)$ for $r\gt-k$. Its hazard rate $h(x)=\frac k\lambda\left(\frac x\lambda\right)^{k-1}$ increases for $k\gt1$, is constant for $k=1$ and decreases for $k\lt1$. The minimum of $n$ iid $\operatorname{Weib}(k,\lambda)$ variables is $\operatorname{Weib}(k,\lambda n^{-1/k})$.
Proof. (1) $E[X^r]=E[e^{rY}]=M_Y(r)$, which is finite for every real $r$ by Theorem 2.14(2). With $r=1,2$: $E[X]=e^{\mu+\sigma^2/2}$ and $E[X^2]=e^{2\mu+2\sigma^2}$, so $\operatorname{Var}(X)=e^{2\mu+\sigma^2}(e^{\sigma^2}-1)$. Since $x\mapsto e^x$ is increasing, $P(X\le e^\mu)=P(Y\le\mu)=\tfrac12$. For the mode, $\frac{d}{dx}\log f(x)=-\frac1x-\frac{\log x-\mu}{\sigma^2x}$ vanishes iff $\log x=\mu-\sigma^2$, where it changes sign from $+$ to $-$. The log of a product of independent lognormals is a sum of independent normals, which is normal by Theorem 2.14(4). The infinite mgf was shown in the ⚠️ box of §1.3: for $t\gt0$, $e^{te^y}$ grows faster than $e^{(y-\mu)^2/(2\sigma^2)}$ as $y\to\infty$.
(2) The affine map is Theorem 1.26 of §1.3. For $1/X$, the map $x\mapsto1/x$ is decreasing on each of $(-\infty,0)$ and $(0,\infty)$, each $y\ne0$ has one preimage $1/y$, and $|h'(y)|=y^{-2}$: $$f_{1/X}(y)=\frac{1}{\pi(1+y^{-2})}\cdot\frac1{y^2}=\frac{1}{\pi(1+y^2)}.$$ Nonexistence of the mean is Example 6(b) of §1.3, and the quartiles come from $\arctan(\pm1)=\pm\frac\pi4$. For the ratio, note that $W=Z_1/Z_2$ equals $(Z_1\operatorname{sgn}Z_2)/|Z_2|$, and $Z_1\operatorname{sgn}Z_2$ is $N(0,1)$ and independent of $|Z_2|$ (given $Z_2$, it is $\pm Z_1$, which is $N(0,1)$ whatever the sign). For independent $Z\sim N(0,1)$ and $V\gt0$ with density $g$, the map $(z,v)\mapsto(z/v,v)$ has inverse $z=wv$ with Jacobian $v$, so $Z/V$ has density $\int_0^\infty v\,\varphi(wv)\,g(v)\,dv$. With $g=2\varphi$ on $(0,\infty)$, $$f_W(w)=\int_0^\infty v\cdot\frac{2}{2\pi}e^{-(1+w^2)v^2/2}dv=\frac1\pi\cdot\frac{1}{1+w^2}.$$
(3) $M_{E_1-E_2}(t)=\frac{1}{1-t}\cdot\frac{1}{1+t}=\frac{1}{1-t^2}$ for $|t|\lt1$, which is the mgf of $\operatorname{Laplace}(0,1)$ computed in Exercise IV.8 of §1.3. By uniqueness and Theorem 1.32(3), $\theta+b(E_1-E_2)$ has mgf $\frac{e^{\theta t}}{1-b^2t^2}$ and is $\operatorname{Laplace}(\theta,b)$. Its variance is $b^2\cdot2\operatorname{Var}(E_1)=2b^2$. Finally, $|X-\theta|$ has density $2\cdot\frac{1}{2b}e^{-y/b}$ on $y\gt0$.
(4) $P(\lambda E^{1/k}\gt x)=P(E\gt(x/\lambda)^k)=e^{-(x/\lambda)^k}$. Then $E[X^r]=\lambda^rE[E^{r/k}]=\lambda^r\Gamma(1+\frac rk)$ by Theorem 2.10(1) with $\alpha=1$. The hazard is the density divided by the survival function, a power $x^{k-1}$ with the stated monotonicity. For the minimum, $P(\min\gt x)=e^{-n(x/\lambda)^k}=e^{-(x/(\lambda n^{-1/k}))^k}$. ∎
Relationships and Reproductive Properties
Most CSIR questions on this section test a relationship rather than a single formula. We collect the closure properties under independent sums, then a map of the transformations proved above, and finally a reference table of moments and generating functions.
Let $X_1,\dots,X_m$ be independent. Then $S=X_1+\dots+X_m$ has the following laws:
- $X_i\sim\operatorname{Bin}(n_i,p)$ ⟹ $S\sim\operatorname{Bin}(\sum n_i,p)$ — the same $p$ is essential;
- $X_i\sim\operatorname{Poi}(\lambda_i)$ ⟹ $S\sim\operatorname{Poi}(\sum\lambda_i)$;
- $X_i\sim\operatorname{NB}(r_i,p)$ ⟹ $S\sim\operatorname{NB}(\sum r_i,p)$; in particular a sum of $r$ iid geometric failure counts is $\operatorname{NB}(r,p)$;
- $X_i\sim\operatorname{Gamma}(\alpha_i,\lambda)$ ⟹ $S\sim\operatorname{Gamma}(\sum\alpha_i,\lambda)$ — the same rate is essential; $\chi^2_{n_1}+\chi^2_{n_2}=\chi^2_{n_1+n_2}$;
- $X_i\sim N(\mu_i,\sigma_i^2)$ ⟹ $S\sim N(\sum\mu_i,\sum\sigma_i^2)$;
- $X_i\sim C(\theta_i,\gamma_i)$ ⟹ $S\sim C(\sum\theta_i,\sum\gamma_i)$ (proof in §2.2).
The uniform, beta, lognormal, Laplace and Weibull families are not closed under independent sums, nor are exponentials with different rates or binomials with different $p$.
Proof. (1)–(5) are Theorems 2.1(2), 2.5(3), 2.3(3), 2.10(4) and 2.14(4); each is a product of pgfs or mgfs followed by the uniqueness theorem. For the negative statements: $U_1+U_2$ is triangular, not uniform (Example 6 of §1.4). A sum of two beta variables can exceed $1$, so it is not beta. Exponentials with different rates and binomials with different $p$ are Example 7(a) and (c) below. Two iid $\operatorname{Laplace}(0,1)$ variables have a sum with mgf $(1-t^2)^{-2}$; a Laplace law symmetric about $0$ has mgf $(1-b^2t^2)^{-1}$, and the two functions differ (one has a double pole at $t=1$, the other a simple pole at $t=\frac1b$). That sums of independent lognormal or Weibull variables leave their families is also true, but we do not prove it here. ∎
| from | operation | result | where |
|---|---|---|---|
| Bernoulli$(p)$ | sum of $n$ iid | $\operatorname{Bin}(n,p)$ | Thm 2.1 |
| geometric failures | sum of $r$ iid | $\operatorname{NB}(r,p)$ | Thm 2.3 |
| $\operatorname{Hyp}(N,K,n)$ | $N\to\infty$, $K/N\to p$ | $\operatorname{Bin}(n,p)$ | Thm 2.4 |
| $\operatorname{Bin}(n,p_n)$ | $np_n\to\lambda$ | $\operatorname{Poi}(\lambda)$ | Thm 2.6 |
| $\operatorname{NB}(r,p_r)$ | $r(1-p_r)\to\lambda$ | $\operatorname{Poi}(\lambda)$ | Ex. II.8 |
| $\operatorname{Poi}(\lambda)$ | thinning with prob. $p$ | $\operatorname{Poi}(\lambda p)$, independent parts | Thm 2.7 |
| $\operatorname{Poi}(\lambda),\operatorname{Poi}(\mu)$ | condition on the sum $n$ | $\operatorname{Bin}(n,\frac{\lambda}{\lambda+\mu})$ | Thm 2.7 |
| $\operatorname{Gamma}(r,\lambda)$ rate | mix a Poisson mean | $\operatorname{NB}(r,\frac{\lambda}{1+\lambda})$ | Ex. II.9 |
| $U(0,1)$ | $-\frac1\lambda\log U$; $U^{1/a}$ | $\operatorname{Exp}(\lambda)$; $\operatorname{Beta}(a,1)$ | §1.3, Thm 2.12 |
| $\operatorname{Exp}(\lambda)$ | sum of $n$ iid; integer part | $\operatorname{Gamma}(n,\lambda)$; geometric | Thm 2.10, Ex. III.2 |
| Exp gaps | count arrivals by $t$ | $\operatorname{Poi}(\lambda t)$ | Thm 2.11 |
| $\operatorname{Gamma}(a,\lambda),\operatorname{Gamma}(b,\lambda)$ | $\frac{X}{X+Y}$ | $\operatorname{Beta}(a,b)$, independent of $X+Y$ | Thm 2.12 |
| $\operatorname{Beta}(k,n-k+1)$ | cdf at $p$ | $P(\operatorname{Bin}(n,p)\ge k)$ | Thm 2.13 |
| $\operatorname{Beta}(a,b)$ prior | binomial given $P$ | beta–binomial | Thm 2.13 |
| $N(0,1)$ | $Z^2$; $\sum_1^nZ_i^2$ | $\chi^2_1=\operatorname{Gamma}(\frac12,\frac12)$; $\chi^2_n$ | Thm 2.10 |
| $N(\mu,\sigma^2)$ | $e^X$ | $\operatorname{LN}(\mu,\sigma^2)$ | Thm 2.15 |
| $N(0,1),N(0,1)$ | ratio $Z_1/Z_2$ | $C(0,1)$ | Thm 2.15 |
| $\operatorname{Exp}(1),\operatorname{Exp}(1)$ | $\theta+b(E_1-E_2)$ | $\operatorname{Laplace}(\theta,b)$ | Thm 2.15 |
| $\operatorname{Exp}(1)$ | $\lambda E^{1/k}$ | $\operatorname{Weib}(k,\lambda)$ | Thm 2.15 |
| law | pmf / pdf | mean | variance | pgf / mgf |
|---|---|---|---|---|
| $\operatorname{Bin}(n,p)$ | $\binom nkp^kq^{n-k}$ | $np$ | $npq$ | $(q+ps)^n$ |
| $\operatorname{Poi}(\lambda)$ | $e^{-\lambda}\lambda^k/k!$ | $\lambda$ | $\lambda$ | $e^{\lambda(s-1)}$ |
| $\operatorname{Geo}(p)$ on $\mathbb{N}$ | $q^{k-1}p$ | $1/p$ | $q/p^2$ | $\frac{ps}{1-qs}$ |
| $\operatorname{NB}(r,p)$ failures | $\binom{k+r-1}{k}p^rq^k$ | $rq/p$ | $rq/p^2$ | $\left(\frac{p}{1-qs}\right)^r$ |
| $\operatorname{Hyp}(N,K,n)$ | $\binom Kk\binom{N-K}{n-k}/\binom Nn$ | $np$, $p=\frac KN$ | $npq\frac{N-n}{N-1}$ | — |
| uniform $\{1..n\}$ | $1/n$ | $\frac{n+1}{2}$ | $\frac{n^2-1}{12}$ | $\frac{s(1-s^n)}{n(1-s)}$ |
| $U(a,b)$ | $\frac1{b-a}$ | $\frac{a+b}2$ | $\frac{(b-a)^2}{12}$ | $\frac{e^{bt}-e^{at}}{t(b-a)}$ |
| $\operatorname{Exp}(\lambda)$ | $\lambda e^{-\lambda x}$ | $\frac1\lambda$ | $\frac1{\lambda^2}$ | $\frac{\lambda}{\lambda-t}$ |
| $\operatorname{Gamma}(\alpha,\lambda)$ | $\frac{\lambda^\alpha x^{\alpha-1}e^{-\lambda x}}{\Gamma(\alpha)}$ | $\frac\alpha\lambda$ | $\frac\alpha{\lambda^2}$ | $\left(\frac{\lambda}{\lambda-t}\right)^\alpha$ |
| $\operatorname{Beta}(a,b)$ | $\frac{x^{a-1}(1-x)^{b-1}}{B(a,b)}$ | $\frac{a}{a+b}$ | $\frac{ab}{(a+b)^2(a+b+1)}$ | no closed form |
| $N(\mu,\sigma^2)$ | $\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}$ | $\mu$ | $\sigma^2$ | $e^{\mu t+\sigma^2t^2/2}$ |
| $\operatorname{LN}(\mu,\sigma^2)$ | see Def. 2.7 | $e^{\mu+\sigma^2/2}$ | $e^{2\mu+\sigma^2}(e^{\sigma^2}-1)$ | $\infty$ for $t\gt0$ |
| $C(\theta,\gamma)$ | $\frac{\gamma}{\pi(\gamma^2+(x-\theta)^2)}$ | none | none | none (cf $e^{i\theta t-\gamma|t|}$) |
| $\operatorname{Laplace}(\theta,b)$ | $\frac1{2b}e^{-|x-\theta|/b}$ | $\theta$ | $2b^2$ | $\frac{e^{\theta t}}{1-b^2t^2}$ |
| $\operatorname{Weib}(k,\lambda)$ | $\frac k\lambda(\frac x\lambda)^{k-1}e^{-(x/\lambda)^k}$ | $\lambda\Gamma(1+\frac1k)$ | $\lambda^2\left[\Gamma(1+\frac2k)-\Gamma(1+\frac1k)^2\right]$ | no closed form |
- Read the mechanism. Fixed number of independent trials → binomial; without replacement → hypergeometric; trials until a success → geometric or negative binomial; rare events in time or space → Poisson; waiting for the $r$-th event → gamma; a random proportion → beta.
- Read the generating function. $(q+ps)^n$, $e^{\lambda(s-1)}$, $\left(\frac{p}{1-qs}\right)^r$, $(1-t/\lambda)^{-\alpha}$, $e^{\mu t+\sigma^2t^2/2}$ identify the law at once (uniqueness). Match the constants carefully: $(1-3t)^{-2}$ is $\operatorname{Gamma}(2,\tfrac13)$, mean $6$.
- Read the kernel. A density proportional to $x^{\alpha-1}e^{-\lambda x}$ on $(0,\infty)$ is gamma, and one proportional to $x^{a-1}(1-x)^{b-1}$ on $(0,1)$ is beta. The normalising constant then follows from $\Gamma$ or $B$, with no integration.
- Convert between Poisson and gamma. $P(\operatorname{Gamma}(n,\lambda)\le t)=P(\operatorname{Poi}(\lambda t)\ge n)$, and binomial tails are beta cdfs.
- Check the convention (rate or mean; trials or failures) and the hypotheses of a closure property (same $p$, same rate, independence).
- Parametrisations differ. "Exponential with parameter $2$" may mean rate $2$ (mean $\tfrac12$) or mean $2$. The gamma law appears with rate $\lambda$ or scale $\theta=1/\lambda$. The geometric law lives on $\{1,2,\dots\}$ (mean $\tfrac1p$) or on $\{0,1,\dots\}$ (mean $\tfrac qp$). Always read off the support and the density first.
- Sums of exponentials with different rates are not gamma (Example 7(a)). The gamma reproductive property needs a common rate.
- Sums of binomials with different $p$ are not binomial (Example 7(c)): the variance is too small.
- Mean equal to variance does not make a law Poisson. $P(X=0)=P(X=2)=\tfrac12$ has mean $1$ and variance $1$.
- Only two laws are memoryless: the geometric among laws on $\mathbb{N}$ and the exponential among positive laws (Theorems 2.2, 2.9). The uniform law on $(0,1)$ has $P(X\gt s+t\mid X\gt s)=\frac{1-s-t}{1-s}\lt1-t$ for $s,t\gt0$, $s+t\lt1$; gamma laws with $\alpha\ne1$ and Weibull laws with $k\ne1$ are not memoryless either.
- The Poisson approximation needs small $p$, not just large $n$. $\operatorname{Bin}(100,\tfrac12)$ has variance $25$, while $\operatorname{Poi}(50)$ has variance $50$. Le Cam's bound $np^2$ quantifies this.
- Without replacement is not with replacement. Ignoring the factor $\frac{N-n}{N-1}$ overstates the hypergeometric variance; it matters when $n/N$ is not small (Example 2).
- The Cauchy law has no mean, and averaging does not help. $\bar X_n\sim C(0,1)$ for every $n$, so the sample mean of Cauchy data never settles. The law of large numbers (§2.3) needs $E|X|\lt\infty$.
- All moments, no mgf. The lognormal law has $E[X^r]$ finite for all $r$ but $E[e^{tX}]=\infty$ for all $t\gt0$.
- Normal marginals are not enough. If $Z\sim N(0,1)$ and $S=\pm1$ with probability $\tfrac12$ each, independent of $Z$, then $SZ\sim N(0,1)$, but $Z+SZ=(1+S)Z$ equals $0$ with probability $\tfrac12$, so it is not normal. Theorem 2.14(4) needs independence (or joint normality, §7.1).
- Gamma laws with shape below $1$ have densities unbounded at $0$, mode $0$ and $E[1/X]=\infty$ for $\alpha\le1$ (Example 7(d)).
- A beta–binomial count is not binomial. Mixing over an uncertain $p$ inflates the variance from $npq$ to $npq\frac{a+b+n}{a+b+1}$ (Example 5).
Exercises
- 1 $X\sim\operatorname{Bin}(8,\tfrac14)$. Find $P(X=2)$, $P(X\ge1)$, the mode and $E[X^2]$.
- 2 For $X\sim\operatorname{Bin}(n,p)$, show that $E\left[\dfrac{1}{X+1}\right]=\dfrac{1-q^{n+1}}{(n+1)p}$.
- 3 $X\sim\operatorname{Bin}(n,p)$ and $Y\sim\operatorname{Bin}(m,p)$ are independent. Show that, given $X+Y=k$, $X$ is hypergeometric $\operatorname{Hyp}(n+m,n,k)$, whatever the value of $p$.
- 4 A fair die is rolled until a six appears, and $N$ is the number of rolls. Find $P(N\gt6)$, $P(N\text{ is even})$ and $E[N\mid N\gt3]$.
- 5 $X,Y$ are iid $\operatorname{Geo}(p)$ on $\mathbb{N}$. Find $P(X=Y)$, and show that given $X+Y=n$ ($n\ge2$), $X$ is uniform on $\{1,\dots,n-1\}$.
- 6 (Sport) A team wins each game independently with probability $0.6$. A series ends when one team has won $4$ games. Find the probability that the team wins the series, and the probability that the series lasts all $7$ games.
- 7 A committee of $5$ is chosen at random from $8$ women and $7$ men. Let $X$ be the number of women. Find $P(X=3)$, $E[X]$ and $\operatorname{Var}(X)$.
- 8 (Ecology: capture–recapture) $100$ fish in a lake are tagged and released. Later $40$ fish are caught and $x$ of them are tagged. Treating the number $N$ of fish as unknown, show that $P_N(X=x)$, as a function of $N$, is maximised at $N=\lfloor4000/x\rfloor$, and evaluate it for $x=12$.
- 1 $X\sim\operatorname{Poi}(2)$. Find $P(X=0)$, $P(X\le2)$, $P(X\ge3)$ and all modes.
- 2 $X$ is Poisson with $P(X=1)=P(X=2)$. Find $E[X]$ and $P(X=4)$.
- 3 For $X\sim\operatorname{Poi}(\lambda)$, find $E[X^3]$ and the third central moment, and deduce the skewness $\lambda^{-1/2}$.
- 4 For $X\sim\operatorname{Poi}(\lambda)$, show that $P(X\text{ is even})=\tfrac12\left(1+e^{-2\lambda}\right)$.
- 5 For $X\sim\operatorname{Poi}(\lambda)$, show that $E\left[\frac{1}{X+1}\right]=\frac{1-e^{-\lambda}}{\lambda}$.
- 6 (Insurance) Claims arrive at an insurer at a Poisson rate of $4$ per month, and each claim is "large" with probability $0.1$, independently. Find the probability of no large claim in a month, of at least two large claims in a quarter, and, given $6$ claims in a month, of exactly one large claim.
- 7 $X\sim\operatorname{Bin}(500,0.004)$. Compare $P(X=2)$ with its Poisson approximation and state Le Cam's bound for the error.
- 8 Prove that if $r\to\infty$ and $p_r\to1$ with $r(1-p_r)\to\lambda\gt0$, then the $\operatorname{NB}(r,p_r)$ probabilities converge to those of $\operatorname{Poi}(\lambda)$.
- 9 (Gamma–Poisson mixture) Given $\Lambda=\theta$, let $X\sim\operatorname{Poi}(\theta)$, where $\Lambda\sim\operatorname{Gamma}(r,\lambda)$. Show that $X\sim\operatorname{NB}\left(r,\frac{\lambda}{1+\lambda}\right)$, and verify $\operatorname{Var}(X)\gt E[X]$.
- 1 $X$ is exponential with median $2$. Find its rate, its mean and $P(X\gt4\mid X\gt1)$.
- 2 $X\sim\operatorname{Exp}(\lambda)$. Show that $\lfloor X\rfloor$ is geometric on $\{0,1,\dots\}$ with $p=1-e^{-\lambda}$, and that $X-\lfloor X\rfloor$ is independent of $\lfloor X\rfloor$ with density $\frac{\lambda e^{-\lambda u}}{1-e^{-\lambda}}$ on $(0,1)$.
- 3 $X\sim\operatorname{Gamma}(3,2)$. Find $E[X]$, $\operatorname{Var}(X)$, $E[1/X]$ and $P(X\le1)$.
- 4 Evaluate $\int_0^\infty x^5e^{-2x}dx$ and $\int_0^1x^3(1-x)^4dx$ without integrating by parts.
- 5 $X\sim\operatorname{Beta}(a,b)$ with $a\gt1$. Show that $E[1/X]=\frac{a+b-1}{a-1}$.
- 6 $X,Y$ are iid $\operatorname{Exp}(\lambda)$. Show that $\frac{X}{X+Y}\sim U(0,1)$, and find the density of $X/Y$. Does $X/Y$ have a mean?
- 7 (Reliability) A machine runs on one component and has two spares; when the working component fails, a spare is switched in instantly. Component lifetimes are iid exponential with mean $200$ hours. Find the mean and standard deviation of the machine's lifetime, and the probability that it runs for more than $800$ hours.
- 8 Write $P(\operatorname{Bin}(5,0.3)\ge3)$ as a beta integral using Theorem 2.13(a), evaluate the integral, and check it against the binomial sum.
Choose $n$, $k$ and $p$. The lab computes $P(\operatorname{Bin}(n,p)\ge k)$ as a finite sum and $\frac{n!}{(k-1)!(n-k)!}\int_0^pt^{k-1}(1-t)^{n-k}dt$ by a 2000-step Simpson rule. They agree to many decimals, as the telescoping proof promises.
- 1 $X\sim N(10,4)$. Find $P(8\lt X\lt13)$, the $90$th percentile and $P(|X-10|\gt3)$.
- 2 Exam marks are $N(\mu,\sigma^2)$. $10\%$ of candidates score at most $60$ and $5\%$ score above $90$. Find $\mu$ and $\sigma$.
- 3 For $X\sim N(\mu,\sigma^2)$, show that $E|X-\mu|=\sigma\sqrt{2/\pi}$, and compute $E[Z^6]$ for $Z\sim N(0,1)$.
- 4 Prove that $\left(\frac1z-\frac1{z^3}\right)\varphi(z)\le1-\Phi(z)$ for $z\gt0$. Give the two Mills bounds at $z=3$ and compare with $1-\Phi(3)$.
- 5 (Income) A lognormal income distribution has median $\text{₹}\,100$ thousand and mean $\text{₹}\,120$ thousand. Find $\sigma^2$, the variance, and the proportion of incomes above $\text{₹}\,200$ thousand.
- 6 $X\sim C(0,1)$. Find $P(|X|\gt1)$ and $P(X\gt\sqrt3)$. By computing $P(|Y|\lt2)$, show that $Y=X+1/X$ does not have a Cauchy law.
- 7 $X\sim\operatorname{Laplace}(0,b)$ has variance $8$. Find $b$, $P(|X|\gt4)$ and the kurtosis $\beta_2$.
- 8 (Reliability) Units have Weibull lifetimes with shape $1.5$ and scale $1000$ h. Find the mean and median life, the hazard rate at $500$ h and at $2000$ h, and the probability that a unit which has survived $1000$ h survives another $500$ h.
E1. CSIR Part B $X$ is a Poisson random variable with $P(X=2)=3P(X=4)$. Then $E[X^2]$ equals (A) $2$; (B) $4$; (C) $6$; (D) $8$.
E2. CSIR Part B GATE ST $X_1,\dots,X_5$ are iid exponential with mean $2$. Then $P(X_1+\dots+X_5\gt10)$ equals (A) $e^{-5}\left(1+5+\tfrac{25}{2}+\tfrac{125}{6}+\tfrac{625}{24}\right)$; (B) $e^{-5}\left(1+5+\tfrac{25}{2}+\tfrac{125}{6}+\tfrac{625}{24}+\tfrac{3125}{120}\right)$; (C) $1-e^{-5}\left(1+5+\tfrac{25}{2}+\tfrac{125}{6}+\tfrac{625}{24}\right)$; (D) $e^{-2}$.
E3. CSIR Part B $U_1,\dots,U_4$ are iid $U(0,1)$ and $0\lt p\lt1$. The probability that at least two of the $U_i$ are at most $p$ equals $\int_0^pc\,t(1-t)^2\,dt$, where $c$ is (A) $4$; (B) $6$; (C) $12$; (D) $24$.
E4. CSIR Part C Which of the following statements are true for all $s,t\gt0$? (One or more options may be correct.) (A) If $X\sim\operatorname{Exp}(\lambda)$, then $P(X\gt2t\mid X\gt t)=P(X\gt t)$. (B) If $X\sim U(0,1)$, then $P(X\gt s+t\mid X\gt s)=P(X\gt t)$ whenever $s+t\lt1$. (C) If $X\sim\operatorname{Gamma}(2,1)$, then $P(X\gt s+t\mid X\gt s)\le P(X\gt t)$. (D) If $P(X\gt x)=e^{-\sqrt x}$ for $x\ge0$, then $P(X\gt s+t\mid X\gt s)\le P(X\gt t)$.
E5. CSIR Part C GATE ST Let $X$ and $Y$ be independent. Which of the following are true? (A) If $X\sim\operatorname{Poi}(1)$ and $Y\sim\operatorname{Poi}(2)$, then $P(X=1\mid X+Y=3)=\tfrac49$. (B) If $X\sim\operatorname{Bin}(3,\tfrac12)$ and $Y\sim\operatorname{Bin}(3,\tfrac13)$, then $X+Y$ is binomial. (C) If $X,Y$ are iid $N(0,1)$, then $\tfrac12(X+Y)^2\sim\chi^2_1$. (D) If $X,Y$ are iid $\operatorname{Exp}(1)$, then $X-Y$ has density $\tfrac12e^{-|x|}$.
E6. CSIR Part C Which of the following are true? (A) If $X_1,\dots,X_n$ are iid standard Cauchy, then $\bar X_n$ is standard Cauchy. (B) If $Z\sim N(0,1)$, then $E\left[e^{Z^2/4}\right]\lt\infty$. (C) If $\log X\sim N(0,1)$, then $E[e^{tX}]\lt\infty$ for all $t$ in some neighbourhood of $0$. (D) If $X\sim\operatorname{Gamma}(\tfrac12,1)$, then $E[1/X]\lt\infty$.
Suggested plan: 3 lectures × 60 min.
- Bernoulli trials and Poisson. Definition 2.1 with the four questions about a sequence of trials. Theorem 2.1 (do the mode proof on the board), Theorem 2.2 with the characterisation, Theorem 2.3 via pgfs, Theorem 2.4 via indicators (stress the negative covariance). Example 1. Then the Poisson law: Theorem 2.6(a), and the coupling proof of Le Cam's inequality if the class is strong. Project Interactive 1 in all three modes. Example 3 and Theorem 2.7.
- Waiting times. Theorem 2.8 (prove $\Gamma(\frac12)=\sqrt\pi$ through the beta integral, which also normalises the normal density). Theorem 2.9 with the Cauchy functional equation. Theorems 2.10–2.11: emphasise the Poisson–gamma identity as a calculation tool. Example 4. Theorems 2.12–2.13 with Example 5 and the check lab.
- The normal family and the map. Theorem 2.14, values of $\Phi$, Mills' bound. Definition 2.7 and Theorem 2.15 (the ratio of normals is Cauchy is a good Jacobian exercise). Interactive 2 for the hazard shapes and for the tails comparison. Theorem 2.16, the two tables and Example 7 as a class poll. Finish with E4–E6.
Discussion questions. (1) The Poisson law has mean equal to variance. Real count data (goals per match, hospital admissions per day) often have variance larger than the mean. What mechanism could cause this, and which law from this section would you fit instead? (2) Why can a sum of independent binomials with different $p$ never be binomial, while a sum of independent Poissons with different means always is Poisson? (3) The normal, Laplace and Cauchy laws in Interactive 2 have the same median and quartiles. Which of them would you choose to model daily stock returns, and why?
Misconceptions to watch for. Mixing up rate and mean of the exponential and gamma laws; using the trials form of the geometric law with the failures formula for the mean; forgetting the finite population correction; applying the gamma reproductive property with different rates; thinking that a symmetric law (Cauchy) must have mean equal to its centre; believing that the Poisson approximation works whenever $n$ is large; adding standard deviations instead of variances for sums of independent normals.
Suggested homework. I(1, 3, 4, 6, 8), II(2, 4, 6, 8, 9), III(2, 3, 6, 7), IV(2, 4, 5, 8); Exam corner E2, E4, E6.
Research question. Are real counts of rare events (goals in football matches, daily emergency admissions, earthquakes above a given magnitude per year) really Poisson, or are they overdispersed, and does it matter?
Why it matters. Hospital managers staff emergency departments, insurers price policies and seismologists issue hazard maps using Poisson models. If the true variance is larger than the mean, a Poisson model underestimates the probability of extreme days: too few beds, too little capital. Exercise II.9 shows one cause: if the Poisson rate itself varies (between days, teams or regions) according to a gamma law, the counts become negative binomial with variance $\mu+\mu^2/r$.
What to do.
- Find a public data set of counts: for example, goals per team per match in a national football league for one season, or daily counts of a notifiable disease. Compute the sample mean $\bar x$, the sample variance $s^2$ and the dispersion index $s^2/\bar x$.
- Under a Poisson model the dispersion index should be close to $1$. Simulate $1000$ data sets of the same size from $\operatorname{Poi}(\bar x)$ with a seeded generator, compute the dispersion index of each, and find where your real value lies in this simulated distribution.
- Fit a negative binomial law by the method of moments: $\hat p=\bar x/s^2$ and $\hat r=\bar x^2/(s^2-\bar x)$ (valid when $s^2\gt\bar x$). Compare the observed frequencies of $0,1,2,\dots$ with the Poisson and negative binomial fitted frequencies in a table and a bar chart.
- Estimate, under each model, the probability of a count at least three standard deviations above the mean, and discuss the practical consequences of the difference.
Tools: Python (numpy, scipy.stats) or R; Interactive 1 in its negative binomial mode as a picture of how $r$ controls the excess variance.
Hand in: A two-page report with the dispersion index and its simulated reference distribution, the fitted-frequency table and chart, and a recommendation on which model to use.
Going further. The chi-square goodness-of-fit test (§5.2) turns the comparison of observed and fitted frequencies into a formal test. Read about zero-inflated Poisson models, used when there are more zeros than either model predicts, and about Bortkiewicz's classical data on deaths by horse kicks in the Prussian army, often quoted as an example of a good Poisson fit.
🧪 Self-check quiz
Q1. A binomial random variable has mean $6$ and variance $4.2$. Its parameters are
Q2. Which of these positive continuous distributions is memoryless?
Q3. $X$ has density proportional to $x^2e^{-2x}$ on $(0,\infty)$. Then $\operatorname{Var}(X)$ is
Q4. $X\sim\operatorname{Poi}(3)$ and $Y\sim\operatorname{Poi}(5)$ are independent. Then $E[X\mid X+Y=8]$ equals
Q5. $Z_1,Z_2$ are independent standard normal variables. The ratio $Z_1/Z_2$ has