Chapter 1 · Probability

1.3 Random Variables, Expectation and Moments

Random variables turn outcomes into numbers. Distribution functions and their jumps, discrete, continuous and mixed laws, quantiles, change of variables, expectation through the tail formula, moments, and the probability and moment generating functions that store them all.

A probability space describes chance, but we rarely care about the outcome $\omega$ itself. A clinic records the waiting time of a patient, an insurer the size of a claim, a quality engineer the number of defective items in a lot. Each is a numerical function of the outcome, a random variable. Its whole probabilistic behaviour is captured by one real function of a real variable, the distribution function $F(x)=P(X\le x)$. Once we have $F$, the sample space can be forgotten. Expectations, variances, quantiles and probabilities of intervals are all read off from $F$, and two random variables with the same $F$ cannot be told apart by any probability calculation.

This section develops that toolkit with full proofs. We characterise distribution functions and classify them into discrete, continuous and mixed types. We introduce the quantile function and use it to simulate any law from a uniform random number, and we derive the change-of-variables formula for the density of $g(X)$. We define expectation carefully, including when it fails to exist, and prove the tail formula $E[X]=\int_0^\infty P(X\gt x)\,dx$, which turns many hard expectations into easy integrals. Finally we study moments and the probability and moment generating functions. CSIR Part B questions often ask for one of these numbers (a mean from a mixed cdf, a variance from an mgf, a density after a transformation), and Part C questions test the boundary cases: functions that are almost but not quite distribution functions, symmetric laws with no mean, and moments that exist without an mgf.

Random Variables and Distribution Functions

📘 Definition 1.16 — Random variable and its distribution

Let $(\Omega,\mathcal{F},P)$ be a probability space. A random variable is a function $X:\Omega\to\mathbb{R}$ such that $\{X\le x\}=\{\omega:X(\omega)\le x\}\in\mathcal{F}$ for every $x\in\mathbb{R}$. Its distribution (or law) is the probability measure $P_X(B)=P(X\in B)$ on the Borel sets $B\in\mathcal{B}(\mathbb{R})$.

The condition guarantees that $\{X\in B\}$ is an event for every Borel set $B$. Indeed, the collection of sets $B\subseteq\mathbb{R}$ with $X^{-1}(B)\in\mathcal{F}$ is a σ-field, since inverse images commute with complements and countable unions. It contains every half-line $(-\infty,x]$, and these generate $\mathcal{B}(\mathbb{R})$. For the same reason, $g(X)$ is a random variable for every Borel function $g$ (for example every continuous or monotone $g$).

📘 Definition 1.17 — Distribution function
The (cumulative) distribution function (cdf) of a random variable $X$ is $$F(x)=F_X(x)=P(X\le x),\qquad x\in\mathbb{R}.$$ We write $F(x-)=\lim_{t\uparrow x}F(t)$ for the left limit.
📐 Theorem 1.23 — Properties of a distribution function

Let $F$ be the cdf of $X$. Then:

  1. $F$ is nondecreasing;
  2. $F$ is right-continuous: $F(x+)=F(x)$ for every $x$;
  3. $\lim_{x\to-\infty}F(x)=0$ and $\lim_{x\to\infty}F(x)=1$;
  4. $P(X\lt x)=F(x-)$ and $P(X=x)=F(x)-F(x-)$, the size of the jump at $x$;
  5. $F$ has at most countably many discontinuities.

Conversely, every function $F:\mathbb{R}\to\mathbb{R}$ with properties 1–3 is the distribution function of some random variable (proved in Theorem 1.25 below).

Proof. (1) If $x\lt y$ then $\{X\le x\}\subseteq\{X\le y\}$, so $F(x)\le F(y)$ by monotonicity of $P$.

(2) Because $F$ is monotone it suffices to take $x_n=x+\tfrac1n$. The events $\{X\le x+\tfrac1n\}$ decrease to $\{X\le x\}$, so continuity from above (Theorem 1.14(5)) gives $F(x+\tfrac1n)\to F(x)$.

(3) The events $\{X\le -n\}$ decrease to $\varnothing$ and $\{X\le n\}$ increase to $\Omega$. Continuity gives the limits $0$ and $1$ along integers, and monotonicity extends them to real $x\to\pm\infty$.

(4) $\{X\le x-\tfrac1n\}$ increases to $\{X\lt x\}$, so $F(x-\tfrac1n)\to P(X\lt x)$ by continuity from below. Then $P(X=x)=P(X\le x)-P(X\lt x)$ $=F(x)-F(x-)$.

(5) Let $D_k$ be the set of points where the jump exceeds $\tfrac1k$. If $D_k$ contained $k$ distinct points $x_1,\dots,x_k$, the disjoint events $\{X=x_i\}$ would have total probability greater than $1$. So $|D_k|\lt k$, and the set of all discontinuities, $\bigcup_kD_k$, is a countable union of finite sets. ∎

Half-open interval$P(a\lt X\le b)=F(b)-F(a)$.
Closed interval$P(a\le X\le b)=F(b)-F(a-)$.
Open interval$P(a\lt X\lt b)=F(b-)-F(a)$.
Right tail$P(X\gt x)=1-F(x)$ and $P(X\ge x)=1-F(x-)$. At a continuity point of $F$ all four interval formulas agree.
📘 Definition 1.18 — Discrete, continuous and mixed distributions

(a) $X$ is discrete if $P(X\in S)=1$ for some countable set $S$. Its probability mass function (pmf) is $p(x)=P(X=x)$, and then $F(x)=\sum_{s\in S,\,s\le x}p(s)$ is a step function with jumps $p(s)$ at the points $s\in S$.

(b) $X$ is (absolutely) continuous with probability density function (pdf) $f$ if $f\ge0$ is integrable and $$F(x)=\int_{-\infty}^xf(t)\,dt\quad\text{for all }x.$$ Then $F$ is continuous, $\int_{-\infty}^\infty f=1$, $P(X\in B)=\int_Bf$ for Borel sets $B$, and $F'(x)=f(x)$ at every point where $f$ is continuous. The density is determined only up to changes on a set of length zero.

(c) $X$ is mixed if $F=\alpha F_d+(1-\alpha)F_c$ with $0\lt\alpha\lt1$, $F_d$ a discrete cdf and $F_c$ a cdf with a density.

📐 Theorem 1.24 — Separating the jumps (Lebesgue decomposition)

Let $F$ be a cdf with jump points $x_1,x_2,\dots$ and jumps $p_i=F(x_i)-F(x_i-)$, and let $\alpha=\sum_ip_i\le1$. Then $F_J(x)=\sum_{i:\,x_i\le x}p_i$ and $F_C=F-F_J$ are both nondecreasing and right-continuous, and $F_C$ is continuous. Hence, when $0\lt\alpha\lt1$, $F=\alpha F_d+(1-\alpha)G$ with $F_d=F_J/\alpha$ a discrete cdf and $G=F_C/(1-\alpha)$ a continuous cdf.

(Statement only.) A continuous cdf $G$ splits further as $G=\beta G_{ac}+(1-\beta)G_s$, where $G_{ac}$ has a density and $G_s$ is singular: continuous, but with $G_s'=0$ except on a set of length zero. The Cantor function is the classic singular cdf.

Proof. $F_J$ is nondecreasing, and it is right-continuous because $F_J(x+h)-F_J(x)=\sum_{x\lt x_i\le x+h}p_i\to0$ as $h\downarrow0$ (the tail of a convergent series of nonnegative terms). For $x\lt y$, the jumps of $F$ in $(x,y]$ add up to at most the total increase: $$F_J(y)-F_J(x)=\sum_{x\lt x_i\le y}p_i\le F(y)-F(x).$$ (Finitely many disjoint events $\{X=x_i\}$ lie inside $\{x\lt X\le y\}$; let their number grow.) So $F_C=F-F_J$ is nondecreasing, and it is right-continuous as a difference of right-continuous functions. At each point, the jump of $F_C$ is the jump of $F$ minus the jump of $F_J$, which is $p_i-p_i=0$ at $x_i$ and $0-0$ elsewhere. So $F_C$ is continuous. The final claim is a rescaling, with $F_J(\infty)=\alpha$ and $F_C(\infty)=1-\alpha$. ∎

✏️ Example 1 — A mixed distribution
A component has lifetime $X$ (in years) with $$F(x)=\begin{cases}0,&x\lt0,\\ \tfrac14+\tfrac x2,&0\le x\lt1,\\ 1,&x\ge1.\end{cases}$$ (a) Verify that $F$ is a cdf and locate its jumps. (b) Find $P(X=0)$, $P(X=1)$, $P(0\lt X\lt1)$, $P(\tfrac12\le X\le1)$. (c) Write $F$ as a mixture of a discrete and a continuous cdf. (d) Find $E[X]$.
(a) $F$ is nondecreasing (constant, then increasing with slope $\tfrac12$, then constant), right-continuous (each piece is closed on the left), and its limits are $0$ and $1$. It jumps at $0$ (from $0$ to $\tfrac14$) and at $1$ (from $F(1-)=\tfrac34$ to $1$).
(b) By Theorem 1.23(4), $P(X=0)=\tfrac14$ (a component that is dead on arrival) and $P(X=1)=1-\tfrac34=\tfrac14$ (for example, replaced at the end of a one-year warranty). $P(0\lt X\lt1)=F(1-)-F(0)=\tfrac34-\tfrac14=\tfrac12$. $P(\tfrac12\le X\le1)=F(1)-F(\tfrac12-)=1-\tfrac12=\tfrac12$.
(c) The jumps total $\alpha=\tfrac12$, so $F=\tfrac12F_d+\tfrac12G$, where $F_d$ puts mass $\tfrac12$ at each of $0$ and $1$, and $G$ is the uniform cdf on $[0,1]$ (the continuous part $F-F_J$ is $\tfrac x2$ on $[0,1]$, which rescales to $x$).
(d) Combine the atoms with the density part $\tfrac12$ on $(0,1)$: $$E[X]=0\cdot\tfrac14+1\cdot\tfrac14+\int_0^1x\cdot\tfrac12\,dx=\tfrac14+\tfrac14=\tfrac12.$$ Check with the tail formula (Theorem 1.27): $\int_0^1(1-F(x))\,dx=\int_0^1\left(\tfrac34-\tfrac x2\right)dx=\tfrac34-\tfrac14=\tfrac12$.
✏️ Example 2 — Which functions are distribution functions?
Decide which of the following are cdfs. For those that are, identify the type and compute the requested quantity. (a) $F(x)=e^{-e^{-x}}$ on $\mathbb{R}$; find its density and $P(X\le0)$. (b) $F(x)=\tfrac12+\tfrac1\pi\arctan x$; find $P(-1\lt X\le1)$. (c) $F(x)=0$ for $x\le0$ and $F(x)=1$ for $x\gt0$. (d) $F(x)=\dfrac{x^2}{1+x^2}$ on $\mathbb{R}$. (e) $F(x)=0$ for $x\lt0$, $F(x)=x$ for $0\le x\lt\tfrac12$, $F(x)=1$ for $x\ge\tfrac12$.
(a) Yes $e^{-x}$ decreases from $\infty$ to $0$, so $F$ increases continuously from $0$ to $1$. This is the Gumbel law, with density $F'(x)=e^{-x}\exp(-e^{-x})$, and $P(X\le0)=e^{-1}\approx0.368$.
(b) Yes $\arctan$ increases continuously from $-\tfrac\pi2$ to $\tfrac\pi2$. This is the standard Cauchy law, with density $\dfrac{1}{\pi(1+x^2)}$, and $P(-1\lt X\le1)=\tfrac1\pi\left(\tfrac\pi4+\tfrac\pi4\right)=\tfrac12$.
(c) No It is nondecreasing with the right limits, but it is not right-continuous at $0$: $F(0)=0\ne1=F(0+)$. (It is $P(X\lt x)$ for $X\equiv0$; the cdf of $X\equiv0$ is $\mathbf{1}\{x\ge0\}$.)
(d) No $F(-1)=\tfrac12\gt0=F(0)$, so $F$ is not monotone, and $F(x)\to1$ as $x\to-\infty$.
(e) Yes, mixed $F$ is nondecreasing and right-continuous, with the right limits. It has density $1$ on $(0,\tfrac12)$, total weight $\tfrac12$, and an atom of mass $1-\tfrac12=\tfrac12$ at $x=\tfrac12$. This is the law of $\min(U,\tfrac12)$ for $U$ uniform on $(0,1)$.

Quantiles and Functions of a Random Variable

A cdf answers "how likely is $X$ to be at most $x$?". The reverse question, "below which value does $X$ fall with probability $u$?", is answered by the quantile function. When $F$ is continuous and strictly increasing, it is simply $F^{-1}$. In general, $F$ can be flat (no value in an interval) or can jump (an atom), and we need a careful definition.

📘 Definition 1.19 — Quantile function, quantiles and medians

The quantile function of a cdf $F$ is $$Q(u)=F^{-1}(u)=\inf\{x\in\mathbb{R}:F(x)\ge u\},\qquad 0\lt u\lt1.$$

A number $x_p$ is a $p$-th quantile of $X$ if $P(X\lt x_p)\le p\le P(X\le x_p)$, i.e. $F(x_p-)\le p\le F(x_p)$. A median is a $\tfrac12$-quantile, and $Q(\tfrac14)$, $Q(\tfrac34)$ are the lower and upper quartiles. The set of $p$-th quantiles is a closed interval whose left endpoint is $Q(p)$. It is a single point unless $F$ is flat at level $p$.

📐 Theorem 1.25 — The quantile transform and the probability integral transform

Let $F$ satisfy properties 1–3 of Theorem 1.23, and let $Q$ be its quantile function.

  1. For $0\lt u\lt1$ and $x\in\mathbb{R}$: $\ Q(u)\le x\iff u\le F(x)$.
  2. (Quantile transform) If $U$ is uniform on $(0,1)$, then $Q(U)$ has cdf $F$. In particular every such $F$ is a distribution function.
  3. (Probability integral transform) If $X$ has a continuous cdf $F$, then $F(X)$ is uniform on $(0,1)$.

Proof. (1) The set $A_u=\{x:F(x)\ge u\}$ is nonempty (since $F(x)\to1\gt u$) and bounded below (since $F(x)\to0\lt u$), so $Q(u)=\inf A_u$ is a real number. Because $F$ is nondecreasing, $A_u$ is an interval unbounded to the right. Because $F$ is right-continuous, $A_u$ contains its infimum: take $x_n\in A_u$ with $x_n\downarrow Q(u)$; then $F(Q(u))=\lim F(x_n)\ge u$. So $A_u=[Q(u),\infty)$, and $x\in A_u\iff x\ge Q(u)$, which is the claim.

(2) By (1), $P(Q(U)\le x)=P(U\le F(x))=F(x)$, because $P(U\le c)=c$ for $c\in[0,1]$. To produce a random variable with cdf $F$, take $\Omega=(0,1)$ with length as the probability (the uniform law) and $X(\omega)=Q(\omega)$. $Q$ is nondecreasing, so $\{Q\le x\}$ is an interval and $X$ is a random variable.

(3) Fix $u\in(0,1)$. By (1), $F(X)\ge u\iff X\ge Q(u)$, so $P(F(X)\ge u)=1-F(Q(u)-)$. Since $F$ is continuous, $F(Q(u)-)=F(Q(u))$. Moreover $F(Q(u))\ge u$ by (1), while $F(x)\lt u$ for all $x\lt Q(u)$, and continuity gives $F(Q(u))\le u$. So $F(Q(u))=u$ and $P(F(X)\lt u)=u$ for all $u\in(0,1)$. Taking right limits in $u$, $P(F(X)\le u)=u$: $F(X)$ is uniform. ∎

Part (2) is how computers sample from any distribution: generate $U$ and return $Q(U)$. For the exponential law, $F(x)=1-e^{-\lambda x}$ gives $Q(u)=-\tfrac1\lambda\log(1-u)$. Part (3) is the basis of $p$-values (§5.1) and of the Kolmogorov–Smirnov test (§5.3). It fails when $F$ has jumps: if $X$ is a fair coin with values $0,1$, then $F(X)$ takes only the values $\tfrac12$ and $1$.

📐 Theorem 1.26 — Change of variables for a density

(a) Let $X$ have density $f_X$ that vanishes outside an open interval $I$. Let $g:I\to J$ be a bijection onto an open interval $J$ that is strictly monotone, with inverse $h=g^{-1}$ continuously differentiable on $J$. Then $Y=g(X)$ has density $$f_Y(y)=f_X(h(y))\,|h'(y)|\quad(y\in J),\qquad f_Y(y)=0\ \ (y\notin J).$$

(b) Suppose instead that $I$ is the union of disjoint open intervals $I_1,\dots,I_k$ (up to finitely many points) and $g$ is strictly monotone on each $I_j$, with continuously differentiable inverse $h_j$ on $g(I_j)$. Then $$f_Y(y)=\sum_{j:\ y\in g(I_j)}f_X(h_j(y))\,|h_j'(y)|.$$

(c) If $X$ is discrete, $P(g(X)=y)=\sum_{x:\,g(x)=y}P(X=x)$.

Proof. (a) Suppose $g$ is increasing, so $h$ is increasing and $h'\ge0$. For $y\in J$, $\{Y\le y\}=\{X\le h(y)\}$ up to an event of probability $0$ (the event $X\notin I$). Substituting $x=h(t)$, $$F_Y(y)=\int_{\inf I}^{h(y)}f_X(x)\,dx=\int_{\inf J}^{y}f_X(h(t))\,h'(t)\,dt.$$ So $F_Y$ is the integral of $f_X(h)h'$, which is the claim. If $g$ is decreasing, $\{Y\le y\}=\{X\ge h(y)\}$ and $$F_Y(y)=\int_{h(y)}^{\sup I}f_X(x)\,dx=\int_{y}^{\sup J}f_X(h(t))\,(-h'(t))\,dt,$$ so $F_Y(y)=1-\int_y^{\sup J}f_X(h)|h'|$, whose derivative is $f_X(h(y))|h'(y)|$; again $F_Y$ is the integral of this function. (b) For a Borel set $B$, $P(Y\in B)=\sum_jP(X\in I_j,\ g(X)\in B)$. Apply (a) to the restriction of $X$ to each $I_j$ and add. (c) Group the atoms of $X$ by their image. ∎

🧭 Problem-solving strategy — the density of $Y=g(X)$
  1. Find the range of $Y$. Outside it, $f_Y=0$.
  2. If $g$ is monotone on the support, use $f_X(h(y))|h'(y)|$. Do not forget the absolute value when $g$ decreases.
  3. If $g$ is not monotone, split the support into monotone pieces and add one term per preimage. Points $y$ with two preimages get two terms, points with one get one term.
  4. When in doubt, work with the cdf: write $P(Y\le y)$ as a probability about $X$ and differentiate. This always works, and it is the only safe method for mixed variables (e.g. $\max(X,0)$).
  5. Check that $\int f_Y=1$.
✏️ Example 3 — Densities of transformed variables
(a) $U$ is uniform on $(0,1)$ and $\lambda\gt0$. Find the law of $Y=-\tfrac1\lambda\log U$. (b) $Z$ has the standard normal density $\varphi(z)=\tfrac{1}{\sqrt{2\pi}}e^{-z^2/2}$. Find the density of $Y=Z^2$. (c) $X$ is uniform on $(-1,2)$. Find the density of $Y=X^2$, and explain why the "one-branch" formula $f_X(\sqrt y)/(2\sqrt y)$ is wrong. (d) $X$ is uniform on $\{-2,-1,0,1,2\}$. Find the pmf of $X^2$.
(a) $g(u)=-\tfrac1\lambda\log u$ maps $(0,1)$ decreasingly onto $(0,\infty)$, with inverse $h(y)=e^{-\lambda y}$ and $|h'(y)|=\lambda e^{-\lambda y}$. So $f_Y(y)=1\cdot\lambda e^{-\lambda y}$ for $y\gt0$: $Y$ is exponential with rate $\lambda$. This is the quantile transform in disguise: $1-U$ is uniform too.
(b) $z\mapsto z^2$ is decreasing on $(-\infty,0)$ and increasing on $(0,\infty)$. Each $y\gt0$ has two preimages $\pm\sqrt y$, each with $|h'(y)|=\frac{1}{2\sqrt y}$. So $$f_Y(y)=\frac{\varphi(\sqrt y)+\varphi(-\sqrt y)}{2\sqrt y}=\frac{1}{\sqrt{2\pi y}}\,e^{-y/2},\quad y\gt0.$$ This is the chi-square law with one degree of freedom (§4.1). The density is unbounded near $0$ but integrable.
(c) $f_X=\tfrac13$ on $(-1,2)$. For $0\lt y\lt1$ both $\sqrt y$ and $-\sqrt y$ lie in $(-1,2)$; for $1\lt y\lt4$ only $\sqrt y$ does. Hence $$f_Y(y)=\frac{1}{3\sqrt y}\ \ (0\lt y\lt1),\qquad f_Y(y)=\frac{1}{6\sqrt y}\ \ (1\lt y\lt4).$$ Check: $\int_0^1\frac{dy}{3\sqrt y}=\tfrac23$ and $\int_1^4\frac{dy}{6\sqrt y}=\tfrac13$, a total of $1$. The one-branch formula gives $\frac{1}{6\sqrt y}$ on all of $(0,4)$, with total mass $\tfrac23$: it forgets the negative values of $X$. Interactive 2 shows the histogram following the correct, two-branch density.
(d) By Theorem 1.26(c), $P(X^2=0)=\tfrac15$, $P(X^2=1)=P(X=\pm1)=\tfrac25$, $P(X^2=4)=\tfrac25$.
✏️ Example 4 — Quantiles of continuous, discrete and mixed laws
(a) Find the quantile function, median and interquartile range of the exponential law with rate $\lambda$. (b) $X$ is the number of heads in three fair coin tosses. Find $Q(\tfrac12)$, $Q(0.9)$ and all medians. (c) For the mixed law of Example 1, find $Q(u)$ for all $u$ and the median, and explain how to simulate $X$.
(a) Solve $1-e^{-\lambda x}=u$: $Q(u)=-\tfrac1\lambda\log(1-u)$. The median is $\tfrac{\log2}{\lambda}\approx\tfrac{0.693}{\lambda}$, below the mean $\tfrac1\lambda$ (the law is right-skewed). The IQR is $Q(\tfrac34)-Q(\tfrac14)=\tfrac1\lambda\left(\log4-\log\tfrac43\right)=\tfrac{\log3}{\lambda}$.
(b) $F(0)=\tfrac18$, $F(1)=\tfrac12$, $F(2)=\tfrac78$, $F(3)=1$. The smallest $x$ with $F(x)\ge\tfrac12$ is $1$, so $Q(\tfrac12)=1$, and $Q(0.9)=3$. A median $m$ needs $F(m-)\le\tfrac12\le F(m)$. This holds for every $m\in[1,2]$: for $1\lt m\lt2$, $F(m-)=F(m)=\tfrac12$, and at $m=2$, $F(2-)=\tfrac12$. So the medians form the interval $[1,2]$, and $Q(\tfrac12)$ picks its left end.
(c) $F$ jumps from $0$ to $\tfrac14$ at $0$, rises linearly to $\tfrac34$, and jumps to $1$ at $1$. Hence $Q(u)=0$ for $0\lt u\le\tfrac14$; $Q(u)=2(u-\tfrac14)$ for $\tfrac14\lt u\le\tfrac34$; $Q(u)=1$ for $\tfrac34\lt u\lt1$. The median is $Q(\tfrac12)=\tfrac12$, and it is unique because $F$ is strictly increasing there. To simulate, draw $U$ and return $Q(U)$. A whole interval of $u$-values is sent to each atom, of length equal to the atom's mass. Interactive 1 shows exactly this.
Interactive 1 — Distribution Function, Quantiles and the Tail-Area Formula

Pick a distribution and move the level $u$ (or press Play to sweep it). The dashed line at height $u$ meets the graph of $F$ at the quantile $Q(u)=\inf\{x:F(x)\ge u\}$. Filled dots mark $F(x)$ at a jump and open dots the left limit $F(x-)$. A whole range of $u$ shares one quantile whenever the line passes through a jump. The green area between $F$ and $1$ to the right of $0$ minus the red area under $F$ to the left of $0$ is $E[X]$ (the tail formula, Theorem 1.27(c) below).

0.50
u—
Q(u)—
P(X = Q(u))—
E[X] = green − red—

Expectation, Variance and Moments

The expectation of $X$ is its probability-weighted average value, the centre of mass of its distribution. The definition must say what happens when the weighted sum or integral does not converge absolutely, because rearranging a conditionally convergent series can change its sum, and an average should not depend on the order in which we list the outcomes.

📘 Definition 1.20 — Expectation

(a) If $X$ is discrete with pmf $p$, then $E[X]=\sum_xx\,p(x)$, provided $\sum_x|x|\,p(x)\lt\infty$.

(b) If $X$ has density $f$, then $E[X]=\int_{-\infty}^\infty x\,f(x)\,dx$, provided $\int|x|f(x)\,dx\lt\infty$.

(c) For a mixed law, $E[X]$ is the sum over the atoms plus the integral against the density part. In general $E[X]=\int_\Omega X\,dP=\int_{\mathbb{R}}x\,dF(x)$ is a Lebesgue (or Lebesgue–Stieltjes) integral.

$X$ is integrable if $E|X|\lt\infty$. For $X\ge0$ we always allow $E[X]=\infty$. In general, with $X^+=\max(X,0)$ and $X^-=\max(-X,0)$, $E[X]=E[X^+]-E[X^-]$ is defined when at least one of the two is finite, and does not exist when both are infinite.

📐 Theorem 1.27 — The tail formula

(a) If $X\ge0$, then $$E[X]=\int_0^\infty P(X\gt x)\,dx=\int_0^\infty\left(1-F(x)\right)dx,$$ where both sides may be $+\infty$. More generally $E[X^r]=\int_0^\infty r\,x^{r-1}P(X\gt x)\,dx$ for $r\gt0$.

(b) If $X$ takes values in $\{0,1,2,\dots\}$, then $E[X]=\sum_{n=1}^\infty P(X\ge n)=\sum_{n=0}^\infty P(X\gt n)$.

(c) For a general integrable $X$, $$E[X]=\int_0^\infty\left(1-F(x)\right)dx-\int_{-\infty}^0F(x)\,dx.$$

(d) If $X\ge0$ and $E[X]\lt\infty$, then $x\,P(X\gt x)\to0$ as $x\to\infty$. The converse is false.

Proof. (a) For every outcome, $X(\omega)=\int_0^\infty\mathbf{1}\{X(\omega)\gt x\}\,dx$, since the integrand is $1$ exactly on $[0,X(\omega))$. Take expectations and interchange the expectation with the $dx$-integral. This is legitimate by Tonelli's theorem, because the integrand is nonnegative: $$E[X]=E\int_0^\infty\mathbf{1}\{X\gt x\}\,dx=\int_0^\infty E\left[\mathbf{1}\{X\gt x\}\right]dx=\int_0^\infty P(X\gt x)\,dx.$$ For a density the interchange is the familiar swap of the order of integration, $\int_0^\infty\int_x^\infty f(t)\,dt\,dx=\int_0^\infty f(t)\int_0^tdx\,dt=\int_0^\infty tf(t)\,dt$. For $X^r$, use $X^r=\int_0^\infty rx^{r-1}\mathbf{1}\{X\gt x\}\,dx$ in the same way.

(b) For integer-valued $X\ge0$, $P(X\gt x)=P(X\ge n)$ for $x\in[n-1,n)$, so the integral in (a) is the sum of the $P(X\ge n)$. Directly: $\sum_{n\ge1}P(X\ge n)=\sum_{n\ge1}\sum_{k\ge n}p(k)$ $=\sum_{k\ge1}k\,p(k)$, swapping the order of a double series of nonnegative terms.

(c) Apply (a) to $X^+$ and $X^-$. $P(X^+\gt x)=1-F(x)$ for $x\ge0$, and $P(X^-\gt x)=P(X\lt-x)=F((-x)-)$. Since $F(t-)=F(t)$ except at countably many $t$, $\int_0^\infty F((-x)-)\,dx=\int_{-\infty}^0F(t)\,dt$.

(d) For $x\gt0$, $x\,\mathbf{1}\{X\gt x\}\le X\,\mathbf{1}\{X\gt x\}$, so $x\,P(X\gt x)\le E\left[X\mathbf{1}\{X\gt x\}\right]$. The right side tends to $0$ by dominated convergence (it is bounded by $X$, which is integrable, and tends to $0$ pointwise). A counterexample to the converse is in Example 6(d). ∎

📐 Theorem 1.28 — Properties of expectation
  1. (Law of the unconscious statistician) For a Borel function $g$, $$E[g(X)]=\sum_xg(x)\,p(x)\quad\text{or}\quad E[g(X)]=\int g(x)f(x)\,dx,$$ whenever the sum or integral converges absolutely. We never need the distribution of $g(X)$ itself.
  2. (Linearity) $E[aX+bY]=aE[X]+bE[Y]$ for integrable $X,Y$ defined on the same space, and $E[c]=c$.
  3. (Monotonicity) If $X\le Y$ then $E[X]\le E[Y]$; in particular $|E[X]|\le E|X|$.
  4. If $X\ge0$ and $E[X]=0$, then $P(X=0)=1$.

Proof. (1) Discrete case. Group the atoms of $X$ by the value of $g$: $$\sum_yy\,P(g(X)=y)=\sum_yy\sum_{x:\,g(x)=y}p(x)=\sum_xg(x)\,p(x).$$ The regrouping is valid because the series converges absolutely. Density case, $g\ge0$. Apply the tail formula to $g(X)$ and swap integrals (Tonelli): $$E[g(X)]=\int_0^\infty P(g(X)\gt t)\,dt=\int_0^\infty\int\mathbf{1}\{g(x)\gt t\}f(x)\,dx\,dt=\int f(x)\int_0^{g(x)}dt\,dx=\int g(x)f(x)\,dx.$$ For general $g$, write $g=g^+-g^-$. (2) and (3) are linearity and monotonicity of the Lebesgue integral; for a discrete space $E[X]=\sum_\omega X(\omega)P(\{\omega\})$, and both properties are visible. The second statement in (3) follows from $-|X|\le X\le|X|$. (4) For each $n$, $X\ge\tfrac1n\mathbf{1}\{X\gt\tfrac1n\}$, so $0=E[X]\ge\tfrac1nP(X\gt\tfrac1n)$. Hence $P(X\gt\tfrac1n)=0$ for all $n$, and $P(X\gt0)=0$ by continuity from below. ∎

📘 Definition 1.21 — Moments, variance, skewness and kurtosis

For $r\gt0$, when the expectations exist: the $r$-th raw moment is $\mu_r'=E[X^r]$, the $r$-th absolute moment is $E|X|^r$, and with $\mu=E[X]$ the $r$-th central moment is $\mu_r=E[(X-\mu)^r]$. The variance is $\operatorname{Var}(X)=\sigma^2=\mu_2$ and $\sigma$ is the standard deviation. For integer-valued $X$, the $r$-th factorial moment is $E[X(X-1)\cdots(X-r+1)]$.

The coefficients of skewness and kurtosis are $\gamma_1=\mu_3/\sigma^3$ and $\beta_2=\mu_4/\sigma^4$, exactly as for data in §1.1; for a normal law $\gamma_1=0$ and $\beta_2=3$.

📐 Theorem 1.29 — Variance and the two optimal centres

Let $E[X^2]\lt\infty$ and $\mu=E[X]$.

  1. $\operatorname{Var}(X)=E[X^2]-\mu^2$, and $\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)$.
  2. $E[(X-c)^2]=\operatorname{Var}(X)+(\mu-c)^2$, so $c=\mu$ uniquely minimises the mean squared deviation.
  3. If $E|X|\lt\infty$, then $c\mapsto E|X-c|$ is minimised exactly at the medians of $X$.
  4. $\operatorname{Var}(X)=0$ if and only if $P(X=\mu)=1$.
  5. (Lower moments exist) If $E|X|^s\lt\infty$ and $0\lt r\lt s$, then $E|X|^r\lt\infty$.

Proof. (1) Expand $(X-\mu)^2=X^2-2\mu X+\mu^2$ and use linearity: $E[X^2]-2\mu^2+\mu^2$. Also $aX+b-(a\mu+b)=a(X-\mu)$.

(2) $E[(X-c)^2]=E[(X-\mu)^2]$ $+2(\mu-c)E[X-\mu]+(\mu-c)^2$, and the middle term is $0$.

(3) Write $|X-c|=(X-c)^++(c-X)^+$ and apply the tail formula to each part: $$E(X-c)^+=\int_0^\infty P(X\gt c+t)\,dt=\int_c^\infty\left(1-F(s)\right)ds,$$ $$E(c-X)^+=\int_0^\infty P(X\lt c-t)\,dt=\int_{-\infty}^cF(s)\,ds,$$ using again that $F(s-)=F(s)$ except at countably many points. Let $m$ be a median and $\phi(c)=E|X-c|$. Subtracting the two expressions, for $c\gt m$, $$\phi(c)-\phi(m)=\int_m^c\left(2F(s)-1\right)ds\ge0,$$ since $F(s)\ge F(m)\ge\tfrac12$ for $s\ge m$; for $c\lt m$, $$\phi(c)-\phi(m)=\int_c^m\left(1-2F(s)\right)ds\ge0,$$ since $F(s)\le F(m-)\le\tfrac12$ for $s\lt m$. So every median is a minimiser. Conversely, if $c$ is not a median, then either $F(c)\lt\tfrac12$ (and $\phi$ strictly decreases just to the right of $c$, because $2F-1\lt0$ on some $[c,c+\delta)$ by right-continuity) or $F(c-)\gt\tfrac12$ (and $\phi$ strictly decreases as we move left from $c$). This is the population version of Theorem 1.2 of §1.1.

(4) If $\operatorname{Var}(X)=E[(X-\mu)^2]=0$, Theorem 1.28(4) applied to $(X-\mu)^2\ge0$ gives $X=\mu$ almost surely. The converse is immediate.

(5) $|x|^r\le1+|x|^s$ for all $x$ (consider $|x|\le1$ and $|x|\gt1$ separately), so $E|X|^r\le1+E|X|^s$. ∎

In particular, a finite variance forces a finite mean, and a finite fourth moment forces finite moments of every lower order. The sharper inequality $\left(E|X|^r\right)^{1/r}\le\left(E|X|^s\right)^{1/s}$ (Lyapunov) follows from Jensen's inequality in §2.2.

📘 Definition 1.22 — Symmetric distributions
$X$ is symmetric about $c$ if $X-c$ and $c-X$ have the same distribution. Equivalently $F(c-x)=1-F((c+x)-)$ for all $x$. If $X$ has a density, this holds iff the density can be chosen with $f(c-x)=f(c+x)$ for all $x$. Examples: the normal law about its mean, the uniform law on $(a,b)$ about $\tfrac{a+b}{2}$, the Cauchy law about $0$, the binomial law with $p=\tfrac12$ about $\tfrac n2$.
📐 Theorem 1.30 — Consequences of symmetry

Let $X$ be symmetric about $c$. Then:

  1. $c$ is a median of $X$;
  2. if $E|X|\lt\infty$ then $E[X]=c$, and every odd central moment $E[(X-c)^{2k+1}]$ that exists is $0$;
  3. if $c=0$ and $F$ is continuous, then $F(-x)=1-F(x)$, and $P(|X|\le x)=2F(x)-1$ for $x\ge0$.

Proof. Put $W=X-c$, so $W$ and $-W$ have the same law. (1) $P(W\lt0)=P(-W\lt0)=P(W\gt0)$, and these two numbers add up to at most $1$, so each is $\le\tfrac12$. Hence $P(X\lt c)\le\tfrac12$ and $P(X\le c)=1-P(X\gt c)\ge\tfrac12$: $c$ is a median. (2) $E[W]=E[-W]=-E[W]$, so $E[W]=0$; likewise $E[W^{2k+1}]=E[(-W)^{2k+1}]=-E[W^{2k+1}]$ when the moment exists. (3) $F(-x)=P(X\le-x)=P(-X\ge x)$ $=P(X\ge x)=1-F(x)$, using continuity at the last step; then $P(-x\le X\le x)=F(x)-F(-x)=2F(x)-1$. ∎

The hypothesis $E|X|\lt\infty$ in (2) cannot be dropped. The Cauchy law is symmetric about $0$, yet it has no mean at all (Example 6). Conversely, a vanishing third central moment does not imply symmetry (Exercise III.8).

✏️ Example 5 — Expectations from tail probabilities
(a) A fair die is rolled until the first six. Find the expected number $N$ of rolls. (b) $X$ is exponential with rate $\lambda$. Find $E[X]$, $E[X^2]$ and $\operatorname{Var}(X)$ from the tail formula. (c) Find the expected value of the larger of two independent fair dice.
(a) $N\ge n$ iff the first $n-1$ rolls are not sixes, so $P(N\ge n)=\left(\tfrac56\right)^{n-1}$ by independence. By Theorem 1.27(b), $$E[N]=\sum_{n=1}^\infty\left(\tfrac56\right)^{n-1}=\frac{1}{1-5/6}=6.$$ We never needed the pmf $P(N=n)=\left(\tfrac56\right)^{n-1}\tfrac16$.
(b) $P(X\gt x)=e^{-\lambda x}$. Then $$E[X]=\int_0^\infty e^{-\lambda x}dx=\frac1\lambda,\qquad E[X^2]=\int_0^\infty2x\,e^{-\lambda x}dx=\frac{2}{\lambda^2},$$ so $\operatorname{Var}(X)=\tfrac{2}{\lambda^2}-\tfrac{1}{\lambda^2}=\tfrac{1}{\lambda^2}$. The standard deviation equals the mean.
(c) Let $M$ be the maximum. $M\le n-1$ iff both dice show at most $n-1$, so $P(M\ge n)=1-\left(\tfrac{n-1}{6}\right)^2$ for $1\le n\le6$. Hence $$E[M]=\sum_{n=1}^6\left[1-\frac{(n-1)^2}{36}\right]=6-\frac{0+1+4+9+16+25}{36}=6-\frac{55}{36}=\frac{161}{36}\approx4.472.$$ Check by symmetry: the minimum is $7-\max$ of the reflected dice, so $E[\min]=7-\tfrac{161}{36}=\tfrac{91}{36}$, and $E[\max]+E[\min]=E[X_1+X_2]=7$.
✏️ Example 6 — Prove or disprove: existence of moments
(a) $p(k)=\dfrac{1}{k(k+1)}$, $k\in\mathbb{N}$, is a pmf with $E[X]\lt\infty$. (b) A random variable that is symmetric about $0$ has mean $0$. (c) If $E[X^2]\lt\infty$ then $E|X|\lt\infty$. (d) If $X\ge0$ and $x\,P(X\gt x)\to0$, then $E[X]\lt\infty$. (e) The Pareto density $f(x)=2x^{-3}$ on $(1,\infty)$ has a finite mean and a finite variance.
(a) False It is a pmf: $\frac{1}{k(k+1)}=\frac1k-\frac1{k+1}$ telescopes, so the sum is $1$, and $P(X\ge n)=\tfrac1n$. But $E[X]=\sum_nP(X\ge n)=\sum_n\tfrac1n=\infty$. Every value is finite, yet the mean is infinite.
(b) False For the Cauchy density $\frac{1}{\pi(1+x^2)}$, $$E[X^+]=\int_0^\infty\frac{x\,dx}{\pi(1+x^2)}=\lim_{R\to\infty}\frac{\log(1+R^2)}{2\pi}=\infty,$$ and $E[X^-]=\infty$ likewise, so $E[X]$ does not exist. (The symmetric limit $\lim_R\int_{-R}^R$ is $0$, but that "principal value" is not an expectation: $\int_{-R}^{2R}$ tends to $\frac{\log 2}{\pi}$.) The median is $0$, though.
(c) True Theorem 1.29(5) with $r=1$, $s=2$: $|x|\le1+x^2$. (So a finite variance implies a finite mean.)
(d) False Let $P(X\gt x)=1$ for $x\lt e$ and $P(X\gt x)=\frac{1}{x\log x}$ for $x\ge e$. This is a legitimate survival function: it is nonincreasing (because $x\log x$ increases), right-continuous, and tends to $0$. (It has an atom of mass $1-e^{-1}$ at $e$.) Then $x\,P(X\gt x)=\frac{1}{\log x}\to0$, but $$E[X]=e+\int_e^\infty\frac{dx}{x\log x}=e+\lim_{R\to\infty}\log\log R=\infty.$$
(e) False $E[X]=\int_1^\infty2x^{-2}dx=2$, but $E[X^2]=\int_1^\infty2x^{-1}dx=\infty$. In general the Pareto density $\alpha x^{-\alpha-1}$ ($x\gt1$) has $E[X^r]\lt\infty$ iff $r\lt\alpha$ (Exercise III.3).
Interactive 2 — The Density of Y = g(X): Histogram versus Formula

Choose a transformation and press Play. A seeded generator draws $X$, computes $Y=g(X)$, and adds it to the blue histogram (scaled as a density). The red curve is the density from Theorem 1.26. In the $X^2$ case with $X$ uniform on $(-1,2)$, the dashed amber curve is the wrong one-branch formula: watch the histogram refuse to follow it below $y=1$. In the Cauchy case, watch the running mean: it never settles, because $E[Y]$ does not exist.

samples—
running mean of Y—
exact E[Y]—
outside the window—

Generating Functions

A generating function packs a whole sequence of probabilities or moments into one function. Differentiating it recovers them, multiplying generating functions corresponds to adding independent random variables (§1.4), and their limits detect convergence in distribution (§2.2). Two kinds are used throughout Unit 4.

📘 Definition 1.23 — Probability and moment generating functions

(a) If $X$ takes values in $\{0,1,2,\dots\}$ with $p_k=P(X=k)$, its probability generating function (pgf) is $$G_X(s)=E\left[s^X\right]=\sum_{k=0}^\infty p_ks^k,\qquad |s|\le1.$$

(b) The moment generating function (mgf) of $X$ is $M_X(t)=E\left[e^{tX}\right]$ for those real $t$ where it is finite. We say $X$ has an mgf if $M_X(t)\lt\infty$ for all $t$ in some interval $(-h,h)$, $h\gt0$. The cumulant generating function is $K_X(t)=\log M_X(t)$. For integer-valued $X$, $M_X(t)=G_X(e^t)$.

📐 Theorem 1.31 — Properties of the pgf
  1. The power series converges for $|s|\le1$ and $G_X(1)=1$. Moreover $p_k=\dfrac{G_X^{(k)}(0)}{k!}$, so $G_X$ determines the distribution.
  2. For $r\in\mathbb{N}$, the factorial moment is $E[X(X-1)\cdots(X-r+1)]=\lim_{s\uparrow1}G_X^{(r)}(s)$, finite or $+\infty$. In particular $$E[X]=G_X'(1),\qquad\operatorname{Var}(X)=G_X''(1)+G_X'(1)-G_X'(1)^2.$$
  3. For constants $a,b\in\{0,1,2,\dots\}$, $G_{aX+b}(s)=s^bG_X(s^a)$.

Proof. (1) $\sum p_k|s|^k\le\sum p_k=1$ for $|s|\le1$, so the radius of convergence is at least $1$. Inside the disc a power series can be differentiated term by term, and $G^{(k)}(0)=k!\,p_k$. (2) For $0\le s\lt1$, $$G^{(r)}(s)=\sum_{k\ge r}k(k-1)\cdots(k-r+1)\,p_k\,s^{k-r}.$$ The terms are nonnegative and increase with $s$, so by monotone convergence the limit as $s\uparrow1$ is $\sum_kk(k-1)\cdots(k-r+1)p_k$. For the variance, $E[X^2]=E[X(X-1)]+E[X]$. (3) $E[s^{aX+b}]=s^bE[(s^a)^X]$. ∎

📐 Theorem 1.32 — Properties of the mgf

Suppose $M_X(t)\lt\infty$ for $|t|\lt h$, where $h\gt0$. Then:

  1. $E|X|^r\lt\infty$ for every $r\gt0$;
  2. $M_X(t)=\sum_{r=0}^\infty\dfrac{t^r}{r!}E[X^r]$ for $|t|\lt h$, and hence $E[X^r]=M_X^{(r)}(0)$;
  3. $M_{aX+b}(t)=e^{bt}M_X(at)$;
  4. $K_X'(0)=E[X]$ and $K_X''(0)=\operatorname{Var}(X)$;
  5. (Uniqueness, statement) if $M_X(t)=M_Y(t)$ for all $t$ in a neighbourhood of $0$, then $X$ and $Y$ have the same distribution.

Proof. Fix $0\lt t_0\lt h$. Then $e^{t_0|x|}\le e^{t_0x}+e^{-t_0x}$, so $E\left[e^{t_0|X|}\right]\le M(t_0)+M(-t_0)\lt\infty$. (1) $|x|^r/e^{t_0|x|}$ is bounded on $\mathbb{R}$ (it tends to $0$ at $\pm\infty$), so $E|X|^r\le C_r\,E\left[e^{t_0|X|}\right]\lt\infty$. (2) For $|t|\le t_0$, the partial sums of $e^{tX}=\sum_r\frac{(tX)^r}{r!}$ are dominated by $\sum_r\frac{|tX|^r}{r!}=e^{|t||X|}\le e^{t_0|X|}$, which is integrable. By dominated convergence we may take expectations term by term: $M(t)=\sum_r\frac{t^r}{r!}E[X^r]$. Since $t_0\lt h$ was arbitrary, this power series converges on $(-h,h)$, and differentiating it $r$ times at $0$ gives $E[X^r]$. (3) $E[e^{t(aX+b)}]=e^{bt}E[e^{(at)X}]$. (4) $K'=M'/M$ and $K''=(M''M-M'^2)/M^2$. At $t=0$, $M(0)=1$, $M'(0)=E[X]$ and $M''(0)=E[X^2]$. (5) The proof uses the characteristic function and its inversion formula (§2.2). ∎

✏️ Example 7 — Reading moments off generating functions
(a) $X$ has pgf $G(s)=\dfrac{(2+s)^3}{27}$. Find $P(X=2)$, $E[X]$ and $\operatorname{Var}(X)$. (b) $X$ has mgf $M(t)=(1-2t)^{-3}$ for $t\lt\tfrac12$. Find $E[X]$, $E[X^2]$ and $\operatorname{Var}(X)$, and check with the cumulant generating function. (c) Identify the distribution with mgf $M(t)=\tfrac13e^{-t}+\tfrac16+\tfrac12e^{2t}$, and find its mean and variance. (d) Find the pgf of the number $N$ of rolls of a fair die up to and including the first six, and use it to find $E[N]$ and $\operatorname{Var}(N)$.
(a) $G(s)=\left(\tfrac23+\tfrac13s\right)^3$, the number of successes in $3$ independent trials with success probability $\tfrac13$. The coefficient of $s^2$ is $\frac{3\cdot2}{27}=\tfrac29$. $G'(s)=\frac{(2+s)^2}{9}$ and $G''(s)=\frac{2(2+s)}{9}$, so $E[X]=G'(1)=1$, $G''(1)=\tfrac23$ and $\operatorname{Var}(X)=\tfrac23+1-1=\tfrac23$.
(b) $M'(t)=6(1-2t)^{-4}$ and $M''(t)=48(1-2t)^{-5}$, so $E[X]=6$, $E[X^2]=48$, $\operatorname{Var}(X)=48-36=12$. Check: $K(t)=-3\log(1-2t)$ gives $K'(t)=\frac{6}{1-2t}$ and $K''(t)=\frac{12}{(1-2t)^2}$, which equal $6$ and $12$ at $t=0$. (This is the gamma law with shape $3$ and scale $2$; §2.1.)
(c) $M(t)=\sum_xe^{tx}P(X=x)$ for a discrete law, so by uniqueness $X$ takes the values $-1,0,2$ with probabilities $\tfrac13,\tfrac16,\tfrac12$. Then $E[X]=-\tfrac13+1=\tfrac23$, $E[X^2]=\tfrac13+2=\tfrac73$, and $\operatorname{Var}(X)=\tfrac73-\tfrac49=\tfrac{17}{9}$.
(d) $P(N=n)=\left(\tfrac56\right)^{n-1}\tfrac16$, so $$G(s)=\sum_{n\ge1}\tfrac16\left(\tfrac56\right)^{n-1}s^n=\frac{s/6}{1-5s/6}=\frac{s}{6-5s},\quad |s|\lt\tfrac65.$$ Then $G'(s)=\frac{6}{(6-5s)^2}$ and $G''(s)=\frac{60}{(6-5s)^3}$, so $E[N]=6$ (as in Example 5) and $\operatorname{Var}(N)=60+6-36=30$.
⚠️ Counterexamples and pitfalls
  1. A continuous cdf need not have a density. The Cantor function is a continuous cdf whose derivative is $0$ except on the Cantor set, a set of length $0$. So $\int F'=0\ne1$. In exam language, "continuous random variable" means "has a density".
  2. Right-continuous, not left-continuous. $P(X\lt x)=F(x-)$, which differs from $F(x)$ at an atom. A function such as $\mathbf{1}\{x\gt0\}$ is not a cdf (Example 2(c)).
  3. A density is not a probability. $f(x)=\frac{1}{2\sqrt x}$ on $(0,1)$ is a density (of $U^2$) that is unbounded, and $f(x)=10$ on $(0,0.1)$ exceeds $1$. Only $\int_Bf$ is a probability.
  4. Symmetric does not mean "mean equals centre". The Cauchy law is symmetric about $0$ and has no mean (Example 6(b)). Symmetry gives $E[X]=c$ only when $E|X|\lt\infty$.
  5. Finite values, infinite mean. $P(X=k)=\frac1{k(k+1)}$ has $E[X]=\infty$. A Pareto law with $\alpha=2$ has a mean but no variance.
  6. $x\,P(X\gt x)\to0$ does not give a finite mean (Example 6(d)). It is necessary, not sufficient.
  7. $E[g(X)]\ne g(E[X])$ in general. For $X$ uniform on $\{1,2\}$, $E[1/X]=\tfrac34$ but $1/E[X]=\tfrac23$. Equality for all $X$ holds only for affine $g$.
  8. All moments finite, but no mgf. For the lognormal $X=e^Z$ ($Z$ standard normal), $E[X^r]=e^{r^2/2}$ for every $r$, but $E[e^{tX}]=\infty$ for every $t\gt0$. Moreover the lognormal moments are shared by other distributions, so moments need not determine a law. Moments do determine it when an mgf exists (Theorem 1.32(5)).
  9. The one-branch change-of-variables formula. For non-monotone $g$, every preimage contributes. For $Y=X^2$ with $X$ uniform on $(-1,2)$, using only $x=\sqrt y$ gives "density" of total mass $\tfrac23$ (Example 3(c)).
  10. $F(X)$ is uniform only for continuous $F$. For a fair coin with values $0,1$, $F(X)\in\{\tfrac12,1\}$. The quantile transform $Q(U)\sim F$, by contrast, works for every $F$.
  11. Medians need not be unique, and $Q(\tfrac12)$ is only the smallest one: for three fair coin tosses every $m\in[1,2]$ is a median (Example 4(b)).
  12. $\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)$, not $a\operatorname{Var}(X)+b$. Shifting does not change spread, and the scale factor is squared.

Exercises

I
Distribution functions and their types.
  • 1 $F(x)=0$ for $x\lt0$, $\tfrac x4$ for $0\le x\lt1$, $\tfrac12$ for $1\le x\lt2$, and $1$ for $x\ge2$. Find $P(X=1)$, $P(X=2)$, $P(\tfrac12\lt X\le\tfrac32)$, $P(X\lt1)$, $P(1\le X\lt2)$ and $E[X]$.
  • 2 Find all constants $a,b$ for which $F(x)=a+b\arctan x$ is a distribution function.
  • 3 Show that if $F$ is a cdf and $n\in\mathbb{N}$, then $F^n$ and $1-(1-F)^n$ are cdfs. (§1.4 interprets them as the laws of a maximum and a minimum.)
  • 4 $X$ has a continuous cdf $F$. Find the cdf of $Y=\max(X,0)$ and decide whether $Y$ is discrete, continuous or mixed.
  • 5 Let $q_1,q_2,\dots$ enumerate the rationals. Show that $F(x)=\sum_{n:\,q_n\le x}2^{-n}$ is a cdf that is discontinuous at every rational number.
  • 6 Prove that $P(X=x)=0$ for every $x$ if and only if $F$ is continuous.
  • 7 Find $c$ so that $f(x)=cx(1-x)$ on $(0,1)$ is a density. Find $F$, $P(X\gt\tfrac12)$ and $P(\tfrac14\lt X\lt\tfrac34)$.
  • 8 Find $c$ so that $p(k)=\dfrac{c}{k(k+1)(k+2)}$, $k\in\mathbb{N}$, is a pmf. Find $E[X]$, and decide whether $E[X^2]$ is finite.
1 $P(X=1)=\tfrac12-\tfrac14=\tfrac14$, $P(X=2)=1-\tfrac12=\tfrac12$, $P(\tfrac12\lt X\le\tfrac32)=\tfrac12-\tfrac18=\tfrac38$, $P(X\lt1)=F(1-)=\tfrac14$, $P(1\le X\lt2)=F(2-)-F(1-)=\tfrac14$. $E[X]=\int_0^1\tfrac x4dx+1\cdot\tfrac14+2\cdot\tfrac12=\tfrac18+\tfrac14+1=\tfrac{11}8$. Check: $\int_0^2(1-F)=\int_0^1(1-\tfrac x4)dx+\int_1^2\tfrac12dx=\tfrac78+\tfrac12=\tfrac{11}8$.
2 The limits force $a-\tfrac{b\pi}{2}=0$ and $a+\tfrac{b\pi}{2}=1$, so $a=\tfrac12$, $b=\tfrac1\pi$. Then $F$ is continuous and increasing (since $b\gt0$): this is the Cauchy cdf.
3 $t\mapsto t^n$ and $t\mapsto1-(1-t)^n$ are continuous and nondecreasing on $[0,1]$, fix $0$ and $1$. Composing with $F$ preserves monotonicity, right-continuity and the limits $0,1$.
4 $F_Y(y)=0$ for $y\lt0$, and $F_Y(y)=P(X\le y)=F(y)$ for $y\ge0$ (since $\max(X,0)\le y\iff X\le y$ when $y\ge0$). There is a jump of $F(0)$ at $0$, and $F_Y$ is continuous elsewhere. So $Y$ is mixed if $0\lt F(0)\lt1$, continuous if $F(0)=0$, and $Y\equiv0$ if $F(0)=1$.
5 $F$ is the cdf of the discrete law $P(X=q_n)=2^{-n}$ (these sum to $1$). Its jump at $q_n$ is $2^{-n}\gt0$, so it is discontinuous at every rational, yet it has only countably many jumps, as Theorem 1.23(5) requires. It is continuous at every irrational.
6 By Theorem 1.23(4), $P(X=x)=F(x)-F(x-)$. Since $F$ is right-continuous, it is continuous at $x$ iff $F(x-)=F(x)$, i.e. iff $P(X=x)=0$.
7 $\int_0^1x(1-x)dx=\tfrac16$, so $c=6$. $F(x)=3x^2-2x^3$ on $[0,1]$. By symmetry about $\tfrac12$, $P(X\gt\tfrac12)=\tfrac12$. $P(\tfrac14\lt X\lt\tfrac34)=F(\tfrac34)-F(\tfrac14)=\tfrac{27}{32}-\tfrac{5}{32}=\tfrac{11}{16}$.
8 $\frac{1}{k(k+1)(k+2)}=\frac12\left(\frac{1}{k(k+1)}-\frac{1}{(k+1)(k+2)}\right)$ telescopes to $\tfrac12\cdot\tfrac12=\tfrac14$, so $c=4$. $E[X]=\sum\frac{4}{(k+1)(k+2)}=4\cdot\tfrac12=2$. $E[X^2]=\sum\frac{4k}{(k+1)(k+2)}=\infty$, because the terms behave like $\tfrac4k$.
II
Quantiles and transformations.
  • 1 $X$ is uniform on $(0,1)$. Find the density of $Y=1/X$, and show that $E[Y]=\infty$.
  • 2 $X$ is exponential with rate $1$. Find the densities of $\sqrt X$ and of $e^{-X}$.
  • 3 $X$ has density $2x$ on $(0,1)$. Show that $-\log X$ is exponential and find its rate.
  • 4 $Z$ is standard normal. Find the density of $|Z|$ and compute $E|Z|$.
  • 5 $X$ is uniform on $(-\tfrac\pi2,\tfrac\pi2)$. Find the density of $\sin X$.
  • 6 Find the quantile function and the median of the Pareto law $F(x)=1-x^{-\alpha}$, $x\ge1$ ($\alpha\gt0$), and explain how to simulate it from a uniform random number.
  • 7 $P(X=0)=0.2$, $P(X=1)=0.3$, $P(X=3)=0.5$. Find $Q(0.2)$, $Q(0.25)$, $Q(0.5)$, $Q(0.9)$ and the set of all medians.
  • 8 $X$ has a continuous, strictly increasing cdf $F$. Show that $-2\log F(X)$ has density $\tfrac12e^{-y/2}$, $y\gt0$. (This is Fisher's device for combining $p$-values.)
1 $h(y)=1/y$ on $(1,\infty)$, $|h'(y)|=y^{-2}$, so $f_Y(y)=y^{-2}$ for $y\gt1$. $E[Y]=\int_1^\infty y\cdot y^{-2}dy=\infty$. (Equivalently $P(Y\gt y)=\tfrac1y$ is not integrable.)
2 $Y=\sqrt X$: $h(y)=y^2$, so $f_Y(y)=2y\,e^{-y^2}$ for $y\gt0$ (a Rayleigh law). $W=e^{-X}$: $h(w)=-\log w$ on $(0,1)$, $|h'(w)|=\tfrac1w$, so $f_W(w)=e^{\log w}\cdot\tfrac1w=1$: $W$ is uniform on $(0,1)$, as Theorem 1.25(3) predicts, since $e^{-X}=1-F(X)$.
3 $P(-\log X\gt y)=P(X\lt e^{-y})=(e^{-y})^2=e^{-2y}$ for $y\gt0$. So $-\log X$ is exponential with rate $2$.
4 For $y\gt0$, $P(|Z|\le y)=2\Phi(y)-1$ (Theorem 1.30(3)), so $f_{|Z|}(y)=2\varphi(y)$. $E|Z|=\int_0^\infty2y\frac{e^{-y^2/2}}{\sqrt{2\pi}}dy=\frac{2}{\sqrt{2\pi}}=\sqrt{2/\pi}\approx0.798$.
5 $\sin$ is increasing on $(-\tfrac\pi2,\tfrac\pi2)$ with inverse $\arcsin$ on $(-1,1)$, and $|h'(y)|=\frac{1}{\sqrt{1-y^2}}$. So $f(y)=\frac{1}{\pi\sqrt{1-y^2}}$ on $(-1,1)$, the arcsine law. It piles up near $\pm1$, where the sine is flattest.
6 Solve $1-x^{-\alpha}=u$: $Q(u)=(1-u)^{-1/\alpha}$. The median is $2^{1/\alpha}$. Simulate by $X=Q(U)$, or equally $X=U^{-1/\alpha}$, because $1-U$ is uniform too.
7 $F(0)=0.2$, $F(1)=0.5$, $F(3)=1$. $Q(0.2)=0$ (since $F(0)\ge0.2$); $Q(0.25)=1$; $Q(0.5)=1$; $Q(0.9)=3$. Medians: $F(m-)\le\tfrac12\le F(m)$ holds for every $m\in[1,3]$, because $F=\tfrac12$ on $[1,3)$ and $F(3-)=\tfrac12$.
8 $U=F(X)$ is uniform by Theorem 1.25(3). $Y=-2\log U$ is a decreasing function with $h(y)=e^{-y/2}$, $|h'(y)|=\tfrac12e^{-y/2}$, so $f_Y(y)=\tfrac12e^{-y/2}$, $y\gt0$: an exponential with mean $2$, which is the chi-square law with $2$ degrees of freedom.
III
Expectation, variance and the tail formula.
  • 1 $X$ has density $6x(1-x)$ on $(0,1)$. Find $E[X]$ and $\operatorname{Var}(X)$.
  • 2 Using the tail formula, find $E[X]$ for $X$ uniform on $(0,\theta)$, and $E[\min(X,c)]$ for $X$ exponential with rate $\lambda$ and a constant $c\gt0$.
  • 3 For the Pareto density $f(x)=\alpha x^{-\alpha-1}$, $x\gt1$, show that $E[X^r]\lt\infty$ iff $r\lt\alpha$, and find $E[X]$ ($\alpha\gt1$) and $\operatorname{Var}(X)$ ($\alpha\gt2$).
  • 4 Three fair dice are rolled. Use $E[X]=\sum_{n\ge1}P(X\ge n)$ to find the expected value of the smallest score.
  • 5 For $X$ exponential with rate $1$, find $\min_cE|X-c|$ and the minimising $c$.
  • 6 $X$ takes values in $[0,1]$ and has mean $\mu$. Prove that $\operatorname{Var}(X)\le\mu(1-\mu)\le\tfrac14$, and find when equality holds in the first inequality.
  • 7 (Insurance) A claim $X$ is exponential with mean $\text{₹}\,2000$. The policy has a deductible of $\text{₹}\,500$, so it pays $Y=\max(X-500,0)$. Find $P(Y=0)$ and $E[Y]$.
  • 8 Show that if $X$ is symmetric about $0$ and $E|X|^3\lt\infty$, then $E[X^3]=0$. Then give an $X$ with $E[X]=0$ and $E[X^3]\ne0$, and an $X$ with $E[X]=E[X^3]=0$ that is not symmetric.
1 $E[X]=\tfrac12$ (symmetry about $\tfrac12$, or direct integration). $E[X^2]=6\int_0^1(x^3-x^4)dx=6\left(\tfrac14-\tfrac15\right)=\tfrac{3}{10}$, so $\operatorname{Var}(X)=\tfrac3{10}-\tfrac14=\tfrac1{20}$.
2 $\int_0^\theta\left(1-\tfrac x\theta\right)dx=\tfrac\theta2$. $\min(X,c)\gt x$ iff $X\gt x$ and $x\lt c$, so $E[\min(X,c)]=\int_0^ce^{-\lambda x}dx=\frac{1-e^{-\lambda c}}{\lambda}$.
3 $E[X^r]=\int_1^\infty\alpha x^{r-\alpha-1}dx$ is finite iff $r-\alpha-1\lt-1$, i.e. $r\lt\alpha$, and then equals $\frac{\alpha}{\alpha-r}$. So $E[X]=\frac{\alpha}{\alpha-1}$ and $\operatorname{Var}(X)=\frac{\alpha}{\alpha-2}-\frac{\alpha^2}{(\alpha-1)^2}=\frac{\alpha}{(\alpha-1)^2(\alpha-2)}$.
4 $P(\min\ge n)=\left(\frac{7-n}{6}\right)^3$ for $1\le n\le6$. Summing, $E[\min]=\frac{216+125+64+27+8+1}{216}=\frac{441}{216}=\frac{49}{24}\approx2.042$.
5 By Theorem 1.29(3) the minimiser is the median $c=\log2$. Using the formula in its proof, $E|X-c|=\int_0^c(1-e^{-x})dx+\int_c^\infty e^{-x}dx=c-1+2e^{-c}$, which at $c=\log2$ equals $\log2\approx0.693$.
6 $0\le X\le1$ gives $X^2\le X$, so $E[X^2]\le\mu$ and $\operatorname{Var}(X)\le\mu-\mu^2$. Next, $\mu(1-\mu)\le\tfrac14$ by AM–GM. Equality in the first step needs $E[X-X^2]=0$ with $X-X^2\ge0$, so $X\in\{0,1\}$ almost surely (Theorem 1.28(4)): $X$ is a Bernoulli variable.
7 With $\theta=2000$, $P(Y=0)=P(X\le500)=1-e^{-0.25}\approx0.2212$. By the tail formula, $E[Y]=\int_{500}^\infty e^{-x/2000}dx=2000\,e^{-0.25}\approx\text{₹}\,1557.60$. (Memorylessness, §2.1, gives the same: the excess over $500$ of a claim above $500$ is again exponential with mean $2000$.)
8 The first claim is Theorem 1.30(2). For the second, take $P(X=-1)=\tfrac23$ and $P(X=2)=\tfrac13$: $E[X]=0$ and $E[X^3]=-\tfrac23+\tfrac83=2$. For the third, take $P(X=-3)=\tfrac{1}{10}$, $P(X=-1)=\tfrac{5}{10}$, $P(X=2)=\tfrac{4}{10}$: then $E[X]=-0.3-0.5+0.8=0$ and $E[X^3]=-2.7-0.5+3.2=0$, but $P(X=2)\ne P(X=-2)=0$.
IV
Generating functions.
  • 1 Find the pgf of $X$ uniform on $\{0,1,\dots,n\}$ and deduce $E[X]$.
  • 2 $P(X=k)=e^{-\lambda}\lambda^k/k!$, $k\ge0$ (Poisson). Find $G_X$, all factorial moments, $E[X]$ and $\operatorname{Var}(X)$.
  • 3 $X$ has pgf $G(s)=\dfrac{1}{2-s}$. Find $P(X=k)$, $E[X]$ and $\operatorname{Var}(X)$.
  • 4 Which of the following are pgfs? (a) $\tfrac12(s+s^2)$; (b) $\frac{1-s}{1+s}$; (c) $s\,e^{s-1}$; (d) $\frac{2}{3-s}$.
  • 5 $X$ is exponential with rate $\lambda$. Find $M_X$ and its domain, and deduce $E[X^r]=r!/\lambda^r$.
  • 6 $X$ is uniform on $(a,b)$. Find $M_X$, and from it the mean and variance.
  • 7 Identify the distribution with mgf $M(t)=\dfrac{e^t}{2-e^t}$, $t\lt\log2$, and find its mean and variance.
  • 8 Show that the Cauchy law has $M(t)=\infty$ for every $t\ne0$. Show that the Laplace density $\tfrac12e^{-|x|}$ has mgf $\frac{1}{1-t^2}$ for $|t|\lt1$, and find its variance and kurtosis $\beta_2$.
1 $G(s)=\frac{1}{n+1}\sum_{k=0}^ns^k=\frac{1-s^{n+1}}{(n+1)(1-s)}$ for $s\ne1$. Differentiating the finite sum, $G'(1)=\frac{1}{n+1}\sum_kk=\frac n2$.
2 $G(s)=e^{-\lambda}\sum\frac{(\lambda s)^k}{k!}=e^{\lambda(s-1)}$, so $G^{(r)}(1)=\lambda^r$: the $r$-th factorial moment is $\lambda^r$. $E[X]=\lambda$ and $\operatorname{Var}(X)=\lambda^2+\lambda-\lambda^2=\lambda$.
3 $\frac{1}{2-s}=\frac12\cdot\frac{1}{1-s/2}=\sum_k2^{-(k+1)}s^k$, so $P(X=k)=2^{-(k+1)}$, $k\ge0$ (failures before the first success, $p=\tfrac12$). $G'(1)=1$ and $G''(1)=2$, so $E[X]=1$ and $\operatorname{Var}(X)=2+1-1=2$.
4 A pgf is a power series with nonnegative coefficients summing to $1$. (a) Yes. (b) No: $\frac{1-s}{1+s}=1-2s+2s^2-\cdots$ has negative coefficients (and its value at $s=1$ is $0$). (c) Yes: it is the pgf of $1+Y$ with $Y$ Poisson$(1)$, by Theorem 1.31(3). (d) Yes: $\frac{2}{3-s}=\sum_k\tfrac23\left(\tfrac13\right)^ks^k$, and the value at $1$ is $1$.
5 $M(t)=\int_0^\infty\lambda e^{-(\lambda-t)x}dx=\frac{\lambda}{\lambda-t}$ for $t\lt\lambda$ (and $\infty$ for $t\ge\lambda$). Expand: $\frac{1}{1-t/\lambda}=\sum_r\frac{t^r}{\lambda^r}$. Comparing with $\sum_r\frac{t^r}{r!}E[X^r]$ gives $E[X^r]=\frac{r!}{\lambda^r}$.
6 $M(t)=\frac{e^{tb}-e^{ta}}{t(b-a)}$ for $t\ne0$, $M(0)=1$. The series is $M(t)=\frac{1}{b-a}\sum_{r\ge1}\frac{t^{r-1}(b^r-a^r)}{r!}$, so $E[X]=\frac{b^2-a^2}{2(b-a)}=\frac{a+b}2$ and $E[X^2]=\frac{b^3-a^3}{3(b-a)}=\frac{a^2+ab+b^2}{3}$. Hence $\operatorname{Var}(X)=\frac{(b-a)^2}{12}$.
7 With $s=e^t$, $M(t)=G(e^t)$ where $G(s)=\frac{s}{2-s}=\sum_{k\ge1}2^{-k}s^k$. So $P(X=k)=2^{-k}$, $k\ge1$ (trials up to the first success, $p=\tfrac12$). $G'(1)=2$ and $G''(1)=4$, so $E[X]=2$ and $\operatorname{Var}(X)=4+2-4=2$.
8 For $t\gt0$, $\frac{e^{tx}}{\pi(1+x^2)}\to\infty$ as $x\to\infty$, so the integral diverges; similarly for $t\lt0$. For the Laplace law with $|t|\lt1$, $M(t)=\tfrac12\int_0^\infty e^{-(1-t)x}dx+\tfrac12\int_0^\infty e^{-(1+t)x}dx=\frac{1}{2(1-t)}+\frac{1}{2(1+t)}=\frac{1}{1-t^2}$. The expansion $\sum_kt^{2k}$ gives $E[X^2]=2!=2$ and $E[X^4]=4!=24$, with odd moments $0$. So $\operatorname{Var}(X)=2$ and $\beta_2=\frac{24}{4}=6$: heavier tails than the normal ($\beta_2=3$).
🎯 Exam corner CSIR Part BCSIR Part CGATE ST

E1. CSIR Part B A random variable has cdf $F(x)=0$ for $x\lt0$, $F(x)=\frac{1+x}{3}$ for $0\le x\lt1$, and $F(x)=1$ for $x\ge1$. Then $E[X]$ equals (A) $\tfrac13$; (B) $\tfrac12$; (C) $\tfrac23$; (D) $\tfrac56$.

Idea $X\ge0$, so use the tail formula; there is no need to split into atoms and density.
Compute $E[X]=\int_0^1\left(1-\frac{1+x}{3}\right)dx=\int_0^1\frac{2-x}{3}dx=\frac{2-\frac12}{3}=\frac12$. Check: atoms $\tfrac13$ at $0$ and $1-\tfrac23=\tfrac13$ at $1$, density $\tfrac13$ on $(0,1)$: $E[X]=\tfrac13+\tfrac13\cdot\tfrac12=\tfrac12$. Option (A) forgets the density part, and (D) adds $\tfrac13$ for the atom at $0$ as if it were at $1$.
Answer (B).

E2. CSIR Part B GATE ST $U$ is uniform on $(0,1)$ and $Y=-2\log U$. Then $P(Y\gt2)$ equals (A) $e^{-2}$; (B) $e^{-1}$; (C) $1-e^{-1}$; (D) $\tfrac12$.

Idea Translate the event back to $U$: $Y\gt2\iff\log U\lt-1\iff U\lt e^{-1}$.
Compute $P(U\lt e^{-1})=e^{-1}\approx0.368$. Indeed $Y$ is exponential with mean $2$, and $P(Y\gt2)=e^{-2/2}$. Option (A) treats $Y$ as having rate $1$.
Answer (B).

E3. CSIR Part B $X$ has mgf $M(t)=(1-3t)^{-2}$ for $t\lt\tfrac13$. Then $\operatorname{Var}(X)$ is (A) $6$; (B) $9$; (C) $18$; (D) $54$.

Idea Use the cumulant generating function $K(t)=-2\log(1-3t)$.
Compute $K'(t)=\frac{6}{1-3t}$ and $K''(t)=\frac{18}{(1-3t)^2}$, so $E[X]=6$ and $\operatorname{Var}(X)=18$. (Via $M$: $M''(0)=54=E[X^2]$, and $54-36=18$. Option (D) is $E[X^2]$, option (A) the mean.)
Answer (C).

E4. CSIR Part C Which of the following functions are distribution functions? (One or more options may be correct.) (A) $F(x)=e^{-e^{-x}}$, $x\in\mathbb{R}$. (B) $F(x)=0$ for $x\lt0$ and $F(x)=1-\tfrac12e^{-x}$ for $x\ge0$. (C) $F(x)=0$ for $x\le0$ and $F(x)=1$ for $x\gt0$. (D) $F(x)=0$ for $x\lt0$ and $F(x)=1-e^{-x}(1+\sin x)$ for $x\ge0$.

(A) Yes Continuous, strictly increasing, with limits $0$ and $1$ (Gumbel law; Example 2(a)).
(B) Yes Nondecreasing, right-continuous, limits $0$ and $1$. It jumps by $\tfrac12$ at $0$: a mixed law, half an atom at $0$ and half an exponential.
(C) No Not right-continuous at $0$: $F(0)=0$ but $F(0+)=1$.
(D) No It looks plausible ($F(0)=0$, $F\to1$, continuous), but at $x=\tfrac{3\pi}{2}$ we have $\sin x=-1$, so $F(\tfrac{3\pi}2)=1$, while $F(2\pi)=1-e^{-2\pi}\lt1$. So $F$ decreases on part of $(\tfrac{3\pi}{2},2\pi)$. Indeed $F'(x)=e^{-x}(1+\sin x-\cos x)$, which is negative there.
Answer (A) and (B).

E5. CSIR Part C Let $X$ be a nonnegative random variable with cdf $F$. Which of the following are true? (A) $E[X]=\int_0^\infty(1-F(x))\,dx$, both sides possibly infinite. (B) If $E[X]\lt\infty$ then $x\,P(X\gt x)\to0$ as $x\to\infty$. (C) If $x\,P(X\gt x)\to0$ then $E[X]\lt\infty$. (D) If $X$ is integer-valued, then $E[X]=\sum_{n=0}^\infty P(X\gt n)$.

(A) True Theorem 1.27(a), by Tonelli's theorem; no integrability assumption is needed.
(B) True $x\,P(X\gt x)\le E[X\mathbf{1}\{X\gt x\}]\to0$ (Theorem 1.27(d)).
(C) False $P(X\gt x)=\frac{1}{x\log x}$ for $x\ge e$ has $xP(X\gt x)=\frac1{\log x}\to0$ but $E[X]=\infty$ (Example 6(d)).
(D) True $P(X\gt n)=P(X\ge n+1)$ for integer $X$, so the sum is $\sum_{m\ge1}P(X\ge m)=E[X]$.
Answer (A), (B) and (D).

E6. CSIR Part C Which of the following are true? (A) If $X$ is symmetric about $0$, then $E[X]=0$. (B) If $E[e^{tX}]\lt\infty$ for all $|t|\lt h$, for some $h\gt0$, then $E|X|^r\lt\infty$ for every $r\gt0$. (C) If $E|X|^r\lt\infty$ for every $r\gt0$, then $E[e^{tX}]\lt\infty$ for all $t$ in some neighbourhood of $0$. (D) If $X$ has a continuous cdf $F$, then $F(X)$ is uniform on $(0,1)$.

(A) False The Cauchy law is symmetric about $0$ but $E[X]$ does not exist.
(B) True Theorem 1.32(1): $|x|^r\le C\left(e^{t_0x}+e^{-t_0x}\right)$ for any fixed $0\lt t_0\lt h$.
(C) False The lognormal $e^Z$ has every moment finite, $E[e^{rZ}]=e^{r^2/2}$, yet $E[e^{te^Z}]=\infty$ for every $t\gt0$, because $e^{te^z}$ grows faster than $e^{z^2/2}$ decays.
(D) True Theorem 1.25(3). Continuity is essential: a fair coin gives $F(X)\in\{\tfrac12,1\}$.
Answer (B) and (D).
🧑‍🏫 For the teacher

Suggested plan: 3 lectures × 60 min.

  1. Distribution functions. Definition 1.16 with the "inverse images form a σ-field" argument. Theorem 1.23 with full proof (each property is one use of continuity of $P$). The interval rule cards. Types of distributions, Theorem 1.24, Example 1 drawn on the board. Example 2 as a class poll: students vote on each function before you reveal the answer. Project Interactive 1 in the "mixed" and "binomial" modes.
  2. Quantiles and transformations. Definition 1.19 and Theorem 1.25 (prove (1) carefully: right-continuity is exactly what makes the infimum attained). Simulation via $Q(U)$. Theorem 1.26 and the strategy box. Example 3, then Interactive 2 in the $X^2$ mode with the wrong formula visible. Example 4 on the board.
  3. Expectation and generating functions. Definition 1.20, the tail formula with the Tonelli proof (draw the region $\{(\omega,x):0\le x\lt X(\omega)\}$). Theorems 1.28 and 1.29 (prove the median property through the tail formula). Examples 5 and 6; Interactive 2 in the Cauchy mode to show a running mean that never settles. Theorems 1.31 and 1.32 with Example 7. Finish with E4 and E6 live.

Discussion questions. (1) Why does the definition of $E[X]$ insist on absolute convergence? Give a series whose sum changes when the outcomes are listed in a different order. (2) Two random variables on different probability spaces have the same cdf. In what sense are they "the same", and which questions can still tell them apart? (3) Is it possible for $E[X]$ to exist while $E[X^2]$ does not? For $E[X^2]$ to exist while $E[X]$ does not?

Misconceptions to watch for. Reading $f(x)$ as a probability; using $F(b)-F(a)$ for a closed interval when $F$ jumps at $a$; forgetting $|h'|$ or a second branch in change of variables; assuming every symmetric law has mean equal to its centre; assuming finite values imply a finite mean; confusing the pgf derivative at $1$ (moments) with the derivative at $0$ (probabilities); writing $\operatorname{Var}(aX+b)=a\operatorname{Var}(X)$.

Suggested homework. I(1, 2, 4, 7, 8), II(1, 3, 5, 7), III(2, 3, 5, 7), IV(2, 3, 4, 8); Exam corner E1, E4, E5.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. When the largest losses dominate, as with insurance claims after floods and cyclones or with crashes in financial returns, does the average loss even exist, and how would we know from data?

Why it matters. Reinsurers, catastrophe-bond markets and national disaster funds price risk from historical losses. Losses from natural catastrophes, city sizes and file sizes on the internet are often modelled by Pareto tails $P(X\gt x)\approx Cx^{-\alpha}$. Example 6 and Exercise III.3 show that the mean exists only if $\alpha\gt1$ and the variance only if $\alpha\gt2$. If the true $\alpha$ is near $1$, a sample mean can be dominated by a single event, and a premium set from "the average" is meaningless.

What to do.

  1. Using the quantile transform $X=U^{-1/\alpha}$ (Exercise II.6) and a seeded random number generator, simulate $10^5$ Pareto losses for $\alpha=0.8$, $1.5$ and $3$.
  2. Plot the running mean $\bar X_n$ against $n$ for each $\alpha$, together with the true mean $\frac{\alpha}{\alpha-1}$ where it exists. Describe what you see in each case. Repeat with three different seeds.
  3. Plot $\log\hat P(X\gt x)$ against $\log x$ from the empirical survival function. Explain why the plot is close to a straight line of slope $-\alpha$, and estimate $\alpha$ by least squares on the largest $10\%$ of the data.
  4. Find a published data set of insured catastrophe losses (or use the daily returns of a stock index) and repeat step 3. Is $\alpha\gt2$ plausible?

Tools: Python or a spreadsheet; Interactive 2 in its Cauchy mode as a reference picture of a law with no mean.

Hand in: A two-page note with the three running-mean plots, your tail-index estimates, and a recommendation on whether a sample mean is a safe basis for pricing in your real data set.

Going further. Read about the Hill estimator of the tail index and about the "mean excess function" $E[X-u\mid X\gt u]$, which is linear in $u$ for a Pareto tail. Both are standard tools of extreme-value theory.

Instructor note: the true means are $\infty$, $3$ and $1.5$ for $\alpha=0.8,1.5,3$. With $\alpha=0.8$ the running mean drifts upward in jumps and never settles; with $\alpha=1.5$ it converges, but slowly and with occasional large jumps, because the variance is infinite; with $\alpha=3$ it settles within a few percent of $1.5$ by $n=10^4$. The log–log survival plot of exact Pareto data has slope exactly $-\alpha$, so students' least-squares slopes should be within about $0.1$ of the true value with $10^4$ points in the fitted tail.

🧪 Self-check quiz

0/5 correct

Q1. For any random variable with cdf $F$, $P(X=a)$ equals

Q2. $X$ takes values in $\{0,1,2,\dots\}$ and $P(X\ge n)=2^{-n}$ for $n\ge1$. Then $E[X]$ is

Q3. $X$ is uniform on $(0,1)$. The density of $Y=X^2$ on $(0,1)$ is

Q4. $X$ has pgf $G(s)=\tfrac14(1+s)^2$. Then $\operatorname{Var}(X)$ is

Q5. Which of these distributions has no mean?