Chapter 1 · Probability

1.1 Descriptive Statistics and Exploratory Data Analysis

Before any model comes a careful look at the data: frequency tables and graphs, measures of centre and spread with their extremal properties, moments, skewness and kurtosis, and the correlation coefficient with its least-squares lines.

A statistician handed a spreadsheet of a thousand numbers is first of all a describer. What do the data look like? Where is their centre, how widely are they spread, are they symmetric or lopsided, are there gross errors or genuinely extreme values, and do two measured variables move together? John Tukey called this stage exploratory data analysis and insisted that it comes before any formal inference: a test or a confidence interval computed on data nobody has looked at can be very misleading. A single mis-typed value can double a standard deviation. A strongly skewed sample makes the mean a poor summary. A curved relation can produce a correlation of zero.

This section develops the standard summaries (graphs, measures of location and dispersion, moments, skewness, kurtosis, correlation and the least-squares line) and, just as importantly, the theorems behind them. The mean minimises squared deviations and the median minimises absolute deviations. The correlation coefficient lies in $[-1,1]$ because it is the cosine of an angle. Kurtosis is bounded below by $1+$ (skewness)$^2$. CSIR NET examines exactly these facts: Part B asks you to compute a combined variance or to find $r$ from two regression lines, and Part C asks which statements about such summaries are always true. The same summaries reappear throughout the course as statistics computed from random samples. In Chapter 4 we ask how well they estimate their population counterparts.

Data, Frequency Tables and Graphical Summaries

📘 Definition 1.1 — Data and variables

A population is the collection of all units of interest, and a sample is the part of it that is observed. A variable is a characteristic recorded for each unit, and the recorded values $x_1,\dots,x_n$ are the data. Variables are classified by the operations that are meaningful on their values:

NominalCategories without order: blood group, state of residence, brand. Only counting and the mode make sense.
OrdinalOrdered categories: pain score $0$–$10$, grade A/B/C. Medians and quantiles make sense, but differences need not.
IntervalNumerical, with meaningful differences but an arbitrary zero: temperature in °C, calendar year. Means and SDs make sense, ratios do not.
RatioNumerical with a true zero: height, income, waiting time. Ratios and the coefficient of variation make sense.

Numerical variables are discrete (counts: number of children, defects) or continuous (measurements, recorded to some precision). Nominal and ordinal variables are called qualitative or categorical; interval and ratio variables are quantitative.

📘 Definition 1.2 — Frequency distributions and histograms

Divide the range of the data into disjoint classes $[a_0,a_1),[a_1,a_2),\dots,[a_{k-1},a_k)$ with widths $h_j=a_j-a_{j-1}$ and mid-points $m_j=\tfrac12(a_{j-1}+a_j)$. The frequency $f_j$ is the number of observations in class $j$; $f_j/n$ is the relative frequency; $F_j=f_1+\cdots+f_j$ is the cumulative frequency. The histogram draws over class $j$ a rectangle of height

$$d_j=\frac{f_j}{n\,h_j}\qquad\text{(the frequency density)},$$

so that the area of the rectangle equals the relative frequency, and the total area is $1$. When all classes have the same width, heights proportional to $f_j$ give the same picture. A stem-and-leaf display splits each value into a stem (leading digits) and a leaf (last digit). It shows the shape like a histogram while keeping every observation. The graph of $F_j/n$ against the class end points $a_j$ is the ogive (cumulative frequency curve).

The rule "area = relative frequency" is not decoration. If the last class of a frequency table is three times as wide as the others and we draw heights equal to frequencies, that class looks three times more important than it is. The next example shows this.

✏️ Example 1 — A frequency table with an unequal class
The waiting times (minutes) of $n=20$ patients at a clinic were $$12,15,18,21,22,24,25,27,28,29,$$ $$31,33,34,36,38,41,45,52,58,71.$$ (a) Draw a stem-and-leaf display. (b) Form the frequency table with classes $[10,20),[20,30),[30,40),[40,50),[50,80)$ and compute the histogram heights. (c) Explain what goes wrong if the bar heights are taken to be the frequencies.
(a) Stem-and-leaf Stems are tens, leaves are units: 1 | 2 5 8, 2 | 1 2 4 5 7 8 9, 3 | 1 3 4 6 8, 4 | 1 5, 5 | 2 8, 6 |, 7 | 1. The display is longer to the right: the data are right-skewed, and $71$ sits apart from the rest.
(b) Table The frequencies are $f=3,7,5,2,3$ (total $20$), the widths $h=10,10,10,10,30$, and the densities $d_j=f_j/(20h_j)$ are $$0.015,\ 0.035,\ 0.025,\ 0.010,\ 0.005\ \text{per minute}.$$ Check: $\sum d_jh_j=0.15+0.35+0.25+0.10+0.15=1$. The cumulative frequencies are $3,10,15,17,20$.
(c) The trap With heights $=f_j$ the last bar would be as tall as the first ($3$) and three times as wide, so its area would suggest that $9/26$ of the data lie in $[50,80)$ instead of $3/20$. The density scale shows correctly that the right tail is thin (height $0.005$, the lowest of all).

Measures of Location and Quantiles

📘 Definition 1.3 — Measures of location

Let $x_1,\dots,x_n$ be numerical data and $x_{(1)}\le x_{(2)}\le\cdots\le x_{(n)}$ the same values in increasing order (the order statistics).

  • The (arithmetic) mean is $\bar x=\frac1n\sum_{i=1}^nx_i$; for weights $w_i\ge0$ with $\sum w_i\gt0$ the weighted mean is $\sum w_ix_i/\sum w_i$. For grouped data, $\bar x\approx\frac1n\sum_jf_jm_j$.
  • The median is the middle order statistic $x_{((n+1)/2)}$ if $n$ is odd, and $\tfrac12\left(x_{(n/2)}+x_{(n/2+1)}\right)$ if $n$ is even. Any number $a$ with at least half the data $\le a$ and at least half $\ge a$ is a median; the convention picks the mid-point of the interval of such numbers.
  • A mode is a most frequent value (for grouped data, a point of highest density).
  • The $\alpha$-trimmed mean ($0\le\alpha\lt\tfrac12$, with $k=\lfloor n\alpha\rfloor$) is the mean of $x_{(k+1)},\dots,x_{(n-k)}$, i.e. the mean after the $k$ smallest and $k$ largest values are discarded.
  • For positive data the geometric mean is $G=\left(\prod x_i\right)^{1/n}$ and the harmonic mean is $H=n\big/\sum(1/x_i)$.
📐 Theorem 1.1 — Properties of the mean

For any data $x_1,\dots,x_n$ and constants $a,b$:

  1. $\sum_{i=1}^n(x_i-\bar x)=0$;
  2. if $y_i=a+bx_i$ then $\bar y=a+b\bar x$;
  3. for every real $c$, $\displaystyle\sum_{i=1}^n(x_i-c)^2=\sum_{i=1}^n(x_i-\bar x)^2+n(\bar x-c)^2$. Hence $c\mapsto\sum(x_i-c)^2$ has the unique minimiser $c=\bar x$.

Proof. (1) $\sum(x_i-\bar x)=\sum x_i-n\bar x=0$. (2) $\frac1n\sum(a+bx_i)=a+b\cdot\frac1n\sum x_i$. (3) Write $x_i-c=(x_i-\bar x)+(\bar x-c)$ and expand: $$\sum(x_i-c)^2=\sum(x_i-\bar x)^2+2(\bar x-c)\sum(x_i-\bar x)+n(\bar x-c)^2,$$ and the middle term vanishes by (1). The last term is $\ge0$, with equality only for $c=\bar x$. ∎

📐 Theorem 1.2 — The median minimises absolute deviations
The function $g(c)=\sum_{i=1}^n|x_i-c|$ is convex and piecewise linear. It is minimised exactly at the medians: at $c=x_{((n+1)/2)}$ alone if $n$ is odd, and at every $c\in\left[x_{(n/2)},x_{(n/2+1)}\right]$ if $n$ is even. The minimum value is $\sum_{i\gt n/2}x_{(i)}-\sum_{i\le n/2}x_{(i)}$ when $n$ is even, and the same with the middle term omitted when $n$ is odd.

Proof. Each $c\mapsto|x_i-c|$ is convex and has slope $-1$ to the left of $x_i$ and $+1$ to the right, so $g$ is convex and piecewise linear. At a point $c$ that is not a data value, its slope is $$g'(c)=\#\{i:x_i\lt c\}-\#\{i:x_i\gt c\}.$$ This slope is negative while fewer than half the observations lie below $c$ and positive once more than half do. If $n$ is odd, $g'\lt0$ to the left of $x_{((n+1)/2)}$ and $g'\gt0$ to its right, so that point is the unique minimiser. If $n$ is even, $g'=0$ on the open interval $\left(x_{(n/2)},x_{(n/2+1)}\right)$ (exactly $n/2$ points on each side), $g'\lt0$ before it and $g'\gt0$ after it; so the minimisers form the closed interval. For $c$ in this interval, $|x_{(i)}-c|=c-x_{(i)}$ for $i\le n/2$ and $x_{(i)}-c$ for $i\gt n/2$; the $c$'s cancel and give the stated value. ∎

📐 Theorem 1.3 — AM ≥ GM ≥ HM
For positive data $x_1,\dots,x_n$, $\ \bar x\ge G\ge H$. Equality holds in either place if and only if all the $x_i$ are equal. For two values, $G^2=\bar x\,H$.

Proof. The function $\log$ is concave, so $\log t\le t-1$ for $t\gt0$, with equality only at $t=1$ (the tangent line at $t=1$). Put $t_i=x_i/G$. Then $\sum\log t_i=\sum\log x_i-n\log G=0$, so $$0=\sum\log t_i\le\sum(t_i-1)=\frac{n\bar x}{G}-n,$$ i.e. $\bar x\ge G$, with equality iff every $t_i=1$. Applying this to the positive numbers $1/x_i$, whose AM is $1/H$ and whose GM is $1/G$, gives $1/H\ge1/G$, i.e. $G\ge H$. For two values, $\bar x\,H=\tfrac{a+b}{2}\cdot\tfrac{2ab}{a+b}=ab=G^2$. ∎

📘 Definition 1.4 — Quartiles, five-number summary, box plot and outliers

The first quartile $Q_1$ is the median of the observations below the median position, and the third quartile $Q_3$ is the median of those above it (when $n$ is odd, the median itself is left out of both halves). The five-number summary is $\left(x_{(1)},Q_1,\text{median},Q_3,x_{(n)}\right)$, and the interquartile range is $\mathrm{IQR}=Q_3-Q_1$. A box plot draws a box from $Q_1$ to $Q_3$ with a line at the median, and whiskers to the most extreme observations inside the fences $$Q_1-1.5\,\mathrm{IQR},\qquad Q_3+1.5\,\mathrm{IQR}.$$ Observations outside the fences are flagged as outliers and plotted individually.

More generally, a $p$-th quantile ($0\lt p\lt1$) is a number with at least a fraction $p$ of the data at or below it and at least $1-p$ at or above it. Several interpolation conventions are in use (statistical software usually takes $x_{(j)}+\theta\left(x_{(j+1)}-x_{(j)}\right)$ with $j+\theta=1+(n-1)p$), and they can give slightly different quartiles for small $n$. Exam questions are designed so that the convention does not matter, or they state it.

🧭 Grouped data: the interpolation formulas

When only a frequency table is available, assume the observations in each class are spread uniformly across it. Let the class containing the required point have lower boundary $L$, width $h$ and frequency $f$, let $F$ be the cumulative frequency before it, and let $N=\sum f_j$.

Median$\mathrm{Me}=L+\dfrac{N/2-F}{f}\,h$ (in the class where the cumulative frequency first reaches $N/2$).
Quartiles$Q_1=L+\dfrac{N/4-F}{f}\,h$, $\ Q_3=L+\dfrac{3N/4-F}{f}\,h$ (in their own classes).
Mode$\mathrm{Mo}=L+\dfrac{f_1-f_0}{2f_1-f_0-f_2}\,h$ in the modal class (frequency $f_1$, neighbours $f_0$ before and $f_2$ after; equal widths).
Empirical ruleFor moderately skewed unimodal data, mean $-$ mode $\approx3$(mean $-$ median). This is an approximation, never a theorem.
✏️ Example 2 — Mean, median, mode and quartiles of grouped data
The marks of $50$ students are grouped as follows.
marks0–1010–2020–3030–4040–50
students5122094
Find the mean, the median, the mode, the quartiles, and the variance computed from mid-points. Compute Bowley's quartile coefficient of skewness $\dfrac{Q_3+Q_1-2\,\mathrm{Me}}{Q_3-Q_1}$.
Mean Mid-points $5,15,25,35,45$: $\sum f_jm_j=25+180+500+315+180=1200$, so $\bar x=1200/50=24$.
Median Cumulative frequencies $5,17,37,46,50$. $N/2=25$ falls in $20$–$30$ ($F=17$, $f=20$): $\mathrm{Me}=20+\frac{25-17}{20}\cdot10=24$.
Mode The modal class is $20$–$30$ with $f_0=12$, $f_1=20$, $f_2=9$: $\mathrm{Mo}=20+\frac{20-12}{40-12-9}\cdot10=20+\frac{80}{19}\approx24.21$.
Quartiles $N/4=12.5$ lies in $10$–$20$ ($F=5$, $f=12$): $Q_1=10+\frac{7.5}{12}\cdot10=16.25$. $3N/4=37.5$ lies in $30$–$40$ ($F=37$, $f=9$): $Q_3=30+\frac{0.5}{9}\cdot10\approx30.56$.
Variance $\sum f_jm_j^2=125+2700+12500+11025+8100=34450$, so $s^2=34450/50-24^2=689-576=113$ and $s\approx10.63$. (Sheppard's correction, Theorem 1.9, would give $113-\tfrac{100}{12}\approx104.67$.)
Skewness Bowley: $\dfrac{30.556+16.25-48}{30.556-16.25}\approx\dfrac{-1.194}{14.306}\approx-0.083$. Pearson's $(\bar x-\mathrm{Mo})/s\approx-0.21/10.63\approx-0.020$. Both are close to $0$: the distribution is nearly symmetric, with a very slightly longer lower tail.
Interactive 1 — One Bad Value: Robust and Non-robust Summaries

The top panel is the density-scale histogram, the middle row shows every observation, and the bottom panel is the box plot with its fences (outliers are hollow circles). The vertical lines mark the mean, the median and the 10% trimmed mean. Drag the slider (or press Play) to move the largest observation further right, as a data-entry error would, and watch which readouts react.

0
mean—
median—
10% trimmed—
SD (divisor n)—
IQR—
MAD—
skewness γ₁—
kurtosis β₂—
✏️ Example 3 — Box plot, outliers and robustness
For the waiting times of Example 1: (a) find the five-number summary, the fences and any outliers; (b) compute the mean, the $10\%$ trimmed mean, the SD (divisor $n$), the mean deviation about the median and the median absolute deviation; (c) suppose $71$ had been mistyped as $171$. Which of these summaries change?
(a) $n=20$, so the median is $\tfrac12(x_{(10)}+x_{(11)})=\tfrac12(29+31)=30$. Lower half $12,\dots,29$: $Q_1=\tfrac12(22+24)=23$. Upper half $31,\dots,71$: $Q_3=\tfrac12(38+41)=39.5$. Five-number summary $(12,23,30,39.5,71)$, $\mathrm{IQR}=16.5$, fences $23-24.75=-1.75$ and $39.5+24.75=64.25$. Only $71$ lies outside: it is an outlier, and the upper whisker ends at $58$.
(b) $\sum x_i=660$, so $\bar x=33$. The $10\%$ trimmed mean drops $12,15,58,71$: $(660-156)/16=31.5$. $\sum(x_i-33)^2=4154$, so $s^2=207.7$ and $s\approx14.41$. The absolute deviations from the median $30$ sum to $218$, so the mean deviation about the median is $10.9$. Their median, the MAD, is $8$.
(c) With $171$ in place of $71$: $\bar x=760/20=38$ (it moves by $100/20=5$), and $s\approx32.60$ (more than doubled). The median ($30$), $Q_1$, $Q_3$, IQR, MAD and the trimmed mean are unchanged, because the largest value is still the largest and none of them uses its size. The mean deviation about the median rises by exactly $100/20=5$, to $15.9$.
Lesson The mean and SD have breakdown point $1/n$: one wild value can carry them anywhere. The median, quartiles and MAD tolerate up to (nearly) half the data being corrupted. The $\alpha$-trimmed mean lies between the two, tolerating a fraction $\alpha$. Try it in Interactive 1.

Dispersion, Moments, Skewness and Kurtosis

📘 Definition 1.5 — Measures of dispersion
  • Range $R=x_{(n)}-x_{(1)}$; interquartile range $\mathrm{IQR}=Q_3-Q_1$.
  • Variance $s^2=\frac1n\sum(x_i-\bar x)^2$ and standard deviation $s=\sqrt{s^2}$ (divisor $n$, used throughout this section). The sample variance $s_{n-1}^2=\frac{1}{n-1}\sum(x_i-\bar x)^2=\frac{n}{n-1}s^2$ is the version that estimates a population variance without bias (Chapter 4).
  • Mean deviation about $A$: $\mathrm{MD}(A)=\frac1n\sum|x_i-A|$, usually taken about the mean or the median.
  • Median absolute deviation: $\mathrm{MAD}=\operatorname{median}_i|x_i-\mathrm{Me}|$, a robust measure of spread.
  • Coefficient of variation (ratio data with $\bar x\gt0$): $\mathrm{CV}=s/\bar x$, often quoted as a percentage. It is unit-free, so it compares variability across different scales.
📐 Theorem 1.4 — Variance identities
  1. $s^2=\overline{x^2}-\bar x^2$, where $\overline{x^2}=\frac1n\sum x_i^2$. More generally, for any $A$, $s^2=\frac1n\sum(x_i-A)^2-(\bar x-A)^2$.
  2. If $y_i=a+bx_i$ then $s_y^2=b^2s_x^2$ and $s_y=|b|\,s_x$: variance ignores a change of origin and scales by the square of a change of scale.
  3. $s^2=\dfrac{1}{2n^2}\displaystyle\sum_{i=1}^n\sum_{j=1}^n(x_i-x_j)^2$: the variance is half the mean squared difference between two observations.

Proof. (1) Theorem 1.1(3) with $c=A$, divided by $n$; for $A=0$ it reads $\overline{x^2}=s^2+\bar x^2$. (2) $y_i-\bar y=b(x_i-\bar x)$ by Theorem 1.1(2); square and average. (3) Expand $(x_i-x_j)^2=x_i^2-2x_ix_j+x_j^2$ and sum over both indices: $$\sum_i\sum_j(x_i-x_j)^2=2n\sum x_i^2-2\Big(\sum x_i\Big)^2=2n^2\left(\overline{x^2}-\bar x^2\right)=2n^2s^2.\ ∎$$

📐 Theorem 1.5 — Inequalities between measures of spread

For any data (with $s$ computed with divisor $n$):

  1. $\mathrm{MD}(\mathrm{Me})\le\mathrm{MD}(\bar x)\le s$;
  2. $|\bar x-\mathrm{Me}|\le s$: the median lies within one standard deviation of the mean;
  3. if all $x_i\in[a,b]$ then $s\le\tfrac12(b-a)$; in particular $s\le R/2$;
  4. (Samuelson's inequality) every observation satisfies $|x_k-\bar x|\le s\sqrt{n-1}$.

Proof. (1) The first inequality is Theorem 1.2: the median minimises $\sum|x_i-c|$. For the second, apply the Cauchy–Schwarz inequality to the vectors $(|x_i-\bar x|)_i$ and $(1,\dots,1)$: $\left(\sum|x_i-\bar x|\right)^2\le n\sum(x_i-\bar x)^2$; divide by $n^2$ and take square roots.

(2) By the triangle inequality and (1), $$|\bar x-\mathrm{Me}|=\Big|\tfrac1n\sum(x_i-\mathrm{Me})\Big|\le\tfrac1n\sum|x_i-\mathrm{Me}|=\mathrm{MD}(\mathrm{Me})\le s.$$

(3) By Theorem 1.1(3), $s^2$ is the minimum of $\frac1n\sum(x_i-c)^2$ over $c$. Taking $c=\tfrac{a+b}{2}$, every term is at most $\left(\tfrac{b-a}{2}\right)^2$, because each $x_i$ is within $\tfrac{b-a}{2}$ of the centre of $[a,b]$. Equality needs every $x_i\in\{a,b\}$ and $\bar x=\tfrac{a+b}2$: half the data at each end.

(4) Let $d_i=x_i-\bar x$. Then $\sum_{i\ne k}d_i=-d_k$ by Theorem 1.1(1), and by Cauchy–Schwarz $$d_k^2=\Big(\sum_{i\ne k}d_i\Big)^2\le(n-1)\sum_{i\ne k}d_i^2=(n-1)\left(ns^2-d_k^2\right).$$ Rearranging, $nd_k^2\le(n-1)ns^2$, i.e. $|d_k|\le s\sqrt{n-1}$. ∎

Samuelson's bound is attained: for the data $(1,\ 0,\ \dots,\ 0)$ with $n-1$ zeros, $\bar x=\tfrac{1}{n}$, $s^2=\tfrac{n-1}{n^2}$, and $x_1-\bar x=\tfrac{n-1}{n}=s\sqrt{n-1}$. So an observation can lie as many as $\sqrt{n-1}$ standard deviations from the mean, but no further.

📐 Theorem 1.6 — Combining groups: within plus between
Suppose $k$ groups have sizes $n_j$, means $\bar x_j$ and variances $s_j^2$ (divisor $n_j$), and let $N=\sum n_j$. The combined data have mean and variance $$\bar x=\frac1N\sum_jn_j\bar x_j,\qquad s^2=\underbrace{\frac1N\sum_jn_js_j^2}_{\text{within groups}}+\underbrace{\frac1N\sum_jn_j(\bar x_j-\bar x)^2}_{\text{between groups}}.$$

Proof. The total of all observations is $\sum_jn_j\bar x_j$, which gives the mean. For group $j$, Theorem 1.1(3) with $c=\bar x$ gives $\sum_{i\in j}(x_i-\bar x)^2=n_js_j^2+n_j(\bar x_j-\bar x)^2$. Sum over $j$ and divide by $N$. ∎

This decomposition is the germ of the analysis of variance (Chapter 6), where a large between-groups part relative to the within-groups part is evidence that the group means differ.

✏️ Example 4 — Pooling two sections of a class
Section A has $30$ students with mean mark $50$ and SD $6$. Section B has $20$ students with mean $60$ and SD $8$ (both SDs with divisor $n$). Find the mean and SD of all $50$ marks. Is the pooled SD between $6$ and $8$?
Mean $\bar x=\dfrac{30\cdot50+20\cdot60}{50}=\dfrac{2700}{50}=54$.
Within $\dfrac{30\cdot36+20\cdot64}{50}=\dfrac{1080+1280}{50}=47.2$.
Between $\dfrac{30(50-54)^2+20(60-54)^2}{50}=\dfrac{480+720}{50}=24$.
Answer $s^2=47.2+24=71.2$, so $s\approx8.44$. That is larger than both group SDs: the gap between the section means adds spread. A pooled SD between the two group SDs would require nearly equal group means.
📘 Definition 1.6 — Moments, skewness and kurtosis

The $r$-th moment about $A$ is $m_r'(A)=\frac1n\sum(x_i-A)^r$ (raw moments are those about $A=0$), and the $r$-th central moment is $m_r=\frac1n\sum(x_i-\bar x)^r$. Thus $m_1=0$ and $m_2=s^2$. The moment measures of shape are

$$\beta_1=\frac{m_3^2}{m_2^3},\quad\gamma_1=\frac{m_3}{m_2^{3/2}},\quad\beta_2=\frac{m_4}{m_2^2},\quad\gamma_2=\beta_2-3.$$

$\gamma_1$ is the coefficient of skewness (positive when the right tail is longer), and $\beta_2$ is the kurtosis ($\gamma_2$ is the excess kurtosis; the normal distribution has $\beta_2=3$). Data with $\beta_2\gt3$ are called leptokurtic (heavier tails than the normal), and those with $\beta_2\lt3$ platykurtic. Two further measures of skewness are Pearson's $(\bar x-\mathrm{Mo})/s$ and Bowley's $(Q_3+Q_1-2\mathrm{Me})/(Q_3-Q_1)\in[-1,1]$.

📐 Theorem 1.7 — Central moments from raw moments; invariance

(a) With $d=m_1'(A)=\bar x-A$ and $m_r'=m_r'(A)$,

$$m_r=\sum_{j=0}^{r}\binom rj m_j'(-d)^{r-j}\quad(m_0'=1).$$

In particular $m_2=m_2'-d^2$, $\ m_3=m_3'-3m_2'd+2d^3$ and $\ m_4=m_4'-4m_3'd+6m_2'd^2-3d^4$.

(b) If $y_i=a+bx_i$ with $b\ne0$, then $m_r(y)=b^rm_r(x)$. Hence $\beta_1$ and $\beta_2$ are unchanged, and $\gamma_1$ is multiplied by $\operatorname{sign}b$.

Proof. (a) $x_i-\bar x=(x_i-A)-d$; expand $\left((x_i-A)-d\right)^r$ by the binomial theorem and average over $i$. Since $m_1'=d$, the terms $j=0$ and $j=1$ combine: $m_0'(-d)^r+\binom r1m_1'(-d)^{r-1}$ $=(-d)^r-r(-d)^r$ $=(1-r)(-d)^r$, which gives the coefficients $2d^3$ for $r=3$ and $-3d^4$ for $r=4$. (b) $y_i-\bar y=b(x_i-\bar x)$, so each central moment picks up the factor $b^r$. In $\beta_1=m_3^2/m_2^3$ the factor is $b^6/b^6$, in $\beta_2$ it is $b^4/b^4$, and in $\gamma_1$ it is $b^3/|b|^3=\operatorname{sign}b$. ∎

📐 Theorem 1.8 — Pearson's inequality: $\beta_2\ge1+\beta_1$
For any data with $s\gt0$, $\ \beta_2\ge\beta_1+1\ge1$. Equality $\beta_2=\beta_1+1$ holds if and only if the data take at most two distinct values.

Proof. Standardise: $z_i=(x_i-\bar x)/s$. Then $\frac1n\sum z_i=0$, $\frac1n\sum z_i^2=1$, $\frac1n\sum z_i^3=\gamma_1$ and $\frac1n\sum z_i^4=\beta_2$. Consider the nonnegative average $$0\le\frac1n\sum_i\left(z_i^2-\gamma_1z_i-1\right)^2.$$ Expanding the square and using the four moments gives $\beta_2+\gamma_1^2+1-2\gamma_1\cdot\gamma_1-2+2\gamma_1\cdot0=\beta_2-\gamma_1^2-1$. Hence $\beta_2\ge1+\gamma_1^2=1+\beta_1$. Equality holds iff $z_i^2-\gamma_1z_i-1=0$ for every $i$, i.e. every $z_i$ is a root of one fixed quadratic: the data take at most two values. Conversely, if the data take two values, a direct computation (Exercise II.7) gives equality. ∎

So kurtosis can never be below $1$, and it is large when skewness is large. $\beta_2=1$ exactly when the data are two values in equal proportions: the most "anti-peaked" possible configuration.

📐 Theorem 1.9 — Sheppard's corrections (statement)
Suppose continuous data are grouped into classes of equal width $h$ and the moments $\bar\mu_r$ are computed by placing every observation at its class mid-point. If the underlying frequency curve is smooth and tapers off gradually to zero at both ends ("high contact"), then better approximations to the true central moments are $$\mu_2\approx\bar\mu_2-\frac{h^2}{12},\qquad\mu_3\approx\bar\mu_3,\qquad\mu_4\approx\bar\mu_4-\frac{h^2}{2}\bar\mu_2+\frac{7h^4}{240}.$$ The corrections should not be applied to discrete data, to J- or U-shaped distributions, or when the classes are wide relative to the spread.

Proof (sketch). Under the stated conditions the grouping error behaves like a variable $U$, uniform on $\left(-\tfrac h2,\tfrac h2\right)$ and independent of the true value $X$, with the grouped value distributed like $X+U$. Since $E[U]=E[U^3]=0$, $E[U^2]=h^2/12$ and $E[U^4]=h^4/80$, expanding the central moments of $X+U$ gives $\bar\mu_2=\mu_2+\tfrac{h^2}{12}$, $\bar\mu_3=\mu_3$ and $\bar\mu_4=\mu_4+6\mu_2\tfrac{h^2}{12}+\tfrac{h^4}{80}$. Solving for $\mu_4$ and substituting $\mu_2=\bar\mu_2-\tfrac{h^2}{12}$ gives $\mu_4=\bar\mu_4-\tfrac{h^2}{2}\bar\mu_2+\left(\tfrac1{24}-\tfrac1{80}\right)h^4$, and $\tfrac1{24}-\tfrac1{80}=\tfrac{7}{240}$. (Making "behaves like" precise uses the Euler–Maclaurin formula; we omit it.) ∎

✏️ Example 5 — Moments, skewness and kurtosis by hand
For the data $1,2,3,3,4,5,10$: (a) compute the moments about $A=3$ and convert them to central moments with Theorem 1.7; (b) find $\gamma_1$ and $\beta_2$ and check Pearson's inequality.
Moments about 3 The deviations $x_i-3$ are $-2,-1,0,0,1,2,7$. Their powers average to $m_1'=\tfrac77=1$, $m_2'=\tfrac{4+1+0+0+1+4+49}{7}=\tfrac{59}{7}$, $m_3'=\tfrac{-8-1+1+8+343}{7}=49$ and $m_4'=\tfrac{16+1+1+16+2401}{7}=\tfrac{2435}{7}$. So $\bar x=3+1=4$.
Central moments With $d=1$: $m_2=\tfrac{59}7-1=\tfrac{52}{7}$; $m_3=49-3\cdot\tfrac{59}{7}+2=\tfrac{180}{7}$; $m_4=\tfrac{2435}{7}-4\cdot49+6\cdot\tfrac{59}{7}-3=\tfrac{1396}{7}$. Check directly: the deviations from $4$ are $-3,-2,-1,-1,0,1,6$, and their squares sum to $52$, their cubes to $180$ and their fourth powers to $1396$.
Shape $\gamma_1=\dfrac{180/7}{(52/7)^{3/2}}\approx1.270$ (right-skewed: the $10$ pulls the tail), $\beta_1\approx1.613$, and $\beta_2=\dfrac{1396/7}{(52/7)^2}=\dfrac{1396\cdot7}{2704}\approx3.614$.
Check $\beta_2\approx3.614\ge1+\beta_1\approx2.613$, as Theorem 1.8 demands. Mean $4$ $\gt$ median $3$, consistent with (but not implied by!) the positive skewness. See Example 6(b).

Correlation and the Least-Squares Line

Now each unit carries two measurements, giving paired data $(x_1,y_1),\dots,(x_n,y_n)$: height and weight, dose and response, temperature and electricity demand. A scatter plot shows the relation, and two numbers summarise its linear part.

📘 Definition 1.7 — Covariance and correlation coefficient
The covariance of paired data is $s_{xy}=\frac1n\sum(x_i-\bar x)(y_i-\bar y)=\overline{xy}-\bar x\,\bar y$. If $s_x,s_y\gt0$, Karl Pearson's correlation coefficient is $$r=r_{xy}=\frac{s_{xy}}{s_xs_y}=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum(x_i-\bar x)^2\,\sum(y_i-\bar y)^2}}.$$
📐 Theorem 1.10 — $-1\le r\le1$, and $r$ is a cosine
Let $u=(x_i-\bar x)_i$ and $v=(y_i-\bar y)_i$ be the centred data vectors in $\mathbb{R}^n$. Then $r=\dfrac{u\cdot v}{\|u\|\,\|v\|}=\cos\theta$, where $\theta\in[0,\pi]$ is the angle between $u$ and $v$. Consequently $-1\le r\le1$. Moreover $r=1$ (respectively $r=-1$) if and only if all the points $(x_i,y_i)$ lie on one line with positive (respectively negative) slope.

Proof. The formula for $r$ is $u\cdot v/(\|u\|\|v\|)$ (the factors $\tfrac1n$ cancel). By the Cauchy–Schwarz inequality, $|u\cdot v|\le\|u\|\|v\|$, so $|r|\le1$, with equality iff $v=cu$ for a scalar $c\ne0$ (neither vector is $0$ because $s_x,s_y\gt0$). Here is a self-contained argument. For real $t$, $$0\le\|v-tu\|^2=\|v\|^2-2t\,u\cdot v+t^2\|u\|^2.$$ A quadratic in $t$ that is never negative has discriminant $4(u\cdot v)^2-4\|u\|^2\|v\|^2\le0$, which is $r^2\le1$. Equality means the quadratic has a real root $t=c$, i.e. $v=cu$: $y_i-\bar y=c(x_i-\bar x)$ for all $i$. All points then lie on a line of slope $c$, and $r=u\cdot(cu)/(\|u\|\cdot|c|\|u\|)=\operatorname{sign}c$. Conversely, if all points lie on a line $y=\alpha+cx$ with $c\ne0$, then $v=cu$. ∎

📐 Theorem 1.11 — Change of origin and scale
If $u_i=a+bx_i$ and $v_i=c+dy_i$ with $bd\ne0$, then $r_{uv}=\operatorname{sign}(bd)\,r_{xy}$. Correlation is unchanged by changes of origin and by changes of scale of the same sign, and it changes sign if exactly one scale factor is negative.

Proof. $u_i-\bar u=b(x_i-\bar x)$ and $v_i-\bar v=d(y_i-\bar y)$, so $s_{uv}=bd\,s_{xy}$, $s_u=|b|s_x$ and $s_v=|d|s_y$. Hence $r_{uv}=\dfrac{bd}{|b||d|}r_{xy}$. ∎

📘 Definition 1.8 — Least-squares lines and regression coefficients
The least-squares line of $y$ on $x$ is the line $y=\alpha+\beta x$ that minimises the sum of squared vertical deviations $Q(\alpha,\beta)=\sum(y_i-\alpha-\beta x_i)^2$. Its slope is the regression coefficient $b_{yx}$. The line of $x$ on $y$ minimises the horizontal deviations $\sum(x_i-\alpha'-\beta'y_i)^2$, and its slope is $b_{xy}$. The differences $e_i=y_i-\hat y_i$ between the observed and the fitted values are the residuals.
📐 Theorem 1.12 — The least-squares lines

Assume $s_x,s_y\gt0$.

  1. The line of $y$ on $x$ is unique: $\ y-\bar y=b_{yx}(x-\bar x)$ with $b_{yx}=\dfrac{s_{xy}}{s_x^2}=r\dfrac{s_y}{s_x}$. Similarly $x-\bar x=b_{xy}(y-\bar y)$ with $b_{xy}=\dfrac{s_{xy}}{s_y^2}=r\dfrac{s_x}{s_y}$. Both lines pass through $(\bar x,\bar y)$.
  2. $b_{yx}b_{xy}=r^2\le1$. The two coefficients and $r$ have the same sign, and $r=\operatorname{sign}(b_{yx})\sqrt{b_{yx}b_{xy}}$.
  3. The residuals satisfy $\sum e_i=0$ and $\sum x_ie_i=0$, and the residual mean square is $\frac1n\sum e_i^2=s_y^2(1-r^2)$.
  4. The two lines coincide iff $|r|=1$. In the $(x,y)$ plane the acute angle $\theta$ between them satisfies $\tan\theta=\dfrac{1-r^2}{|r|}\cdot\dfrac{s_xs_y}{s_x^2+s_y^2}$ (for $r\ne0$); for $r=0$ they are the perpendicular lines $y=\bar y$ and $x=\bar x$.

Proof. (1) For fixed $\beta$, $Q$ is a sum of squares of $(y_i-\beta x_i)-\alpha$, so Theorem 1.1(3) shows it is minimised by $\alpha=\bar y-\beta\bar x$; the line then passes through $(\bar x,\bar y)$. Substituting, $$Q=\sum\left((y_i-\bar y)-\beta(x_i-\bar x)\right)^2=n\left(s_y^2-2\beta s_{xy}+\beta^2s_x^2\right),$$ a quadratic in $\beta$ with positive leading coefficient, minimised uniquely at $\beta=s_{xy}/s_x^2$. Interchanging the roles of $x$ and $y$ gives $b_{xy}$.

(2) $b_{yx}b_{xy}=s_{xy}^2/(s_x^2s_y^2)=r^2$, which is at most $1$ by Theorem 1.10. All three quantities have the sign of $s_{xy}$.

(3) The normal equations $\partial Q/\partial\alpha=\partial Q/\partial\beta=0$ read $\sum e_i=0$ and $\sum x_ie_i=0$. The minimum value is $Q=n\left(s_y^2-s_{xy}^2/s_x^2\right)=ns_y^2(1-r^2)$.

(4) The slope of the first line is $m_1=rs_y/s_x$. Solving the second for $y$ gives slope $m_2=1/b_{xy}=s_y/(rs_x)$. Then $m_1=m_2$ iff $r^2=1$, and $$\tan\theta=\left|\frac{m_2-m_1}{1+m_1m_2}\right|=\frac{\frac{s_y}{s_x}\left(\frac1{|r|}-|r|\right)}{1+\frac{s_y^2}{s_x^2}}=\frac{1-r^2}{|r|}\cdot\frac{s_xs_y}{s_x^2+s_y^2}.\ ∎$$

The quantity $r^2$ is the coefficient of determination: by (3), the line of $y$ on $x$ removes the fraction $r^2$ of the variance of $y$, and $1-r^2$ is left in the residuals. In Chapter 6 the same algebra, in matrix form, becomes the theory of linear models.

🧭 Problem-solving strategy — two regression lines are given
  1. Means: both lines pass through $(\bar x,\bar y)$, so solve the two equations simultaneously.
  2. Which line is which? Tentatively take the first as $y$ on $x$ and read off $b_{yx}$ (solve it for $y$), and take the second as $x$ on $y$ and read off $b_{xy}$ (solve it for $x$). If $b_{yx}b_{xy}\le1$, the choice is consistent. If the product exceeds $1$, swap the roles; exactly one choice works unless $|r|=1$ or the coefficients have opposite signs (then the data are impossible).
  3. Correlation: $r=\pm\sqrt{b_{yx}b_{xy}}$, with the common sign of the coefficients.
  4. Standard deviations: $b_{yx}=rs_y/s_x$ gives one SD from the other.
✏️ Example 6 — Prove or disprove
Decide whether each statement holds for all data sets. (a) If all observations lie in $[0,10]$, then $s\le5$. (b) If the mean exceeds the median, then $\gamma_1\gt0$. (c) If $r_{xy}=r_{yz}=0.9$, then $r_{xz}\gt0$. (d) If $r_{xy}=r_{yz}=0.5$, then $r_{xz}\ge0$.
(a) True Theorem 1.5(3) with $[a,b]=[0,10]$. The bound $5$ is attained by data consisting of equally many $0$s and $10$s.
(b) False Take $0,1,1,2,2$: the mean is $1.2$ and the median is $1$, but the deviations $-1.2,-0.2,-0.2,0.8,0.8$ have cubes summing to $-1.728-0.016+1.024=-0.72$, so $m_3=-0.144\lt0$ and $\gamma_1\lt0$. The sign of mean $-$ median and the sign of $\gamma_1$ measure different things and can disagree.
(c) True By Theorem 1.10, $r$ is the cosine of the angle between centred vectors. Let $\theta=\arccos0.9\approx25.84^\circ$. Angles between vectors satisfy the triangle inequality on the sphere, so the angle between the $x$- and $z$-vectors is at most $2\theta\approx51.7^\circ\lt90^\circ$. Hence $r_{xz}\ge\cos2\theta=2(0.9)^2-1=0.62\gt0$.
(d) False Here $2\arccos0.5=120^\circ$, and the bound $\cos120^\circ=-0.5$ is attained. Take the three units $(x,y,z)=(1,1,0),\ (-1,0,1),\ (0,-1,-1)$. All three variables have mean $0$ and $\sum x^2=\sum y^2=\sum z^2=2$, while $\sum xy=1$, $\sum yz=1$ and $\sum xz=-1$. So $r_{xy}=r_{yz}=0.5$ but $r_{xz}=-0.5$. Correlation is not transitive.
✏️ Example 7 — Computing $r$ and the regression lines; recovering them from the lines
(a) For the data $(x,y)=(2,4),(3,5),(5,9),(6,8),(9,14)$, find $r$, both least-squares lines and the residual mean square of $y$ on $x$. (b) The two regression lines of some other data are $4x-5y+14=0$ and $20x-9y-26=0$, and $s_x=3$. Find $\bar x$, $\bar y$, $r$ and $s_y$.
(a) Summaries $\bar x=25/5=5$ and $\bar y=40/5=8$. The deviations are $x-5:\ -3,-2,0,1,4$ and $y-8:\ -4,-3,1,0,6$. So $ns_x^2=9+4+0+1+16=30$, $ns_y^2=16+9+1+0+36=62$ and $ns_{xy}=12+6+0+0+24=42$.
(a) Results $r=\dfrac{42}{\sqrt{30\cdot62}}=\dfrac{42}{\sqrt{1860}}\approx0.974$. $b_{yx}=42/30=1.4$, so $y-8=1.4(x-5)$, i.e. $y=1.4x+1$. $b_{xy}=42/62=21/31\approx0.677$, so $x-5=0.677(y-8)$. Check: $1.4\cdot\tfrac{21}{31}=\tfrac{29.4}{31}\approx0.948=r^2$. Residual mean square: $s_y^2(1-r^2)=12.4\left(1-\tfrac{1764}{1860}\right)=0.64$. Directly, the fitted values $3.8,5.2,8,9.4,13.6$ give residuals $0.2,-0.2,1,-1.4,0.4$ (summing to $0$), whose squares average $3.2/5=0.64$.
(b) Means Solve $4x-5y=-14$ and $20x-9y=26$: five times the first is $20x-25y=-70$; subtracting from the second gives $16y=96$, so $\bar y=6$ and $\bar x=(5\cdot6-14)/4=4$.
(b) Which is which Try $4x-5y+14=0$ as $y$ on $x$: $y=0.8x+2.8$, so $b_{yx}=0.8$. Then $20x-9y-26=0$ as $x$ on $y$: $x=0.45y+1.3$, so $b_{xy}=0.45$. The product is $0.36\le1$, which is consistent. (The other assignment would give $b_{xy}=5/4$ and $b_{yx}=20/9$, product $25/9\gt1$, which is impossible.) Hence $r=+\sqrt{0.36}=0.6$, and $s_y=b_{yx}s_x/r=0.8\cdot3/0.6=4$. Check: $b_{xy}=rs_x/s_y=0.6\cdot3/4=0.45$.
Interactive 2 — Correlation, the Two Regression Lines and Residuals

Thirty points are generated with population correlation $\rho$ (seeded, so the cloud is reproducible). The red line is $y$ on $x$ (it minimises the dashed vertical residuals) and the blue line is $x$ on $y$. Both pass through the green point $(\bar x,\bar y)$. As $|r|\to1$ the "scissors" close; at $r=0$ they open to a right angle. Try the curved cloud ($r\approx0$ despite perfect dependence) and the single outlier.

0.60
sample r—
b_yx (red slope)—
b_xy—
b_yx · b_xy = r²—
residual mean square—
angle between lines—
⚠️ Counterexamples and pitfalls
  1. $r=0$ does not mean "no relation". For $x=-2,-1,0,1,2$ and $y=x^2$, $s_{xy}=\frac15\sum x^3=0$, so $r=0$, although $y$ is a function of $x$. $r$ measures linear association only.
  2. Correlation is not transitive. $r_{xy}=r_{yz}=0.5$ with $r_{xz}=-0.5$ is possible (Example 6(d)). Only when $\arccos r_{xy}+\arccos r_{yz}\lt90^\circ$ is $r_{xz}$ forced to be positive.
  3. Two regression lines, not one. $b_{yx}\ne1/b_{xy}$ unless $|r|=1$. The line of $x$ on $y$ is not obtained by solving the line of $y$ on $x$ for $x$.
  4. Impossible coefficients. $b_{yx}=1.6$, $b_{xy}=0.7$ cannot occur (the product $1.12$ exceeds $1$), and coefficients of opposite signs cannot occur. Both can be "regression coefficients" greater than $1$ in absolute value only in a question that has an error.
  5. Sign flips under negative scaling. $r(x,\,5-2y)=-r(x,y)$; $r(-x,-y)=r(x,y)$. Variance ignores the origin, but the CV does not: it changes when a constant is added.
  6. Mean $\gt$ median does not imply $\gamma_1\gt0$ ($0,1,1,2,2$), and $\gamma_1=0$ does not imply symmetry or mean $=$ median: $0,0,5,5,5,9$ has $m_3=0$, mean $4$, median $5$.
  7. Pooled spread is not the average spread. The combined SD of two groups can exceed both group SDs (Example 4). The combined median is not the average of the two medians.
  8. One outlier breaks the mean, the SD and $r$. Moving one value changes $\bar x$ by (change)$/n$ and can make $s$ arbitrarily large. A single far point can turn $r\approx0$ into $r\approx0.9$. Medians, quartiles, MAD and trimmed means resist this.
  9. Histogram heights must be densities when class widths differ (Example 1).
  10. The CV needs ratio data with a positive mean. The CV of temperatures in °C is meaningless (it changes when you switch to kelvin), and it explodes when $\bar x\approx0$.
  11. Kurtosis is about tails, not "peakedness". $\beta_2\ge1$ always, and $\beta_2=1$ for two equally frequent values. Unimodal data can have $\beta_2\lt3$: for $1,2,2,3$, $\beta_2=2$.
  12. Divisors. $s^2$ (divisor $n$) and $s_{n-1}^2$ (divisor $n-1$) differ: for $1,2,\dots,20$ they are $33.25$ and $35$. Read the question.
  13. Aggregation can reverse a correlation. In groups $\{(1,10),(2,11)\}$ and $\{(10,1),(11,2)\}$ each group has $r=+1$, but the pooled data have $r\approx-0.976$.

Exercises

I
Data, location and quantiles.
  • 1 Classify each variable as nominal, ordinal, interval or ratio, and say whether it is discrete or continuous: (a) blood group; (b) pain score on a $0$–$10$ scale; (c) body temperature in °C; (d) monthly income in rupees; (e) number of children in a family.
  • 2 For the data $7,9,10,10,12,13,15,20$, find the mean, the median, the mode and the $25\%$ trimmed mean.
  • 3 For the data of part 2, find the minimum of $\sum(x_i-c)^2$ over $c$, and the value of that sum at $c=10$.
  • 4 Minimise $\sum|x_i-c|$ for (a) $3,5,8,10,24$; (b) $3,5,8,10$. Give all minimisers and the minimum values.
  • 5 Find the mean, median and mode of the distribution with classes $0$–$20$, $20$–$40$, $40$–$60$, $60$–$80$, $80$–$100$ and frequencies $6,14,22,12,6$.
  • 6 The values $0,1,2,3,4$ occur with frequencies $2,5,f,6,3$, and the mean is $2.1$. Find $f$.
  • 7 For $2,3,6$ compute the AM, GM and HM, and compare $G^2$ with the product of the AM and HM. Prove that $G^2=\bar x\,H$ for any two positive numbers.
  • 8 For $3,7,8,9,10,11,12,13,14,30$, find the five-number summary, the IQR and the fences, and identify any outliers.
1 (a) Nominal (qualitative). (b) Ordinal and discrete: the steps need not be equal. (c) Interval and continuous: $0$°C is not "no temperature", so ratios are meaningless. (d) Ratio; money is recorded in paise, so it is discrete in principle but treated as continuous. (e) Ratio and discrete.
2 $\sum x_i=96$, so the mean is $12$; the median is $\tfrac12(10+12)=11$; the mode is $10$. With $k=\lfloor8\cdot0.25\rfloor=2$, drop $7,9,15,20$: $(10+10+12+13)/4=11.25$.
3 The minimum is at $c=\bar x=12$: the squared deviations $25,9,4,4,0,1,9,64$ sum to $116$. At $c=10$, Theorem 1.1(3) gives $116+8(12-10)^2=148$.
4 (a) The unique minimiser is the median $8$, with minimum $5+3+0+2+16=26$. (b) Every $c\in[5,8]$ is a minimiser, with minimum $(8+10)-(3+5)=10$.
5 Mid-points $10,30,50,70,90$: the mean is $2960/60\approx49.33$. Cumulative frequencies $6,20,42$: the median is $40+\frac{30-20}{22}\cdot20\approx49.09$. Mode: $40+\frac{22-14}{44-14-12}\cdot20=40+\frac{160}{18}\approx48.89$.
6 $\sum fx=0+5+2f+18+12=35+2f$ and $N=16+f$. Solving $35+2f=2.1(16+f)$ gives $0.1f=1.4$, so $f=14$. Check: $63/30=2.1$.
7 AM $=11/3\approx3.667$, GM $=36^{1/3}\approx3.302$, HM $=3/(1/2+1/3+1/6)=3$. AM$\cdot$HM $=11$ but $G^2\approx10.90$: the identity fails for three numbers. For two numbers see the proof of Theorem 1.3.
8 Median $\tfrac12(10+11)=10.5$, $Q_1=8$ (median of $3,7,8,9,10$), $Q_3=13$ (median of $11,\dots,30$). Five-number summary $(3,8,10.5,13,30)$, $\mathrm{IQR}=5$, fences $0.5$ and $20.5$. Only $30$ is an outlier ($3\ge0.5$ is not).
II
Dispersion and moments.
  • 1 For $4,6,8,10,12$, find the variance (divisor $n$), the SD, the CV and the mean deviation about the mean.
  • 2 If $\bar x=5$ and $s_x=4$, find the mean and SD of $y=3-2x$.
  • 3 Prove that the SD (divisor $n$) of $1,2,\dots,n$ is $\sqrt{(n^2-1)/12}$.
  • 4 The mean and SD of $25$ observations were computed as $40$ and $5$. It was then found that an observation $36$ had been copied as $63$. Find the correct mean and SD.
  • 5 Ten observations have mean $50$ and SD $6$ (divisor $n$). Can one of them equal $70$?
  • 6 The first four moments of a data set about $2$ are $1,6,20,90$. Find the mean, $m_2,m_3,m_4$, $\beta_1$ and $\beta_2$, and check Pearson's inequality.
  • 7 A data set consists of $k$ zeros and $n-k$ ones ($0\lt k\lt n$), and $p=(n-k)/n$. Find $\gamma_1$ and $\beta_2$ in terms of $p$ and verify $\beta_2=\beta_1+1$.
  • 8 Grouped data with class width $h=4$ give $\bar\mu_2=20$ and $\bar\mu_4=1000$. Apply Sheppard's corrections.
1 The mean is $8$; the squared deviations $16,4,0,4,16$ give $s^2=8$ and $s=2\sqrt2\approx2.828$; $\mathrm{CV}=2.828/8\approx35.4\%$; $\mathrm{MD}(\bar x)=(4+2+0+2+4)/5=2.4$.
2 $\bar y=3-2\cdot5=-7$ and $s_y=|-2|\cdot4=8$ (Theorems 1.1(2) and 1.4(2)).
3 $\bar x=\tfrac{n+1}2$ and $\overline{x^2}=\tfrac{(n+1)(2n+1)}{6}$, so $s^2=\tfrac{(n+1)(2n+1)}6-\tfrac{(n+1)^2}4=\tfrac{(n+1)(4n+2-3n-3)}{12}=\tfrac{n^2-1}{12}$.
4 $\sum x=1000$, corrected to $1000-63+36=973$, so the mean is $38.92$. $\sum x^2=25(25+1600)=40625$, corrected to $40625-3969+1296=37952$. Then $s^2=37952/25-38.92^2=1518.08-1514.7664=3.3136$, so $s\approx1.820$.
5 No. By Samuelson's inequality, $|x_k-50|\le6\sqrt9=18$, so every observation lies in $[32,68]$.
6 $d=1$, so the mean is $3$. $m_2=6-1=5$; $m_3=20-3\cdot6\cdot1+2=4$; $m_4=90-4\cdot20+6\cdot6-3=43$. $\beta_1=16/125=0.128$ and $\beta_2=43/25=1.72\ge1.128$. Consistent.
7 The mean is $p$, and with $q=1-p$, $m_r=q(-p)^r+p\,q^r$. So $m_2=pq$, $m_3=pq(q-p)$ and $m_4=pq(p^3+q^3)=pq(1-3pq)$. Hence $\gamma_1=\dfrac{q-p}{\sqrt{pq}}$, $\beta_1=\dfrac{(q-p)^2}{pq}=\dfrac{1-4pq}{pq}$ and $\beta_2=\dfrac{1-3pq}{pq}=\beta_1+1$.
8 $\mu_2\approx20-16/12\approx18.667$; $\mu_4\approx1000-\tfrac{16}{2}\cdot20+\tfrac{7\cdot256}{240}=1000-160+7.467=847.47$.
III
Correlation and regression.
  • 1 For $(x,y)=(1,2),(3,4),(4,4),(6,7),(8,9),(9,10)$, compute $r$, $b_{yx}$ and $b_{xy}$, and write both regression lines.
  • 2 Show that for $x=-2,-1,0,1,2$ and $y=x^2$ the correlation is $0$. Find a relation $y=g(x)$ on the same $x$-values with $r=1$ and $g$ not linear, or explain why none exists.
  • 3 Let $u=(x-10)/2$ and $v=(30-y)/5$. If $r_{uv}=0.6$ and $b_{vu}=0.8$, find $r_{xy}$ and $b_{yx}$.
  • 4 The regression lines of a bivariate data set are $x+2y-7=0$ and $25x+8y-91=0$, and $s_x=5$. Identify the lines, and find the means, $r$ and $s_y$.
  • 5 Using Theorem 1.12(4), show that the angle between the two regression lines increases as $|r|$ decreases (for fixed $s_x,s_y$).
  • 6 Prove that if $b_{yx}\gt1$ then $b_{xy}\lt1$, and that $\tfrac12(b_{yx}+b_{xy})\ge r$ when $r\gt0$.
  • 7 Show that the residuals of the line of $y$ on $x$ have zero correlation with $x$ and with the fitted values $\hat y_i$.
  • 8 Applied. In a city, the correlation between the daily maximum temperature (°C) and ice-cream sales (cups) is $0.82$, and the regression slope of sales on temperature is $12$ cups per °C. What are the correlation and the slope if the temperature is recorded in °F?
1 $\bar x=31/6\approx5.167$, $\bar y=6$; $s_x^2\approx7.806$, $s_y^2\approx8.333$, $s_{xy}=8$. So $r=8/\sqrt{7.806\cdot8.333}\approx0.992$, $b_{yx}=8/7.806\approx1.025$ and $b_{xy}=8/8.333=0.96$. Lines: $y-6=1.025(x-5.167)$, i.e. $y\approx1.025x+0.705$, and $x-5.167=0.96(y-6)$, i.e. $x\approx0.96y-0.593$.
2 $\bar x=0$ and $\sum x_iy_i=\sum x_i^3=0$, so $s_{xy}=0$. For $r=1$ the points must be collinear (Theorem 1.10), so $g$ must agree with a linear function at the five points. A non-linear $g$ does exist: $g(x)=x+x(x^2-1)(x^2-4)$ is a quintic, but the added term vanishes at $x=0,\pm1,\pm2$, so $g(x_i)=x_i$ at every data point and $r=1$. Correlation sees only the values at the data points.
3 $x=10+2u$ and $y=30-5v$, so $r_{xy}=\operatorname{sign}(2\cdot(-5))r_{uv}=-0.6$. Also $s_{xy}=-10s_{uv}$ and $s_x^2=4s_u^2$, so $b_{yx}=-\tfrac{10}{4}b_{vu}=-2$.
4 Means: from $x=7-2y$, $25(7-2y)+8y=91$, so $y=2$ and $x=3$. Taking $x+2y-7=0$ as $y$ on $x$ gives $b_{yx}=-0.5$; taking $25x+8y-91=0$ as $x$ on $y$ gives $b_{xy}=-8/25=-0.32$; the product $0.16\le1$ is consistent. (The other way gives $b_{xy}=-2$, $b_{yx}=-25/8$, product $6.25$, which is impossible.) So $r=-0.4$ and $s_y=b_{yx}s_x/r=(-0.5)(5)/(-0.4)=6.25$.
5 $\tan\theta=\dfrac{1-r^2}{|r|}\cdot C$ with $C=\dfrac{s_xs_y}{s_x^2+s_y^2}\gt0$. The map $t\mapsto(1-t^2)/t=1/t-t$ is strictly decreasing on $(0,1]$, so $\theta$ increases as $|r|$ decreases, from $0$ at $|r|=1$ towards $90^\circ$ as $r\to0$.
6 $b_{yx}b_{xy}=r^2\le1$, so $b_{yx}\gt1$ forces $0\lt b_{xy}\lt1/b_{yx}\lt1$. For $r\gt0$, both coefficients are positive and AM $\ge$ GM gives $\tfrac12(b_{yx}+b_{xy})\ge\sqrt{b_{yx}b_{xy}}=r$.
7 The normal equations give $\sum e_i=0$ and $\sum x_ie_i=0$, so $\sum(x_i-\bar x)(e_i-\bar e)=\sum x_ie_i-\bar x\sum e_i=0$. Since $\hat y_i=\alpha+\beta x_i$ is a linear function of $x_i$, the covariance with $\hat y$ also vanishes.
8 $F=\tfrac95C+32$ is a change of origin and of positive scale, so $r=0.82$ is unchanged (Theorem 1.11). Since $C=\tfrac59(F-32)$, the slope per °F is $12\cdot\tfrac59\approx6.67$ cups per °F.
IV
Proofs and challenges.
  • 1 Prove $|r|\le1$ by expanding $\sum(z_i\pm w_i)^2\ge0$, where $z_i=(x_i-\bar x)/s_x$ and $w_i=(y_i-\bar y)/s_y$.
  • 2 For nonnegative data with $\bar x\gt0$, prove that $\mathrm{CV}\le\sqrt{n-1}$, and find when equality holds.
  • 3 Give data for which $\mathrm{MD}(\mathrm{Me})\lt\mathrm{MD}(\bar x)\lt s$ (all inequalities strict), and data for which all three are equal.
  • 4 Prove that $\beta_2=1$ if and only if the data consist of two distinct values, each occurring equally often.
  • 5 If $r_{xy}=r_{yz}=r$ with $\tfrac{1}{\sqrt2}\le r\le1$, prove that $r_{xz}\ge2r^2-1\ge0$.
  • 6 Compute $r$ for the pooled data $(1,10),(2,11),(10,1),(11,2)$ and explain why each half has $r=1$.
  • 7 Show that $\sum_{i\lt j}|x_i-x_j|\le\dfrac{n}{\sqrt2}\sqrt{\sum_{i\lt j}(x_i-x_j)^2}$, and deduce a bound for Gini's mean difference $\tfrac{1}{\binom n2}\sum_{i\lt j}|x_i-x_j|$ in terms of $s$.
1 $\sum z_i^2=\sum w_i^2=n$ and $\sum z_iw_i=nr$. So $0\le\sum(z_i\pm w_i)^2=2n(1\pm r)$, which gives $r\ge-1$ and $r\le1$. Equality $r=\pm1$ needs $w_i=\pm z_i$ for every $i$: collinearity.
2 For $x_i\ge0$, $\sum x_i^2\le\left(\sum x_i\right)^2$, so $\overline{x^2}\le n\bar x^2$ and $s^2=\overline{x^2}-\bar x^2\le(n-1)\bar x^2$. Equality holds iff all cross terms $x_ix_j$ ($i\ne j$) vanish, i.e. exactly one observation is nonzero.
3 For $0,0,3$: Me $=0$ and $\bar x=1$, so $\mathrm{MD}(\mathrm{Me})=1$, $\mathrm{MD}(\bar x)=\tfrac{1+1+2}{3}=\tfrac43$ and $s=\sqrt2\approx1.414$. For $0,2$: Me $=\bar x=1$ and $\mathrm{MD}(\mathrm{Me})=\mathrm{MD}(\bar x)=s=1$. (Equality in Cauchy–Schwarz needs all $|x_i-\bar x|$ equal.)
4 $\beta_2=1$ forces $\beta_1=0$ and $\beta_2=\beta_1+1$ (Theorem 1.8), so the data take two values. By II.7, $\gamma_1=(q-p)/\sqrt{pq}=0$ then gives $p=q=\tfrac12$. Conversely, for $p=\tfrac12$, II.7 gives $\beta_2=(1-\tfrac34)/\tfrac14=1$.
5 With $\theta=\arccos r\le45^\circ$, the angle between the $x$- and $z$-vectors is at most $2\theta\le90^\circ$, and cosine is decreasing on $[0,\pi]$, so $r_{xz}\ge\cos2\theta=2r^2-1\ge0$.
6 $\bar x=\bar y=6$; deviations $x:-5,-4,4,5$ and $y:4,5,-5,-4$. So $\sum dx\,dy=-20-20-20-20=-80$ and $\sum dx^2=\sum dy^2=82$, giving $r=-80/82\approx-0.976$. Each half consists of two points, which always lie on a line; here the slope is $+1$, so $r=+1$.
7 Cauchy–Schwarz over the $\binom n2$ pairs: $\left(\sum_{i\lt j}|x_i-x_j|\right)^2\le\binom n2\sum_{i\lt j}(x_i-x_j)^2\le\tfrac{n^2}{2}\sum_{i\lt j}(x_i-x_j)^2$. By Theorem 1.4(3), $\sum_{i\lt j}(x_i-x_j)^2=n^2s^2$. Hence Gini's mean difference is at most $s\sqrt{n^2/\binom n2}=s\sqrt{2n/(n-1)}$.
🎯 Exam corner CSIR Part BCSIR Part CGATE ST

E1. CSIR Part B GATE ST The variance, computed with divisor $n$, of the first $20$ natural numbers is (A) $33.25$; (B) $35$; (C) $36.75$; (D) $31.5$.

Idea Exercise II.3: $s^2=(n^2-1)/12$.
Options $(400-1)/12=33.25$. Option (B) is the divisor-$(n-1)$ value $n(n+1)/12=35$, a deliberate trap; (C) and (D) match no formula.
Answer (A).

E2. CSIR Part B The two lines of regression of a bivariate data set are $9x-10y+35=0$ and $5x-2y-9=0$. The correlation coefficient is (A) $0.6$; (B) $-0.6$; (C) $0.36$; (D) $5/3$.

Idea Use the strategy box: only one assignment gives $b_{yx}b_{xy}\le1$.
Assignment First as $y$ on $x$: $y=0.9x+3.5$, so $b_{yx}=0.9$. Second as $x$ on $y$: $x=0.4y+1.8$, so $b_{xy}=0.4$. The product is $0.36$, which is fine. The reverse gives $b_{xy}=10/9$ and $b_{yx}=2.5$ with product $25/9\gt1$, which is impossible. (The means are $\bar x=5$, $\bar y=8$.)
Answer (A): $r=+\sqrt{0.36}=0.6$, positive like both coefficients. (C) is $r^2$, and (D) comes from the impossible assignment.

E3. CSIR Part B Let $g(a)=\sum_{i=1}^6|x_i-a|$ for the data $2,5,9,11,20,26$. Which is true? (A) $g$ has a unique minimiser, the mean $73/6$; (B) $g$ is minimised exactly on $[9,11]$ and its minimum value is $41$; (C) $g$ is minimised only at $a=10$, with minimum value $40$; (D) $g$ is minimised exactly on $[5,20]$.

Idea Theorem 1.2: for even $n$ the minimisers form the closed interval between the two middle order statistics.
Value $n=6$, so the interval is $[x_{(3)},x_{(4)}]=[9,11]$, and the minimum is $(11+20+26)-(2+5+9)=41$. Check: $g(10)=8+5+1+1+10+16=41$ and $g(73/6)\approx43.33$.
Answer (B).

E4. CSIR Part C Which of the following statements are true? (One or more options may be correct.) (A) If $r_{xy}=0.8$, then the correlation between $u=2x+3$ and $v=5-y$ is $-0.8$. (B) A data set can have regression coefficients $b_{yx}=1.6$ and $b_{xy}=0.7$. (C) If $r_{xy}=r_{yz}=0.9$, then $r_{xz}\gt0$. (D) If $r_{xy}=0$, then $y$ is not a function of $x$ on the data.

(A) True Theorem 1.11 with $b=2$ and $d=-1$: the sign flips once.
(B) False $b_{yx}b_{xy}=1.12\gt1=\max r^2$.
(C) True The angle argument of Example 6(c): $r_{xz}\ge2(0.81)-1=0.62$.
(D) False $y=x^2$ on $-2,\dots,2$ has $r=0$.
Answer (A) and (C).

E5. CSIR Part C GATE ST For data with positive variance, which of the following are true? (A) $\beta_2\ge1$ always. (B) $\beta_2\ge3$ whenever the data have a single mode. (C) If $\gamma_1=0$, then the mean equals the median. (D) If the data take exactly two distinct values, then $\beta_2=\beta_1+1$.

(A) True Theorem 1.8: $\beta_2\ge1+\beta_1\ge1$.
(B) False $1,2,2,3$ has the single mode $2$, but $m_2=\tfrac12$ and $m_4=\tfrac12$, so $\beta_2=2$.
(C) False $0,0,5,5,5,9$: the deviations from the mean $4$ are $-4,-4,1,1,1,5$, whose cubes sum to $0$, so $\gamma_1=0$, yet the median is $5$.
(D) True This is the equality case of Theorem 1.8 (Exercise II.7).
Answer (A) and (D).

E6. CSIR Part C A data set has $20$ distinct observations. Its largest observation is increased by $100$. Which of the following are necessarily unchanged? (A) the median; (B) the $10\%$ trimmed mean; (C) the standard deviation; (D) the mean deviation about the median.

(A) Unchanged The largest value remains the largest, so $x_{(10)}$ and $x_{(11)}$ do not change.
(B) Unchanged $k=2$: the trimmed mean averages $x_{(3)},\dots,x_{(18)}$, and the changed value is discarded.
(C) Changed As a function of the largest value $t$, $ns^2$ has derivative $2(t-\bar x)\gt0$ (the largest of distinct values exceeds the mean, and it stays above the mean as $t$ grows). So $s$ strictly increases.
(D) Changed The median is unchanged and $|t+100-\mathrm{Me}|=|t-\mathrm{Me}|+100$, so $\mathrm{MD}(\mathrm{Me})$ increases by exactly $5$.
Answer (A) and (B).
🧑‍🏫 For the teacher

Suggested plan: 3 lectures × 60 min.

  1. Looking at data. Variable types (Definition 1.1) with students' own examples. Build Example 1 on the board: the stem-and-leaf display first, then the table, then the histogram drawn twice (wrong heights, then densities). Theorems 1.1 and 1.2 with their proofs: the slope argument for the median is short and memorable. Theorem 1.3. Example 2 for grouped data.
  2. Spread and shape. Box plots and fences with Example 3. Project Interactive 1 on the clinic data and drag the largest value: ask the class to predict which readouts will move before dragging. Theorems 1.4–1.6 (prove Samuelson's inequality; students rarely see it). Example 4, then moments, Theorem 1.7, Example 5, Pearson's inequality (Theorem 1.8) and a brief Sheppard sketch.
  3. Two variables. Theorem 1.10 as "$r$ is a cosine", which makes Example 6(c, d) a picture rather than a calculation. Theorems 1.11–1.12, the strategy box, Example 7. Project Interactive 2: sweep $\rho$ to watch the scissors close, then show the curved cloud and the outlier. Finish with E2 and E4 live.

Discussion questions. (1) A newspaper reports that "average household income rose by 8%" while the median fell. Is this possible, and what would it suggest about the distribution? (2) Why does the least-squares line of $y$ on $x$ differ from that of $x$ on $y$? Which would a doctor predicting weight from height want? (3) Is there a data set whose box plot shows an outlier on each side while its mean equals its median? Construct one.

Misconceptions to watch for. "$r=0$ means independent / unrelated"; "mean $\gt$ median proves positive skewness"; "the regression line of $x$ on $y$ is the inverse of that of $y$ on $x$"; "pooled SD lies between the group SDs"; using frequencies rather than densities for unequal classes; applying Sheppard's correction to discrete data; mixing the divisors $n$ and $n-1$.

Suggested homework. I(2, 4, 5, 8), II(3, 4, 5, 6), III(1, 3, 4, 8), IV(2, 3, 5); Exam corner E1, E3, E5, E6.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. How much bad data can a summary survive, and what does that robustness cost when the data are clean?

Why it matters. Air-quality networks, hospital records and sensor logs routinely contain gross errors: a stuck sensor reading 9999, a weight typed in grams instead of kilograms. Agencies that publish daily pollution indices, and clinical trials that report "typical" responses, must choose summaries that do not explode on such errors, yet do not throw away information when the data are clean. Robust statistics, founded by Tukey, Huber and Hampel, studies exactly this trade-off.

What to do.

  1. Take a real data set of 30–100 numbers. A month of daily PM2.5 readings for a city from a public monitoring portal works well, as does a list of hospital waiting times. Compute the mean, median, $10\%$ and $25\%$ trimmed means, SD, IQR and MAD.
  2. Replace $k=1,2,3,\dots$ observations by $10^6$ and record each summary. Find the smallest $k$ at which each one "breaks down" (becomes absurd), and express it as a fraction of $n$: the breakdown point.
  3. Now simulate clean data: $2000$ samples of size $n=25$ from $N(0,1)$, generated with a seeded random number generator and the Box–Muller transform. Estimate the variance of the mean, the median and the trimmed means across samples. The ratio (variance of the mean)/(variance of the estimator) is the estimator's efficiency.
  4. Repeat with contaminated data: each observation comes from $N(0,1)$ with probability $0.95$ and from $N(0,100)$ with probability $0.05$. Which estimator is best now?

Tools: Python (NumPy) or R, or a spreadsheet for steps 1–2.

Hand in: A one-page table of breakdown points and efficiencies under both models, and a recommendation (with reasons) for the summary a pollution agency should publish.

Going further. Read about Huber's M-estimator, which minimises $\sum\rho(x_i-c)$ for a function $\rho$ that is quadratic near $0$ and linear far out. This interpolates between Theorem 1.1 (the mean) and Theorem 1.2 (the median).

Instructor note: the breakdown points are $1/n$ for the mean and SD, about $1/2$ for the median and MAD, about $1/4$ for the IQR, and about $\alpha$ for the $\alpha$-trimmed mean. For large $n$ at the normal model, the efficiencies relative to the mean are $2/\pi\approx0.637$ for the median, about $0.943$ for the $10\%$ trimmed mean and about $0.837$ for the $25\%$ trimmed mean; simulations with $n=25$ come close to these. Under $5\%$ contamination by $N(0,100)$ the mean's variance is inflated about sixfold, and the trimmed means win clearly.

🧪 Self-check quiz

0/5 correct

Q1. The value of $c$ that minimises $\sum_{i=1}^n|x_i-c|$ is

Q2. The regression coefficients of a data set are $b_{yx}=-0.9$ and $b_{xy}=-0.4$. Then $r$ equals

Q3. Two groups have sizes $10$ and $10$, means $20$ and $30$, and SDs $4$ and $4$ (divisor $n$). The SD of the combined data is

Q4. For every data set with positive variance, the kurtosis $\beta_2$ satisfies

Q5. Which summary is unchanged when the largest of $20$ distinct observations is multiplied by $10$?