1.1 Descriptive Statistics and Exploratory Data Analysis
Before any model comes a careful look at the data: frequency tables and graphs, measures of centre and spread with their extremal properties, moments, skewness and kurtosis, and the correlation coefficient with its least-squares lines.
A statistician handed a spreadsheet of a thousand numbers is first of all a describer. What do the data look like? Where is their centre, how widely are they spread, are they symmetric or lopsided, are there gross errors or genuinely extreme values, and do two measured variables move together? John Tukey called this stage exploratory data analysis and insisted that it comes before any formal inference: a test or a confidence interval computed on data nobody has looked at can be very misleading. A single mis-typed value can double a standard deviation. A strongly skewed sample makes the mean a poor summary. A curved relation can produce a correlation of zero.
This section develops the standard summaries (graphs, measures of location and dispersion, moments, skewness, kurtosis, correlation and the least-squares line) and, just as importantly, the theorems behind them. The mean minimises squared deviations and the median minimises absolute deviations. The correlation coefficient lies in $[-1,1]$ because it is the cosine of an angle. Kurtosis is bounded below by $1+$ (skewness)$^2$. CSIR NET examines exactly these facts: Part B asks you to compute a combined variance or to find $r$ from two regression lines, and Part C asks which statements about such summaries are always true. The same summaries reappear throughout the course as statistics computed from random samples. In Chapter 4 we ask how well they estimate their population counterparts.
Data, Frequency Tables and Graphical Summaries
A population is the collection of all units of interest, and a sample is the part of it that is observed. A variable is a characteristic recorded for each unit, and the recorded values $x_1,\dots,x_n$ are the data. Variables are classified by the operations that are meaningful on their values:
Numerical variables are discrete (counts: number of children, defects) or continuous (measurements, recorded to some precision). Nominal and ordinal variables are called qualitative or categorical; interval and ratio variables are quantitative.
Divide the range of the data into disjoint classes $[a_0,a_1),[a_1,a_2),\dots,[a_{k-1},a_k)$ with widths $h_j=a_j-a_{j-1}$ and mid-points $m_j=\tfrac12(a_{j-1}+a_j)$. The frequency $f_j$ is the number of observations in class $j$; $f_j/n$ is the relative frequency; $F_j=f_1+\cdots+f_j$ is the cumulative frequency. The histogram draws over class $j$ a rectangle of height
$$d_j=\frac{f_j}{n\,h_j}\qquad\text{(the frequency density)},$$so that the area of the rectangle equals the relative frequency, and the total area is $1$. When all classes have the same width, heights proportional to $f_j$ give the same picture. A stem-and-leaf display splits each value into a stem (leading digits) and a leaf (last digit). It shows the shape like a histogram while keeping every observation. The graph of $F_j/n$ against the class end points $a_j$ is the ogive (cumulative frequency curve).
The rule "area = relative frequency" is not decoration. If the last class of a frequency table is three times as wide as the others and we draw heights equal to frequencies, that class looks three times more important than it is. The next example shows this.
1 | 2 5 8, 2 | 1 2 4 5 7 8 9, 3 | 1 3 4 6 8, 4 | 1 5, 5 | 2 8, 6 |, 7 | 1. The display is longer to the right: the data are right-skewed, and $71$ sits apart from the rest.Measures of Location and Quantiles
Let $x_1,\dots,x_n$ be numerical data and $x_{(1)}\le x_{(2)}\le\cdots\le x_{(n)}$ the same values in increasing order (the order statistics).
- The (arithmetic) mean is $\bar x=\frac1n\sum_{i=1}^nx_i$; for weights $w_i\ge0$ with $\sum w_i\gt0$ the weighted mean is $\sum w_ix_i/\sum w_i$. For grouped data, $\bar x\approx\frac1n\sum_jf_jm_j$.
- The median is the middle order statistic $x_{((n+1)/2)}$ if $n$ is odd, and $\tfrac12\left(x_{(n/2)}+x_{(n/2+1)}\right)$ if $n$ is even. Any number $a$ with at least half the data $\le a$ and at least half $\ge a$ is a median; the convention picks the mid-point of the interval of such numbers.
- A mode is a most frequent value (for grouped data, a point of highest density).
- The $\alpha$-trimmed mean ($0\le\alpha\lt\tfrac12$, with $k=\lfloor n\alpha\rfloor$) is the mean of $x_{(k+1)},\dots,x_{(n-k)}$, i.e. the mean after the $k$ smallest and $k$ largest values are discarded.
- For positive data the geometric mean is $G=\left(\prod x_i\right)^{1/n}$ and the harmonic mean is $H=n\big/\sum(1/x_i)$.
For any data $x_1,\dots,x_n$ and constants $a,b$:
- $\sum_{i=1}^n(x_i-\bar x)=0$;
- if $y_i=a+bx_i$ then $\bar y=a+b\bar x$;
- for every real $c$, $\displaystyle\sum_{i=1}^n(x_i-c)^2=\sum_{i=1}^n(x_i-\bar x)^2+n(\bar x-c)^2$. Hence $c\mapsto\sum(x_i-c)^2$ has the unique minimiser $c=\bar x$.
Proof. (1) $\sum(x_i-\bar x)=\sum x_i-n\bar x=0$. (2) $\frac1n\sum(a+bx_i)=a+b\cdot\frac1n\sum x_i$. (3) Write $x_i-c=(x_i-\bar x)+(\bar x-c)$ and expand: $$\sum(x_i-c)^2=\sum(x_i-\bar x)^2+2(\bar x-c)\sum(x_i-\bar x)+n(\bar x-c)^2,$$ and the middle term vanishes by (1). The last term is $\ge0$, with equality only for $c=\bar x$. ∎
Proof. Each $c\mapsto|x_i-c|$ is convex and has slope $-1$ to the left of $x_i$ and $+1$ to the right, so $g$ is convex and piecewise linear. At a point $c$ that is not a data value, its slope is $$g'(c)=\#\{i:x_i\lt c\}-\#\{i:x_i\gt c\}.$$ This slope is negative while fewer than half the observations lie below $c$ and positive once more than half do. If $n$ is odd, $g'\lt0$ to the left of $x_{((n+1)/2)}$ and $g'\gt0$ to its right, so that point is the unique minimiser. If $n$ is even, $g'=0$ on the open interval $\left(x_{(n/2)},x_{(n/2+1)}\right)$ (exactly $n/2$ points on each side), $g'\lt0$ before it and $g'\gt0$ after it; so the minimisers form the closed interval. For $c$ in this interval, $|x_{(i)}-c|=c-x_{(i)}$ for $i\le n/2$ and $x_{(i)}-c$ for $i\gt n/2$; the $c$'s cancel and give the stated value. ∎
Proof. The function $\log$ is concave, so $\log t\le t-1$ for $t\gt0$, with equality only at $t=1$ (the tangent line at $t=1$). Put $t_i=x_i/G$. Then $\sum\log t_i=\sum\log x_i-n\log G=0$, so $$0=\sum\log t_i\le\sum(t_i-1)=\frac{n\bar x}{G}-n,$$ i.e. $\bar x\ge G$, with equality iff every $t_i=1$. Applying this to the positive numbers $1/x_i$, whose AM is $1/H$ and whose GM is $1/G$, gives $1/H\ge1/G$, i.e. $G\ge H$. For two values, $\bar x\,H=\tfrac{a+b}{2}\cdot\tfrac{2ab}{a+b}=ab=G^2$. ∎
The first quartile $Q_1$ is the median of the observations below the median position, and the third quartile $Q_3$ is the median of those above it (when $n$ is odd, the median itself is left out of both halves). The five-number summary is $\left(x_{(1)},Q_1,\text{median},Q_3,x_{(n)}\right)$, and the interquartile range is $\mathrm{IQR}=Q_3-Q_1$. A box plot draws a box from $Q_1$ to $Q_3$ with a line at the median, and whiskers to the most extreme observations inside the fences $$Q_1-1.5\,\mathrm{IQR},\qquad Q_3+1.5\,\mathrm{IQR}.$$ Observations outside the fences are flagged as outliers and plotted individually.
More generally, a $p$-th quantile ($0\lt p\lt1$) is a number with at least a fraction $p$ of the data at or below it and at least $1-p$ at or above it. Several interpolation conventions are in use (statistical software usually takes $x_{(j)}+\theta\left(x_{(j+1)}-x_{(j)}\right)$ with $j+\theta=1+(n-1)p$), and they can give slightly different quartiles for small $n$. Exam questions are designed so that the convention does not matter, or they state it.
When only a frequency table is available, assume the observations in each class are spread uniformly across it. Let the class containing the required point have lower boundary $L$, width $h$ and frequency $f$, let $F$ be the cumulative frequency before it, and let $N=\sum f_j$.
| marks | 0–10 | 10–20 | 20–30 | 30–40 | 40–50 |
|---|---|---|---|---|---|
| students | 5 | 12 | 20 | 9 | 4 |
Dispersion, Moments, Skewness and Kurtosis
- Range $R=x_{(n)}-x_{(1)}$; interquartile range $\mathrm{IQR}=Q_3-Q_1$.
- Variance $s^2=\frac1n\sum(x_i-\bar x)^2$ and standard deviation $s=\sqrt{s^2}$ (divisor $n$, used throughout this section). The sample variance $s_{n-1}^2=\frac{1}{n-1}\sum(x_i-\bar x)^2=\frac{n}{n-1}s^2$ is the version that estimates a population variance without bias (Chapter 4).
- Mean deviation about $A$: $\mathrm{MD}(A)=\frac1n\sum|x_i-A|$, usually taken about the mean or the median.
- Median absolute deviation: $\mathrm{MAD}=\operatorname{median}_i|x_i-\mathrm{Me}|$, a robust measure of spread.
- Coefficient of variation (ratio data with $\bar x\gt0$): $\mathrm{CV}=s/\bar x$, often quoted as a percentage. It is unit-free, so it compares variability across different scales.
- $s^2=\overline{x^2}-\bar x^2$, where $\overline{x^2}=\frac1n\sum x_i^2$. More generally, for any $A$, $s^2=\frac1n\sum(x_i-A)^2-(\bar x-A)^2$.
- If $y_i=a+bx_i$ then $s_y^2=b^2s_x^2$ and $s_y=|b|\,s_x$: variance ignores a change of origin and scales by the square of a change of scale.
- $s^2=\dfrac{1}{2n^2}\displaystyle\sum_{i=1}^n\sum_{j=1}^n(x_i-x_j)^2$: the variance is half the mean squared difference between two observations.
Proof. (1) Theorem 1.1(3) with $c=A$, divided by $n$; for $A=0$ it reads $\overline{x^2}=s^2+\bar x^2$. (2) $y_i-\bar y=b(x_i-\bar x)$ by Theorem 1.1(2); square and average. (3) Expand $(x_i-x_j)^2=x_i^2-2x_ix_j+x_j^2$ and sum over both indices: $$\sum_i\sum_j(x_i-x_j)^2=2n\sum x_i^2-2\Big(\sum x_i\Big)^2=2n^2\left(\overline{x^2}-\bar x^2\right)=2n^2s^2.\ ∎$$
For any data (with $s$ computed with divisor $n$):
- $\mathrm{MD}(\mathrm{Me})\le\mathrm{MD}(\bar x)\le s$;
- $|\bar x-\mathrm{Me}|\le s$: the median lies within one standard deviation of the mean;
- if all $x_i\in[a,b]$ then $s\le\tfrac12(b-a)$; in particular $s\le R/2$;
- (Samuelson's inequality) every observation satisfies $|x_k-\bar x|\le s\sqrt{n-1}$.
Proof. (1) The first inequality is Theorem 1.2: the median minimises $\sum|x_i-c|$. For the second, apply the Cauchy–Schwarz inequality to the vectors $(|x_i-\bar x|)_i$ and $(1,\dots,1)$: $\left(\sum|x_i-\bar x|\right)^2\le n\sum(x_i-\bar x)^2$; divide by $n^2$ and take square roots.
(2) By the triangle inequality and (1), $$|\bar x-\mathrm{Me}|=\Big|\tfrac1n\sum(x_i-\mathrm{Me})\Big|\le\tfrac1n\sum|x_i-\mathrm{Me}|=\mathrm{MD}(\mathrm{Me})\le s.$$
(3) By Theorem 1.1(3), $s^2$ is the minimum of $\frac1n\sum(x_i-c)^2$ over $c$. Taking $c=\tfrac{a+b}{2}$, every term is at most $\left(\tfrac{b-a}{2}\right)^2$, because each $x_i$ is within $\tfrac{b-a}{2}$ of the centre of $[a,b]$. Equality needs every $x_i\in\{a,b\}$ and $\bar x=\tfrac{a+b}2$: half the data at each end.
(4) Let $d_i=x_i-\bar x$. Then $\sum_{i\ne k}d_i=-d_k$ by Theorem 1.1(1), and by Cauchy–Schwarz $$d_k^2=\Big(\sum_{i\ne k}d_i\Big)^2\le(n-1)\sum_{i\ne k}d_i^2=(n-1)\left(ns^2-d_k^2\right).$$ Rearranging, $nd_k^2\le(n-1)ns^2$, i.e. $|d_k|\le s\sqrt{n-1}$. ∎
Samuelson's bound is attained: for the data $(1,\ 0,\ \dots,\ 0)$ with $n-1$ zeros, $\bar x=\tfrac{1}{n}$, $s^2=\tfrac{n-1}{n^2}$, and $x_1-\bar x=\tfrac{n-1}{n}=s\sqrt{n-1}$. So an observation can lie as many as $\sqrt{n-1}$ standard deviations from the mean, but no further.
Proof. The total of all observations is $\sum_jn_j\bar x_j$, which gives the mean. For group $j$, Theorem 1.1(3) with $c=\bar x$ gives $\sum_{i\in j}(x_i-\bar x)^2=n_js_j^2+n_j(\bar x_j-\bar x)^2$. Sum over $j$ and divide by $N$. ∎
This decomposition is the germ of the analysis of variance (Chapter 6), where a large between-groups part relative to the within-groups part is evidence that the group means differ.
The $r$-th moment about $A$ is $m_r'(A)=\frac1n\sum(x_i-A)^r$ (raw moments are those about $A=0$), and the $r$-th central moment is $m_r=\frac1n\sum(x_i-\bar x)^r$. Thus $m_1=0$ and $m_2=s^2$. The moment measures of shape are
$$\beta_1=\frac{m_3^2}{m_2^3},\quad\gamma_1=\frac{m_3}{m_2^{3/2}},\quad\beta_2=\frac{m_4}{m_2^2},\quad\gamma_2=\beta_2-3.$$$\gamma_1$ is the coefficient of skewness (positive when the right tail is longer), and $\beta_2$ is the kurtosis ($\gamma_2$ is the excess kurtosis; the normal distribution has $\beta_2=3$). Data with $\beta_2\gt3$ are called leptokurtic (heavier tails than the normal), and those with $\beta_2\lt3$ platykurtic. Two further measures of skewness are Pearson's $(\bar x-\mathrm{Mo})/s$ and Bowley's $(Q_3+Q_1-2\mathrm{Me})/(Q_3-Q_1)\in[-1,1]$.
(a) With $d=m_1'(A)=\bar x-A$ and $m_r'=m_r'(A)$,
$$m_r=\sum_{j=0}^{r}\binom rj m_j'(-d)^{r-j}\quad(m_0'=1).$$In particular $m_2=m_2'-d^2$, $\ m_3=m_3'-3m_2'd+2d^3$ and $\ m_4=m_4'-4m_3'd+6m_2'd^2-3d^4$.
(b) If $y_i=a+bx_i$ with $b\ne0$, then $m_r(y)=b^rm_r(x)$. Hence $\beta_1$ and $\beta_2$ are unchanged, and $\gamma_1$ is multiplied by $\operatorname{sign}b$.
Proof. (a) $x_i-\bar x=(x_i-A)-d$; expand $\left((x_i-A)-d\right)^r$ by the binomial theorem and average over $i$. Since $m_1'=d$, the terms $j=0$ and $j=1$ combine: $m_0'(-d)^r+\binom r1m_1'(-d)^{r-1}$ $=(-d)^r-r(-d)^r$ $=(1-r)(-d)^r$, which gives the coefficients $2d^3$ for $r=3$ and $-3d^4$ for $r=4$. (b) $y_i-\bar y=b(x_i-\bar x)$, so each central moment picks up the factor $b^r$. In $\beta_1=m_3^2/m_2^3$ the factor is $b^6/b^6$, in $\beta_2$ it is $b^4/b^4$, and in $\gamma_1$ it is $b^3/|b|^3=\operatorname{sign}b$. ∎
Proof. Standardise: $z_i=(x_i-\bar x)/s$. Then $\frac1n\sum z_i=0$, $\frac1n\sum z_i^2=1$, $\frac1n\sum z_i^3=\gamma_1$ and $\frac1n\sum z_i^4=\beta_2$. Consider the nonnegative average $$0\le\frac1n\sum_i\left(z_i^2-\gamma_1z_i-1\right)^2.$$ Expanding the square and using the four moments gives $\beta_2+\gamma_1^2+1-2\gamma_1\cdot\gamma_1-2+2\gamma_1\cdot0=\beta_2-\gamma_1^2-1$. Hence $\beta_2\ge1+\gamma_1^2=1+\beta_1$. Equality holds iff $z_i^2-\gamma_1z_i-1=0$ for every $i$, i.e. every $z_i$ is a root of one fixed quadratic: the data take at most two values. Conversely, if the data take two values, a direct computation (Exercise II.7) gives equality. ∎
So kurtosis can never be below $1$, and it is large when skewness is large. $\beta_2=1$ exactly when the data are two values in equal proportions: the most "anti-peaked" possible configuration.
Proof (sketch). Under the stated conditions the grouping error behaves like a variable $U$, uniform on $\left(-\tfrac h2,\tfrac h2\right)$ and independent of the true value $X$, with the grouped value distributed like $X+U$. Since $E[U]=E[U^3]=0$, $E[U^2]=h^2/12$ and $E[U^4]=h^4/80$, expanding the central moments of $X+U$ gives $\bar\mu_2=\mu_2+\tfrac{h^2}{12}$, $\bar\mu_3=\mu_3$ and $\bar\mu_4=\mu_4+6\mu_2\tfrac{h^2}{12}+\tfrac{h^4}{80}$. Solving for $\mu_4$ and substituting $\mu_2=\bar\mu_2-\tfrac{h^2}{12}$ gives $\mu_4=\bar\mu_4-\tfrac{h^2}{2}\bar\mu_2+\left(\tfrac1{24}-\tfrac1{80}\right)h^4$, and $\tfrac1{24}-\tfrac1{80}=\tfrac{7}{240}$. (Making "behaves like" precise uses the Euler–Maclaurin formula; we omit it.) ∎
Correlation and the Least-Squares Line
Now each unit carries two measurements, giving paired data $(x_1,y_1),\dots,(x_n,y_n)$: height and weight, dose and response, temperature and electricity demand. A scatter plot shows the relation, and two numbers summarise its linear part.
Proof. The formula for $r$ is $u\cdot v/(\|u\|\|v\|)$ (the factors $\tfrac1n$ cancel). By the Cauchy–Schwarz inequality, $|u\cdot v|\le\|u\|\|v\|$, so $|r|\le1$, with equality iff $v=cu$ for a scalar $c\ne0$ (neither vector is $0$ because $s_x,s_y\gt0$). Here is a self-contained argument. For real $t$, $$0\le\|v-tu\|^2=\|v\|^2-2t\,u\cdot v+t^2\|u\|^2.$$ A quadratic in $t$ that is never negative has discriminant $4(u\cdot v)^2-4\|u\|^2\|v\|^2\le0$, which is $r^2\le1$. Equality means the quadratic has a real root $t=c$, i.e. $v=cu$: $y_i-\bar y=c(x_i-\bar x)$ for all $i$. All points then lie on a line of slope $c$, and $r=u\cdot(cu)/(\|u\|\cdot|c|\|u\|)=\operatorname{sign}c$. Conversely, if all points lie on a line $y=\alpha+cx$ with $c\ne0$, then $v=cu$. ∎
Proof. $u_i-\bar u=b(x_i-\bar x)$ and $v_i-\bar v=d(y_i-\bar y)$, so $s_{uv}=bd\,s_{xy}$, $s_u=|b|s_x$ and $s_v=|d|s_y$. Hence $r_{uv}=\dfrac{bd}{|b||d|}r_{xy}$. ∎
Assume $s_x,s_y\gt0$.
- The line of $y$ on $x$ is unique: $\ y-\bar y=b_{yx}(x-\bar x)$ with $b_{yx}=\dfrac{s_{xy}}{s_x^2}=r\dfrac{s_y}{s_x}$. Similarly $x-\bar x=b_{xy}(y-\bar y)$ with $b_{xy}=\dfrac{s_{xy}}{s_y^2}=r\dfrac{s_x}{s_y}$. Both lines pass through $(\bar x,\bar y)$.
- $b_{yx}b_{xy}=r^2\le1$. The two coefficients and $r$ have the same sign, and $r=\operatorname{sign}(b_{yx})\sqrt{b_{yx}b_{xy}}$.
- The residuals satisfy $\sum e_i=0$ and $\sum x_ie_i=0$, and the residual mean square is $\frac1n\sum e_i^2=s_y^2(1-r^2)$.
- The two lines coincide iff $|r|=1$. In the $(x,y)$ plane the acute angle $\theta$ between them satisfies $\tan\theta=\dfrac{1-r^2}{|r|}\cdot\dfrac{s_xs_y}{s_x^2+s_y^2}$ (for $r\ne0$); for $r=0$ they are the perpendicular lines $y=\bar y$ and $x=\bar x$.
Proof. (1) For fixed $\beta$, $Q$ is a sum of squares of $(y_i-\beta x_i)-\alpha$, so Theorem 1.1(3) shows it is minimised by $\alpha=\bar y-\beta\bar x$; the line then passes through $(\bar x,\bar y)$. Substituting, $$Q=\sum\left((y_i-\bar y)-\beta(x_i-\bar x)\right)^2=n\left(s_y^2-2\beta s_{xy}+\beta^2s_x^2\right),$$ a quadratic in $\beta$ with positive leading coefficient, minimised uniquely at $\beta=s_{xy}/s_x^2$. Interchanging the roles of $x$ and $y$ gives $b_{xy}$.
(2) $b_{yx}b_{xy}=s_{xy}^2/(s_x^2s_y^2)=r^2$, which is at most $1$ by Theorem 1.10. All three quantities have the sign of $s_{xy}$.
(3) The normal equations $\partial Q/\partial\alpha=\partial Q/\partial\beta=0$ read $\sum e_i=0$ and $\sum x_ie_i=0$. The minimum value is $Q=n\left(s_y^2-s_{xy}^2/s_x^2\right)=ns_y^2(1-r^2)$.
(4) The slope of the first line is $m_1=rs_y/s_x$. Solving the second for $y$ gives slope $m_2=1/b_{xy}=s_y/(rs_x)$. Then $m_1=m_2$ iff $r^2=1$, and $$\tan\theta=\left|\frac{m_2-m_1}{1+m_1m_2}\right|=\frac{\frac{s_y}{s_x}\left(\frac1{|r|}-|r|\right)}{1+\frac{s_y^2}{s_x^2}}=\frac{1-r^2}{|r|}\cdot\frac{s_xs_y}{s_x^2+s_y^2}.\ ∎$$
The quantity $r^2$ is the coefficient of determination: by (3), the line of $y$ on $x$ removes the fraction $r^2$ of the variance of $y$, and $1-r^2$ is left in the residuals. In Chapter 6 the same algebra, in matrix form, becomes the theory of linear models.
- Means: both lines pass through $(\bar x,\bar y)$, so solve the two equations simultaneously.
- Which line is which? Tentatively take the first as $y$ on $x$ and read off $b_{yx}$ (solve it for $y$), and take the second as $x$ on $y$ and read off $b_{xy}$ (solve it for $x$). If $b_{yx}b_{xy}\le1$, the choice is consistent. If the product exceeds $1$, swap the roles; exactly one choice works unless $|r|=1$ or the coefficients have opposite signs (then the data are impossible).
- Correlation: $r=\pm\sqrt{b_{yx}b_{xy}}$, with the common sign of the coefficients.
- Standard deviations: $b_{yx}=rs_y/s_x$ gives one SD from the other.
- $r=0$ does not mean "no relation". For $x=-2,-1,0,1,2$ and $y=x^2$, $s_{xy}=\frac15\sum x^3=0$, so $r=0$, although $y$ is a function of $x$. $r$ measures linear association only.
- Correlation is not transitive. $r_{xy}=r_{yz}=0.5$ with $r_{xz}=-0.5$ is possible (Example 6(d)). Only when $\arccos r_{xy}+\arccos r_{yz}\lt90^\circ$ is $r_{xz}$ forced to be positive.
- Two regression lines, not one. $b_{yx}\ne1/b_{xy}$ unless $|r|=1$. The line of $x$ on $y$ is not obtained by solving the line of $y$ on $x$ for $x$.
- Impossible coefficients. $b_{yx}=1.6$, $b_{xy}=0.7$ cannot occur (the product $1.12$ exceeds $1$), and coefficients of opposite signs cannot occur. Both can be "regression coefficients" greater than $1$ in absolute value only in a question that has an error.
- Sign flips under negative scaling. $r(x,\,5-2y)=-r(x,y)$; $r(-x,-y)=r(x,y)$. Variance ignores the origin, but the CV does not: it changes when a constant is added.
- Mean $\gt$ median does not imply $\gamma_1\gt0$ ($0,1,1,2,2$), and $\gamma_1=0$ does not imply symmetry or mean $=$ median: $0,0,5,5,5,9$ has $m_3=0$, mean $4$, median $5$.
- Pooled spread is not the average spread. The combined SD of two groups can exceed both group SDs (Example 4). The combined median is not the average of the two medians.
- One outlier breaks the mean, the SD and $r$. Moving one value changes $\bar x$ by (change)$/n$ and can make $s$ arbitrarily large. A single far point can turn $r\approx0$ into $r\approx0.9$. Medians, quartiles, MAD and trimmed means resist this.
- Histogram heights must be densities when class widths differ (Example 1).
- The CV needs ratio data with a positive mean. The CV of temperatures in °C is meaningless (it changes when you switch to kelvin), and it explodes when $\bar x\approx0$.
- Kurtosis is about tails, not "peakedness". $\beta_2\ge1$ always, and $\beta_2=1$ for two equally frequent values. Unimodal data can have $\beta_2\lt3$: for $1,2,2,3$, $\beta_2=2$.
- Divisors. $s^2$ (divisor $n$) and $s_{n-1}^2$ (divisor $n-1$) differ: for $1,2,\dots,20$ they are $33.25$ and $35$. Read the question.
- Aggregation can reverse a correlation. In groups $\{(1,10),(2,11)\}$ and $\{(10,1),(11,2)\}$ each group has $r=+1$, but the pooled data have $r\approx-0.976$.
Exercises
- 1 Classify each variable as nominal, ordinal, interval or ratio, and say whether it is discrete or continuous: (a) blood group; (b) pain score on a $0$–$10$ scale; (c) body temperature in °C; (d) monthly income in rupees; (e) number of children in a family.
- 2 For the data $7,9,10,10,12,13,15,20$, find the mean, the median, the mode and the $25\%$ trimmed mean.
- 3 For the data of part 2, find the minimum of $\sum(x_i-c)^2$ over $c$, and the value of that sum at $c=10$.
- 4 Minimise $\sum|x_i-c|$ for (a) $3,5,8,10,24$; (b) $3,5,8,10$. Give all minimisers and the minimum values.
- 5 Find the mean, median and mode of the distribution with classes $0$–$20$, $20$–$40$, $40$–$60$, $60$–$80$, $80$–$100$ and frequencies $6,14,22,12,6$.
- 6 The values $0,1,2,3,4$ occur with frequencies $2,5,f,6,3$, and the mean is $2.1$. Find $f$.
- 7 For $2,3,6$ compute the AM, GM and HM, and compare $G^2$ with the product of the AM and HM. Prove that $G^2=\bar x\,H$ for any two positive numbers.
- 8 For $3,7,8,9,10,11,12,13,14,30$, find the five-number summary, the IQR and the fences, and identify any outliers.
- 1 For $4,6,8,10,12$, find the variance (divisor $n$), the SD, the CV and the mean deviation about the mean.
- 2 If $\bar x=5$ and $s_x=4$, find the mean and SD of $y=3-2x$.
- 3 Prove that the SD (divisor $n$) of $1,2,\dots,n$ is $\sqrt{(n^2-1)/12}$.
- 4 The mean and SD of $25$ observations were computed as $40$ and $5$. It was then found that an observation $36$ had been copied as $63$. Find the correct mean and SD.
- 5 Ten observations have mean $50$ and SD $6$ (divisor $n$). Can one of them equal $70$?
- 6 The first four moments of a data set about $2$ are $1,6,20,90$. Find the mean, $m_2,m_3,m_4$, $\beta_1$ and $\beta_2$, and check Pearson's inequality.
- 7 A data set consists of $k$ zeros and $n-k$ ones ($0\lt k\lt n$), and $p=(n-k)/n$. Find $\gamma_1$ and $\beta_2$ in terms of $p$ and verify $\beta_2=\beta_1+1$.
- 8 Grouped data with class width $h=4$ give $\bar\mu_2=20$ and $\bar\mu_4=1000$. Apply Sheppard's corrections.
- 1 For $(x,y)=(1,2),(3,4),(4,4),(6,7),(8,9),(9,10)$, compute $r$, $b_{yx}$ and $b_{xy}$, and write both regression lines.
- 2 Show that for $x=-2,-1,0,1,2$ and $y=x^2$ the correlation is $0$. Find a relation $y=g(x)$ on the same $x$-values with $r=1$ and $g$ not linear, or explain why none exists.
- 3 Let $u=(x-10)/2$ and $v=(30-y)/5$. If $r_{uv}=0.6$ and $b_{vu}=0.8$, find $r_{xy}$ and $b_{yx}$.
- 4 The regression lines of a bivariate data set are $x+2y-7=0$ and $25x+8y-91=0$, and $s_x=5$. Identify the lines, and find the means, $r$ and $s_y$.
- 5 Using Theorem 1.12(4), show that the angle between the two regression lines increases as $|r|$ decreases (for fixed $s_x,s_y$).
- 6 Prove that if $b_{yx}\gt1$ then $b_{xy}\lt1$, and that $\tfrac12(b_{yx}+b_{xy})\ge r$ when $r\gt0$.
- 7 Show that the residuals of the line of $y$ on $x$ have zero correlation with $x$ and with the fitted values $\hat y_i$.
- 8 Applied. In a city, the correlation between the daily maximum temperature (°C) and ice-cream sales (cups) is $0.82$, and the regression slope of sales on temperature is $12$ cups per °C. What are the correlation and the slope if the temperature is recorded in °F?
- 1 Prove $|r|\le1$ by expanding $\sum(z_i\pm w_i)^2\ge0$, where $z_i=(x_i-\bar x)/s_x$ and $w_i=(y_i-\bar y)/s_y$.
- 2 For nonnegative data with $\bar x\gt0$, prove that $\mathrm{CV}\le\sqrt{n-1}$, and find when equality holds.
- 3 Give data for which $\mathrm{MD}(\mathrm{Me})\lt\mathrm{MD}(\bar x)\lt s$ (all inequalities strict), and data for which all three are equal.
- 4 Prove that $\beta_2=1$ if and only if the data consist of two distinct values, each occurring equally often.
- 5 If $r_{xy}=r_{yz}=r$ with $\tfrac{1}{\sqrt2}\le r\le1$, prove that $r_{xz}\ge2r^2-1\ge0$.
- 6 Compute $r$ for the pooled data $(1,10),(2,11),(10,1),(11,2)$ and explain why each half has $r=1$.
- 7 Show that $\sum_{i\lt j}|x_i-x_j|\le\dfrac{n}{\sqrt2}\sqrt{\sum_{i\lt j}(x_i-x_j)^2}$, and deduce a bound for Gini's mean difference $\tfrac{1}{\binom n2}\sum_{i\lt j}|x_i-x_j|$ in terms of $s$.
E1. CSIR Part B GATE ST The variance, computed with divisor $n$, of the first $20$ natural numbers is (A) $33.25$; (B) $35$; (C) $36.75$; (D) $31.5$.
E2. CSIR Part B The two lines of regression of a bivariate data set are $9x-10y+35=0$ and $5x-2y-9=0$. The correlation coefficient is (A) $0.6$; (B) $-0.6$; (C) $0.36$; (D) $5/3$.
E3. CSIR Part B Let $g(a)=\sum_{i=1}^6|x_i-a|$ for the data $2,5,9,11,20,26$. Which is true? (A) $g$ has a unique minimiser, the mean $73/6$; (B) $g$ is minimised exactly on $[9,11]$ and its minimum value is $41$; (C) $g$ is minimised only at $a=10$, with minimum value $40$; (D) $g$ is minimised exactly on $[5,20]$.
E4. CSIR Part C Which of the following statements are true? (One or more options may be correct.) (A) If $r_{xy}=0.8$, then the correlation between $u=2x+3$ and $v=5-y$ is $-0.8$. (B) A data set can have regression coefficients $b_{yx}=1.6$ and $b_{xy}=0.7$. (C) If $r_{xy}=r_{yz}=0.9$, then $r_{xz}\gt0$. (D) If $r_{xy}=0$, then $y$ is not a function of $x$ on the data.
E5. CSIR Part C GATE ST For data with positive variance, which of the following are true? (A) $\beta_2\ge1$ always. (B) $\beta_2\ge3$ whenever the data have a single mode. (C) If $\gamma_1=0$, then the mean equals the median. (D) If the data take exactly two distinct values, then $\beta_2=\beta_1+1$.
E6. CSIR Part C A data set has $20$ distinct observations. Its largest observation is increased by $100$. Which of the following are necessarily unchanged? (A) the median; (B) the $10\%$ trimmed mean; (C) the standard deviation; (D) the mean deviation about the median.
Suggested plan: 3 lectures × 60 min.
- Looking at data. Variable types (Definition 1.1) with students' own examples. Build Example 1 on the board: the stem-and-leaf display first, then the table, then the histogram drawn twice (wrong heights, then densities). Theorems 1.1 and 1.2 with their proofs: the slope argument for the median is short and memorable. Theorem 1.3. Example 2 for grouped data.
- Spread and shape. Box plots and fences with Example 3. Project Interactive 1 on the clinic data and drag the largest value: ask the class to predict which readouts will move before dragging. Theorems 1.4–1.6 (prove Samuelson's inequality; students rarely see it). Example 4, then moments, Theorem 1.7, Example 5, Pearson's inequality (Theorem 1.8) and a brief Sheppard sketch.
- Two variables. Theorem 1.10 as "$r$ is a cosine", which makes Example 6(c, d) a picture rather than a calculation. Theorems 1.11–1.12, the strategy box, Example 7. Project Interactive 2: sweep $\rho$ to watch the scissors close, then show the curved cloud and the outlier. Finish with E2 and E4 live.
Discussion questions. (1) A newspaper reports that "average household income rose by 8%" while the median fell. Is this possible, and what would it suggest about the distribution? (2) Why does the least-squares line of $y$ on $x$ differ from that of $x$ on $y$? Which would a doctor predicting weight from height want? (3) Is there a data set whose box plot shows an outlier on each side while its mean equals its median? Construct one.
Misconceptions to watch for. "$r=0$ means independent / unrelated"; "mean $\gt$ median proves positive skewness"; "the regression line of $x$ on $y$ is the inverse of that of $y$ on $x$"; "pooled SD lies between the group SDs"; using frequencies rather than densities for unequal classes; applying Sheppard's correction to discrete data; mixing the divisors $n$ and $n-1$.
Suggested homework. I(2, 4, 5, 8), II(3, 4, 5, 6), III(1, 3, 4, 8), IV(2, 3, 5); Exam corner E1, E3, E5, E6.
Research question. How much bad data can a summary survive, and what does that robustness cost when the data are clean?
Why it matters. Air-quality networks, hospital records and sensor logs routinely contain gross errors: a stuck sensor reading 9999, a weight typed in grams instead of kilograms. Agencies that publish daily pollution indices, and clinical trials that report "typical" responses, must choose summaries that do not explode on such errors, yet do not throw away information when the data are clean. Robust statistics, founded by Tukey, Huber and Hampel, studies exactly this trade-off.
What to do.
- Take a real data set of 30–100 numbers. A month of daily PM2.5 readings for a city from a public monitoring portal works well, as does a list of hospital waiting times. Compute the mean, median, $10\%$ and $25\%$ trimmed means, SD, IQR and MAD.
- Replace $k=1,2,3,\dots$ observations by $10^6$ and record each summary. Find the smallest $k$ at which each one "breaks down" (becomes absurd), and express it as a fraction of $n$: the breakdown point.
- Now simulate clean data: $2000$ samples of size $n=25$ from $N(0,1)$, generated with a seeded random number generator and the Box–Muller transform. Estimate the variance of the mean, the median and the trimmed means across samples. The ratio (variance of the mean)/(variance of the estimator) is the estimator's efficiency.
- Repeat with contaminated data: each observation comes from $N(0,1)$ with probability $0.95$ and from $N(0,100)$ with probability $0.05$. Which estimator is best now?
Tools: Python (NumPy) or R, or a spreadsheet for steps 1–2.
Hand in: A one-page table of breakdown points and efficiencies under both models, and a recommendation (with reasons) for the summary a pollution agency should publish.
Going further. Read about Huber's M-estimator, which minimises $\sum\rho(x_i-c)$ for a function $\rho$ that is quadratic near $0$ and linear far out. This interpolates between Theorem 1.1 (the mean) and Theorem 1.2 (the median).
🧪 Self-check quiz
Q1. The value of $c$ that minimises $\sum_{i=1}^n|x_i-c|$ is
Q2. The regression coefficients of a data set are $b_{yx}=-0.9$ and $b_{xy}=-0.4$. Then $r$ equals
Q3. Two groups have sizes $10$ and $10$, means $20$ and $30$, and SDs $4$ and $4$ (divisor $n$). The SD of the combined data is
Q4. For every data set with positive variance, the kurtosis $\beta_2$ satisfies
Q5. Which summary is unchanged when the largest of $20$ distinct observations is multiplied by $10$?