Chapter 8 · Sample Surveys and Design of Experiments

8.2 PPS Sampling; Ratio and Regression Estimators

When units differ in size, or a cheap auxiliary variable is known for every unit, equal-probability sampling wastes information. Sampling with probability proportional to size (Hansen–Hurwitz, Horvitz–Thompson) and the ratio and regression estimators put that information to work.

Suppose we want the total milk production of the $250$ villages of a district. The villages differ enormously: a few large ones have thousands of cattle, many small ones a few dozen. From the last livestock census we know, for every village, the number of households, a size measure roughly proportional to what we want. Section 8.1 would draw villages with equal probabilities and multiply the sample mean by $N$. Its error would then be dominated by the luck of catching or missing the few big villages. Two better ideas use the size measure. The first is to select big villages with higher probability and divide each observation by its selection probability; this is probability proportional to size (PPS) sampling. The second is to keep the simple random sample but adjust the estimate: if the sampled villages turn out to be larger than average, scale the estimate down. These are the ratio and regression estimators.

Both ideas recur throughout survey practice: crop-cutting surveys with area as the auxiliary variable, business surveys with last year's turnover, forest inventories with basal area, household surveys with census counts. Both have characteristic failure modes, and CSIR NET loves those. An unbiased estimator can be absurd (Basu's elephants). A variance estimator can be negative. A positive correlation is not enough for the ratio estimator to help. This section develops the exact theory (unbiasedness and variance under the design) and the first-order theory (bias and mean squared error of the ratio and regression estimators), and says precisely when each method wins.

PPS Sampling with Replacement: the Hansen–Hurwitz Estimator

We keep the notation of §8.1: the population is $U=\{1,\dots,N\}$ with values $Y_i$, total $Y$ and mean $\bar Y$. In addition, a positive auxiliary (size) variable $X_i\gt0$ is known for every unit before sampling, with total $X=\sum X_i$ and mean $\bar X$.

📘 Definition 8.5 — PPS sampling with replacement and the Hansen–Hurwitz estimator

Let $p_i=X_i/X$ (the normed size, $\sum p_i=1$). In PPS sampling with replacement (PPSWR) we make $n$ independent draws, each selecting unit $i$ with probability $p_i$. If the $j$-th draw selects a unit with value $y_j$ and normed size $p_j'$, the Hansen–Hurwitz estimator of the total is

$$\hat Y_{\mathrm{HH}}=\frac1n\sum_{j=1}^n\frac{y_j}{p_j'} .$$

A unit drawn twice is counted twice. The mean is estimated by $\hat Y_{\mathrm{HH}}/N$.

📐 Theorem 8.12 — Properties of the Hansen–Hurwitz estimator

Under PPSWR with $n$ draws:

  1. $\hat Y_{\mathrm{HH}}$ is unbiased for $Y$;
  2. its variance is $$\operatorname{Var}(\hat Y_{\mathrm{HH}})=\frac1n\sum_{i=1}^Np_i\Big(\frac{Y_i}{p_i}-Y\Big)^2=\frac1n\Big(\sum_{i=1}^N\frac{Y_i^2}{p_i}-Y^2\Big);$$
  3. for $n\ge2$, with $z_j=y_j/p_j'$, an unbiased estimator of this variance is $$v(\hat Y_{\mathrm{HH}})=\frac{1}{n(n-1)}\sum_{j=1}^n\big(z_j-\hat Y_{\mathrm{HH}}\big)^2 ;$$
  4. the variance is zero if and only if $Y_i/X_i$ is the same for every unit, i.e. $Y_i\propto X_i$.

Proof. Each draw produces the random variable $z=Y_I/p_I$, where $P(I=i)=p_i$, and the $n$ values $z_1,\dots,z_n$ are iid. Now $E z=\sum_ip_i\frac{Y_i}{p_i}=Y$ and $\operatorname{Var}z=\sum_ip_i\left(\frac{Y_i}{p_i}-Y\right)^2$. Expanding the square gives $\sum Y_i^2/p_i-2Y^2+Y^2$. So $\hat Y_{\mathrm{HH}}=\bar z$ is the mean of an iid sample, which gives (1) and (2), and the sample variance of the $z_j$ is unbiased for $\operatorname{Var}z$ (§4.1), which gives (3). For (4), $\sum p_i(Y_i/p_i-Y)^2=0$ iff $Y_i/p_i=Y$ for every $i$ (all $p_i\gt0$), i.e. $Y_i=Yp_i=(Y/X)X_i$. ∎

Part (4) is the whole idea. PPS turns the estimation of $Y$ into the estimation of the mean of the ratios $Y_i/p_i$. These are nearly constant when $Y$ is nearly proportional to the size, so the estimator is very precise. With $p_i=1/N$ the estimator reduces to $N\bar y$ under SRSWR, and Theorem 8.12 reproduces Theorem 8.3.

How to draw units with probability proportional to $X_i$. When the sizes $X_i$ are integers there are two classical schemes.

📘 Definition 8.6 — Cumulative total method and Lahiri's method
  • Cumulative total method. Form the cumulative totals $T_i=X_1+\dots+X_i$ ($T_0=0$, $T_N=X$). Draw a random integer $R$ uniformly from $\{1,\dots,X\}$ and select the unit $i$ with $T_{i-1}\lt R\le T_i$. Each unit owns $X_i$ of the $X$ numbers, so it is selected with probability $X_i/X$.
  • Lahiri's method (1951). Let $M\ge\max_iX_i$. Draw a pair $(i,R)$ with $i$ uniform on $\{1,\dots,N\}$ and $R$ uniform on $\{1,\dots,M\}$, independently. If $R\le X_i$, select unit $i$; otherwise reject the pair and draw a new one. Repeat until a unit is selected.

The cumulative total method needs all cumulative totals to be computed and listed, which is tedious for large $N$. Lahiri's method needs only the individual sizes and an upper bound $M$.

📐 Theorem 8.13 — Lahiri's method selects with probability proportional to size

In Lahiri's method each trial selects some unit with probability $\theta=\frac{X}{NM}$. The selected unit is $i$ with probability exactly $X_i/X$, and the number of trials needed has the geometric distribution with mean $NM/X=M/\bar X$.

Proof. In a single trial, $P(\text{unit }i\text{ chosen and accepted})=\frac1N\cdot\frac{X_i}{M}$, because $R\le X_i\le M$ has probability $X_i/M$. Summing over $i$, a trial succeeds with probability $\theta=\frac{X}{NM}\gt0$. Trials are independent, so the first success occurs at trial $t$ with probability $(1-\theta)^{t-1}\theta$, a geometric law with mean $1/\theta$. The selected unit is the one accepted at the first successful trial, and by independence $$P(\text{selected unit}=i)=\sum_{t\ge1}(1-\theta)^{t-1}\cdot\frac{X_i}{NM}=\frac{X_i/(NM)}{\theta}=\frac{X_i}{X}.\ ∎$$

Choosing $M=\max X_i$ minimises the expected number of rejections. If one uses an $M$ that is smaller than some $X_i$, those units are under-selected and the method is wrong.

✏️ Example 1 — Cumulative totals and the Hansen–Hurwitz estimate
Eight villages have the numbers of households $X_i=40,25,60,15,30,50,20,60$ (total $300$). (a) Using the random numbers $112$, $287$, $39$ from $1$–$300$, select a PPSWR sample of three villages. (b) The selected villages report $150$, $132$ and $104$ (in that order) hectares under irrigation. Estimate the district total with its standard error.
Cumulative totals $40,65,125,140,170,220,240,300$. The number $112$ lies in $(65,125]$, giving village $3$; $287\in(240,300]$ gives village $8$; $39\in(0,40]$ gives village $1$.
Ratios $p_3=p_8=60/300=0.2$ and $p_1=40/300=\frac2{15}$. So $z=150/0.2=750$, $132/0.2=660$, $104\cdot\frac{15}{2}=780$.
Estimate $\hat Y_{\mathrm{HH}}=\frac{750+660+780}{3}=730$ hectares. The deviations are $20,-70,50$, so $$v=\frac{400+4900+2500}{3\cdot2}=1300$$ and the standard error is $\sqrt{1300}=36.1$ hectares.
Comment The three ratios $z_j$ (irrigated hectares per household, times $300$) are close to each other. That closeness is why the standard error is only $5\%$ of the estimate from three villages.
✏️ Example 2 — Lahiri's method in action
For the villages of Example 1 use Lahiri's method with $M=60$. Process the pairs $(i,R)=(3,47),(4,22),(7,18),(2,31),(5,30)$ in order and report the selected villages. What is the expected number of pairs needed per selected village?
Pairs $(3,47)$: $X_3=60\ge47$, accept village 3. $(4,22)$: $X_4=15\lt22$, reject. $(7,18)$: $X_7=20\ge18$, accept village 7. $(2,31)$: $X_2=25\lt31$, reject. $(5,30)$: $X_5=30\ge30$, accept village 5.
Sample Villages $3,7,5$.
Efficiency $\theta=\frac{X}{NM}=\frac{300}{8\cdot60}=0.625$, so on average $1/\theta=1.6$ pairs per selected unit, and $37.5\%$ of the pairs are rejected. Here $5$ pairs gave $3$ units.

Sampling without Replacement: Inclusion Probabilities and the Horvitz–Thompson Estimator

Sampling with replacement wastes effort on repeated units. For designs without replacement the natural tool is the inclusion probability $\pi_i=P(i\in s)$, which we met in Definition 8.1. Horvitz and Thompson (1952) observed that weighting each sampled value by $1/\pi_i$ gives an unbiased estimator of the total for any design.

📘 Definition 8.7 — Horvitz–Thompson estimator

Let $p(\cdot)$ be a sampling design without replacement, with first- and second-order inclusion probabilities $\pi_i$ and $\pi_{ij}$ ($i\ne j$), and let $Z_i=\mathbf 1\{i\in s\}$. If every $\pi_i\gt0$, the Horvitz–Thompson (HT) estimator of $Y$ is

$$\hat Y_{\mathrm{HT}}=\sum_{i\in s}\frac{y_i}{\pi_i}=\sum_{i=1}^NZ_i\frac{Y_i}{\pi_i}.$$

The design has fixed size $n$ if every possible sample has exactly $n$ units. A fixed-size design with $\pi_i=nX_i/X$ is called a $\pi$ps design.

📐 Theorem 8.14 — Identities for inclusion probabilities

For any design without replacement, $\sum_i\pi_i=E[n(s)]$, the expected sample size. If the design has fixed size $n$, then for every $i$

$$\sum_{i=1}^N\pi_i=n,\qquad \sum_{j\ne i}\pi_{ij}=(n-1)\pi_i,\qquad \sum_i\sum_{j\ne i}\pi_{ij}=n(n-1).$$

Proof. $n(s)=\sum_iZ_i$, so $E[n(s)]=\sum_iE Z_i=\sum\pi_i$. For a fixed-size design $\sum_{j\ne i}Z_j=n-Z_i$, so $Z_i\sum_{j\ne i}Z_j=Z_i(n-1)$ (using $Z_i^2=Z_i$). Taking expectations gives $\sum_{j\ne i}\pi_{ij}=(n-1)\pi_i$, and summing over $i$ gives $n(n-1)$. ∎

📐 Theorem 8.15 — The Horvitz–Thompson theorem

Let every $\pi_i\gt0$.

  1. $\hat Y_{\mathrm{HT}}$ is unbiased for $Y$, for every population $(Y_1,\dots,Y_N)$.
  2. Its variance is $$\operatorname{Var}(\hat Y_{\mathrm{HT}})=\sum_{i}\frac{1-\pi_i}{\pi_i}Y_i^2+\sum_{i\ne j}\frac{\pi_{ij}-\pi_i\pi_j}{\pi_i\pi_j}Y_iY_j .$$
  3. (Sen–Yates–Grundy form.) If the design has fixed size $n$, $$\operatorname{Var}(\hat Y_{\mathrm{HT}})=\frac12\sum_{i\ne j}\big(\pi_i\pi_j-\pi_{ij}\big)\Big(\frac{Y_i}{\pi_i}-\frac{Y_j}{\pi_j}\Big)^2 .$$ In particular the variance is $0$ when $\pi_i\propto Y_i$.
  4. If every $\pi_{ij}\gt0$, the following are unbiased estimators of the variance (the second for fixed-size designs): $$v_{\mathrm{HT}}=\sum_{i\in s}\frac{1-\pi_i}{\pi_i^2}y_i^2+\sum_{\substack{i,j\in s\\ i\ne j}}\frac{\pi_{ij}-\pi_i\pi_j}{\pi_{ij}\,\pi_i\pi_j}y_iy_j,$$ $$v_{\mathrm{SYG}}=\sum_{\substack{i,j\in s\\ i\lt j}}\frac{\pi_i\pi_j-\pi_{ij}}{\pi_{ij}}\Big(\frac{y_i}{\pi_i}-\frac{y_j}{\pi_j}\Big)^2 .$$

Proof. Put $a_i=Y_i/\pi_i$, so $\hat Y_{\mathrm{HT}}=\sum_iZ_ia_i$ with $E Z_i=\pi_i$, $\operatorname{Var}Z_i=\pi_i(1-\pi_i)$ and $\operatorname{Cov}(Z_i,Z_j)=\pi_{ij}-\pi_i\pi_j$. (1) $E\hat Y_{\mathrm{HT}}=\sum\pi_ia_i=\sum Y_i$. (2) The variance of a linear combination of the $Z_i$ is $$\sum_ia_i^2\pi_i(1-\pi_i)+\sum_{i\ne j}a_ia_j(\pi_{ij}-\pi_i\pi_j),$$ which is the stated formula.

(3) Expand $(a_i-a_j)^2=a_i^2+a_j^2-2a_ia_j$. By symmetry in $i,j$, $$\tfrac12\sum_{i\ne j}(\pi_i\pi_j-\pi_{ij})(a_i-a_j)^2=\sum_ia_i^2\sum_{j\ne i}(\pi_i\pi_j-\pi_{ij})-\sum_{i\ne j}(\pi_i\pi_j-\pi_{ij})a_ia_j .$$ By Theorem 8.14 the inner sum is $\pi_i(n-\pi_i)-(n-1)\pi_i=\pi_i(1-\pi_i)$. So the first term is $\sum a_i^2\pi_i(1-\pi_i)$, and the second is $\sum_{i\ne j}(\pi_{ij}-\pi_i\pi_j)a_ia_j$; together they give (2). If $\pi_i=cY_i$, all $a_i$ are equal and every term vanishes.

(4) For any numbers $g_i$ and $h_{ij}$, $E\sum_{i\in s}g_i=\sum_i\pi_ig_i$ and $E\sum_{i\ne j\in s}h_{ij}=\sum_{i\ne j}\pi_{ij}h_{ij}$. Dividing each term of (2) or (3) by the probability that it is observed ($\pi_i$ or $\pi_{ij}$) therefore gives unbiased estimators. ∎

Two remarks. First, the HT estimator is unbiased only if every $\pi_i\gt0$: a unit that can never be sampled contributes nothing, and its value is invisible to every estimator. Second, unbiased variance estimation needs $\pi_{ij}\gt0$ for all pairs. Systematic sampling (§8.1) fails this. For SRSWOR, $\pi_i=n/N$ and $\hat Y_{\mathrm{HT}}=N\bar y$, and Theorem 8.15 reproduces Theorem 8.2 (Exercise II.3). The condition $\pi_i\pi_j\ge\pi_{ij}$ for all pairs (the Sen–Yates–Grundy condition) guarantees $v_{\mathrm{SYG}}\ge0$. It holds for SRSWOR and for the standard $\pi$ps schemes, but not for every design.

Draw-by-draw PPS without replacement. The simplest scheme draws the first unit with probabilities $p_i$, the second with probabilities $p_j/(1-p_i)$ among the remaining units, and so on. For $n=2$,

$$\pi_{ij}=p_ip_j\Big(\frac1{1-p_i}+\frac1{1-p_j}\Big),\qquad \pi_i=p_i\Big(1+\sum_{j\ne i}\frac{p_j}{1-p_j}\Big).$$

These $\pi_i$ are not equal to $2p_i$ in general. Using $y_i/(2p_i)$ as if they were is a classic error.

✏️ Example 3 — The HT estimator under draw-by-draw PPS ($N=4$, $n=2$)
A population has normed sizes $p=(0.1,0.2,0.3,0.4)$ and values $Y=(6,10,17,21)$, so $Y=54$. Two units are drawn by draw-by-draw PPS without replacement. Find all $\pi_i$ and $\pi_{ij}$, list the HT estimate for every sample, and verify unbiasedness and the Sen–Yates–Grundy variance. Compare with the Hansen–Hurwitz estimator ($n=2$ draws with replacement) and with $N\bar y$ under SRSWOR.
Sample probabilities $P\{i,j\}=\pi_{ij}=p_ip_j\left(\frac1{1-p_i}+\frac1{1-p_j}\right)$. For example, $$\pi_{12}=0.02\left(\tfrac1{0.9}+\tfrac1{0.8}\right)=0.04722.$$ In all: $\pi_{12}=0.04722$, $\pi_{13}=0.07619$, $\pi_{14}=0.11111$, $\pi_{23}=0.16071$, $\pi_{24}=0.23333$, $\pi_{34}=0.37143$. These sum to $1$.
First order $\pi_i=\sum_{j\ne i}\pi_{ij}$ (Theorem 8.14 with $n=2$): $\pi=(0.23452,\ 0.44127,\ 0.60833,\ 0.71587)$, sum $2$ ✓. Compare $2p=(0.2,0.4,0.6,0.8)$: the small units are over-represented and the largest is under-represented.
HT values For samples $\{1,2\},\{1,3\},\{1,4\},\{2,3\},\{2,4\},\{3,4\}$: $48.25$, $53.53$, $54.92$, $50.61$, $52.00$, $57.28$. Weighted by the $\pi_{ij}$ they average exactly $54$ ✓. The variance computed directly is $8.457$, and the SYG formula gives the same $8.457$ ✓.
Comparison Hansen–Hurwitz with $2$ draws: $\frac12\sum p_i(Y_i/p_i-54)^2=4.917$. SRSWOR: $N^2(1-f)S^2/n=182.7$. Both PPS methods are far better than equal-probability sampling. Here HH even beats HT, because $Y_i/p_i=(60,50,56.7,52.5)$ is nearly constant, while $Y_i/\pi_i=(25.6,22.7,27.9,29.3)$ is less so: the draw-by-draw $\pi_i$ are not proportional to size. A genuine $\pi$ps scheme (Sampford's, or Durbin's) or Des Raj's ordered estimator (Exercise IV.2) avoids this loss.
✏️ Example 4 — Prove or disprove: "an unbiased estimator of a total is always sensible" (Basu's elephants)
A circus owner must estimate the total weight of his $50$ elephants by weighing one. From last year he knows that Sambo is of average weight. The circus statistician insists on a probability design: Sambo is selected with probability $\pi_{\mathrm{Sambo}}=0.99$ and each of the other $49$ elephants with probability $0.01/49=1/4900$. What does the HT estimator give? Is it unbiased? Is it sensible?
Estimates If Sambo (say $4000$ kg) is selected, $\hat Y_{\mathrm{HT}}=4000/0.99=4040$ kg for fifty elephants. If another elephant of weight $w$ is selected, $\hat Y_{\mathrm{HT}}=4900w$; for Jumbo, weighing $6000$ kg, that is $29\,400\,000$ kg.
Unbiased Yes. Every $\pi_i\gt0$, so Theorem 8.15(1) applies: $0.99\cdot\frac{Y_{\mathrm{Sambo}}}{0.99}+\sum\frac1{4900}\cdot4900w_i=Y$.
Sensible No. The estimate is about $1/50$ of the truth with probability $0.99$, and enormous otherwise. Its variance is huge because $\pi_i$ is nowhere near proportional to $Y_i$ (Theorem 8.15(3)). The owner's natural estimate, $50\times4000$, uses the design-free knowledge that Sambo is typical.
Moral The statement is false. Unbiasedness alone guarantees nothing. The HT estimator is good only when $\pi_i$ is roughly proportional to $Y_i$. Basu's 1971 parable started a long debate on design-based versus model-based inference in finite populations.
Interactive 1 — Proportional to Size, or Equal Chances?

A population of $N=20$ units with known sizes $X_i$ (left: each unit at $(X_i,Y_i)$; the ringed units are the current PPS draws). Each step makes $n$ draws with replacement in two ways: PPS with $p_i=X_i/X$ (Hansen–Hurwitz $\hat Y_{\mathrm{HH}}$) and equal probabilities ($N\bar y$). The histograms collect the estimates of the total, and the curves are the normal laws with the exact variances of Theorem 8.12. Change how $Y$ depends on $X$: PPS is excellent when $Y\propto X$, useless when $Y$ is unrelated to $X$, and harmful when $Y$ falls as $X$ grows.

4
true total Y—
SE of Ŷ_HH (exact)—
SE of N ȳ (exact)—
efficiency of PPS—

The Ratio Estimator

Now return to simple random sampling, but observe the auxiliary variable on each sampled unit together with $y$. If the sample happens to contain unusually large units, then $\bar x\gt\bar X$, and $\bar y$ is probably too large as well. The ratio estimator corrects for this.

📘 Definition 8.8 — Ratio estimator

Let $(y_i,x_i)$, $i=1,\dots,n$, be an SRSWOR from the population of pairs $(Y_i,X_i)$ with $X_i\gt0$ and $\bar X$ known. Write $R=\bar Y/\bar X=Y/X$ for the population ratio, $S_x^2,S_y^2$ for the population variances, $S_{xy}=\frac1{N-1}\sum(Y_i-\bar Y)(X_i-\bar X)$, $\rho=S_{xy}/(S_xS_y)$, and $C_x=S_x/\bar X$, $C_y=S_y/\bar Y$ for the coefficients of variation. The ratio estimators of $R$, $\bar Y$ and $Y$ are

$$\hat R=\frac{\bar y}{\bar x},\qquad \bar y_R=\hat R\,\bar X=\frac{\bar y}{\bar x}\bar X,\qquad \hat Y_R=\hat R\,X .$$
📐 Theorem 8.16 — Exact bias of the ratio estimator (Hartley–Ross bound)

Under SRSWOR,

$$E[\hat R]-R=-\frac{\operatorname{Cov}(\hat R,\bar x)}{\bar X},$$

and consequently

$$\frac{|E[\bar y_R]-\bar Y|}{\sqrt{\operatorname{Var}(\bar y_R)}}\le\operatorname{CV}(\bar x)=\sqrt{\frac{1-f}{n}}\;C_x .$$

So the bias is negligible compared with the standard error whenever the coefficient of variation of $\bar x$ is small.

Proof. Since $\hat R\bar x=\bar y$, we have $E[\hat R\bar x]=\bar Y=R\bar X$. Therefore $$\operatorname{Cov}(\hat R,\bar x)=E[\hat R\bar x]-E[\hat R]\,\bar X=\bar X\big(R-E\hat R\big).$$ By the Cauchy–Schwarz inequality, $|\operatorname{Cov}(\hat R,\bar x)|\le\sqrt{\operatorname{Var}\hat R}\sqrt{\operatorname{Var}\bar x}$. Hence $|E\hat R-R|\le\sqrt{\operatorname{Var}\hat R}\cdot\sqrt{\operatorname{Var}\bar x}/\bar X$, and $\sqrt{\operatorname{Var}\bar x}/\bar X=\sqrt{(1-f)/n}\,C_x$ by Theorem 8.2. Multiplying $\hat R$ by the constant $\bar X$ multiplies the bias and the standard deviation by the same factor. ∎

📐 Theorem 8.17 — First-order bias and MSE of the ratio estimator

Under SRSWOR, to first order in $1/n$ (that is, neglecting terms of order $n^{-2}$),

$$E[\bar y_R]-\bar Y\approx\frac{1-f}{n}\,\bar Y\big(C_x^2-\rho\,C_xC_y\big),$$ $$\operatorname{MSE}(\bar y_R)\approx\frac{1-f}{n}\big(S_y^2+R^2S_x^2-2R\rho S_xS_y\big)=\frac{1-f}{n}\cdot\frac{1}{N-1}\sum_{i=1}^N\big(Y_i-RX_i\big)^2 .$$

Consequently $\bar y_R$ is more precise than $\bar y$ (to this order) if and only if

$$\rho\gt\frac{C_x}{2C_y}=\frac{R\,S_x}{2S_y}.$$

The MSE is estimated by $v(\bar y_R)=\frac{1-f}{n}\cdot\frac1{n-1}\sum_{i=1}^n(y_i-\hat Rx_i)^2$.

Proof (careful sketch). Write $\bar y=\bar Y(1+\varepsilon_0)$ and $\bar x=\bar X(1+\varepsilon_1)$. Then $E\varepsilon_0=E\varepsilon_1=0$, and by Theorem 8.2 and Exercise IV.3 of §8.1, $$E\varepsilon_0^2=\tfrac{1-f}{n}C_y^2,\qquad E\varepsilon_1^2=\tfrac{1-f}{n}C_x^2,\qquad E\varepsilon_0\varepsilon_1=\tfrac{1-f}{n}\rho C_xC_y .$$ On the event $|\varepsilon_1|\lt1$, which has probability tending to $1$, expand $$\bar y_R=\bar Y(1+\varepsilon_0)(1+\varepsilon_1)^{-1}=\bar Y\big(1+\varepsilon_0-\varepsilon_1-\varepsilon_0\varepsilon_1+\varepsilon_1^2-\cdots\big).$$ The moments of $\varepsilon_0,\varepsilon_1$ of order $k$ are $O(n^{-k/2})$ (for SRSWOR as for iid sampling). Keeping the terms of order $1/n$: the bias is $\bar Y(E\varepsilon_1^2-E\varepsilon_0\varepsilon_1)$, and the MSE is $\bar Y^2E(\varepsilon_0-\varepsilon_1)^2=\bar Y^2\frac{1-f}{n}(C_y^2+C_x^2-2\rho C_xC_y)$. Since $\bar YC_y=S_y$ and $\bar YC_x=RS_x$, this is the stated form. The second form follows by expanding $\sum(Y_i-RX_i)^2=\sum[(Y_i-\bar Y)-R(X_i-\bar X)]^2$, using $\bar Y=R\bar X$. The MSE of $\bar y_R$ is less than $\frac{1-f}{n}S_y^2$ iff $R^2S_x^2\lt2R\rho S_xS_y$, i.e. $\rho\gt RS_x/(2S_y)=C_x/(2C_y)$ (for $R\gt0$). The estimator $v(\bar y_R)$ replaces $Y_i-RX_i$ by the sample residuals $y_i-\hat Rx_i$, in the same way as $s^2$ replaces $S^2$. ∎

A rigorous version of this argument controls the remainder on the event $|\varepsilon_1|\ge\frac12$ by a large-deviation bound. That is the content of the usual "for large $n$" qualification. In CSIR problems the first-order formulae are always intended.

When is the ratio estimator the right choice? The MSE involves $Y_i-RX_i$, the deviations from a line through the origin with slope $R$. The next theorem makes this precise: the ratio estimator is optimal under exactly such a model.

📐 Theorem 8.18 — The ratio estimator as a best linear unbiased predictor

Suppose that, given the $X_i$, the values follow the model $Y_i=\beta X_i+\varepsilon_i$, where the $\varepsilon_i$ are uncorrelated with $E\varepsilon_i=0$ and $\operatorname{Var}\varepsilon_i=\sigma^2X_i$. Then, for any fixed sample $s$, the BLUE of $\beta$ is $\hat\beta=\sum_{i\in s}y_i\big/\sum_{i\in s}x_i=\hat R$. The best linear unbiased predictor of the total $Y=\sum_{i\in s}y_i+\sum_{i\notin s}Y_i$ is $\hat RX$, the ratio estimator.

Proof. In matrix form, $y=x\beta+\varepsilon$ with $\operatorname{Cov}\varepsilon=\sigma^2V$, $V=\operatorname{diag}(x_i)$. By Aitken's generalised least squares theorem (§6.1), the BLUE is $$\hat\beta=(x^{\mathsf T}V^{-1}x)^{-1}x^{\mathsf T}V^{-1}y=\frac{\sum x_i\cdot y_i/x_i}{\sum x_i^2/x_i}=\frac{\sum y_i}{\sum x_i}.$$ The unobserved part $\sum_{i\notin s}Y_i$ has model mean $\beta\sum_{i\notin s}X_i$, and its errors are uncorrelated with the sample errors. So the best linear unbiased predictor replaces $\beta$ by $\hat\beta$ (Gauss–Markov applied to the linear function $\beta\sum_{i\notin s}X_i$). This gives $\sum_sy_i+\hat R(X-\sum_sx_i)=\hat RX$, because $\sum_sy_i=\hat R\sum_sx_i$. ∎

The ratio estimator is biased under SRSWOR. A clever change of design removes the bias altogether.

📐 Theorem 8.19 — Midzuno–Sen: an exactly unbiased ratio estimator

In the Midzuno scheme, the first unit is drawn with probability $p_i=X_i/X$ and the remaining $n-1$ units by SRSWOR from the other $N-1$. Then every sample $s$ of size $n$ has probability

$$P(s)=\frac{\sum_{i\in s}p_i}{\binom{N-1}{n-1}}=\frac{\bar x_s}{\bar X}\cdot\frac1{\binom Nn}.$$

Hence $\bar y_R=\frac{\bar y}{\bar x}\bar X$ is exactly unbiased for $\bar Y$, and $\pi_i=p_i+(1-p_i)\frac{n-1}{N-1}$.

Proof. The sample $s$ arises when the first draw is some $i\in s$ (probability $p_i$) and the remaining draws give exactly $s\setminus\{i\}$ (probability $1/\binom{N-1}{n-1}$). Summing over $i\in s$ gives the first expression. Since $\sum_{i\in s}p_i=n\bar x_s/(N\bar X)$ and $\binom Nn=\frac Nn\binom{N-1}{n-1}$, the second form follows. Then $$E[\bar y_R]=\sum_sP(s)\frac{\bar y_s}{\bar x_s}\bar X=\frac{1}{\binom Nn}\sum_s\bar y_s=\bar Y,$$ the last step being the unbiasedness of $\bar y$ under SRSWOR. Unit $i$ is included if it is drawn first (probability $p_i$), or if it is not drawn first and is then among the $n-1$ units chosen from $N-1$. ∎

The Regression Estimator and Comparisons

When the relation between $y$ and $x$ is a straight line that does not pass through the origin, the ratio adjustment over- or under-corrects. The remedy is to adjust along the regression line instead.

📘 Definition 8.9 — Difference and regression estimators

For a constant $b_0$, the difference-type estimator is $\bar y_{d}=\bar y+b_0(\bar X-\bar x)$; $b_0=1$ gives the difference estimator. The (linear) regression estimator uses the sample least squares slope $b=s_{xy}/s_x^2$:

$$\bar y_{lr}=\bar y+b\,(\bar X-\bar x),\qquad b=\frac{\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)}{\sum_{i=1}^n(x_i-\bar x)^2}.$$

Geometrically, $\bar y_{lr}$ is the height at $x=\bar X$ of the least squares line fitted to the sample (the simple regression of §6.3). Write $B=S_{xy}/S_x^2$ for the population slope.

📐 Theorem 8.20 — Variance of the regression estimator; the three estimators compared

Under SRSWOR:

  1. for fixed $b_0$, $\bar y_d$ is unbiased with $$\operatorname{Var}(\bar y_d)=\frac{1-f}{n}\big(S_y^2-2b_0S_{xy}+b_0^2S_x^2\big),$$ minimised at $b_0=B$, with minimum $\frac{1-f}{n}S_y^2(1-\rho^2)$;
  2. with the estimated slope $b$, $\bar y_{lr}$ has bias of order $1/n$, and to first order $$\operatorname{MSE}(\bar y_{lr})\approx\frac{1-f}{n}S_y^2(1-\rho^2);$$
  3. to first order, $V_{lr}\le\operatorname{MSE}(\bar y_R)$ and $V_{lr}\le\operatorname{Var}(\bar y)$, with $$\operatorname{Var}(\bar y)-V_{lr}=\frac{1-f}{n}\rho^2S_y^2,\qquad \operatorname{MSE}(\bar y_R)-V_{lr}=\frac{1-f}{n}\big(RS_x-\rho S_y\big)^2 .$$ The second difference is $0$ iff $B=R$, i.e. iff the population regression line passes through the origin.

The MSE of $\bar y_{lr}$ is estimated by $v(\bar y_{lr})=\frac{1-f}{n}\cdot\frac1{n-2}\sum_{i=1}^n\big[(y_i-\bar y)-b(x_i-\bar x)\big]^2$.

Proof. (1) $\bar y_d=b_0\bar X+\overline{(y-b_0x)}$ is a constant plus the sample mean of the variable $U_i=Y_i-b_0X_i$. By Theorem 8.2 it is unbiased for $b_0\bar X+\bar Y-b_0\bar X=\bar Y$, with variance $\frac{1-f}{n}S_U^2$, and $S_U^2=S_y^2-2b_0S_{xy}+b_0^2S_x^2$. This quadratic in $b_0$ is minimised at $b_0=S_{xy}/S_x^2=B$, with value $S_y^2-S_{xy}^2/S_x^2=S_y^2(1-\rho^2)$.

(2) Write $\bar y_{lr}-\bar Y=\big[(\bar y-\bar Y)-B(\bar x-\bar X)\big]-(b-B)(\bar x-\bar X)$. The bracket is $\bar y_d-\bar Y$ with $b_0=B$, which has mean $0$ and the variance of (1). The sample moments $s_{xy}$ and $s_x^2$ converge to $S_{xy}$ and $S_x^2$ with errors $O_p(n^{-1/2})$, so $b-B=O_p(n^{-1/2})$; also $\bar x-\bar X=O_p(n^{-1/2})$. Their product is $O_p(n^{-1})$. It contributes $O(1/n)$ to the bias and $O(n^{-3/2})$ to the MSE, both negligible at first order.

(3) The first difference is $\frac{1-f}{n}[S_y^2-S_y^2(1-\rho^2)]$. For the second, divide by $\frac{1-f}{n}$:

$$\big(S_y^2+R^2S_x^2-2R\rho S_xS_y\big)-S_y^2(1-\rho^2)=(RS_x-\rho S_y)^2 .$$

It vanishes iff $R=\rho S_y/S_x=B$. Since $\bar Y=R\bar X$ always, the line $y=\bar Y+B(x-\bar X)$ passes through the origin iff $B=R$. ∎

🧭 Problem-solving strategy — Which estimator for which population?
  • Size known for every unit before sampling, $y$ roughly proportional to size → select by PPS. Use Hansen–Hurwitz (with replacement), or Horvitz–Thompson with a $\pi$ps design (without replacement).
  • Auxiliary $x$ observed on the sample, $\bar X$ known, $y\approx\beta x$ (line through the origin, spread growing with $x$) → ratio estimator. It beats $\bar y$ iff $\rho\gt C_x/(2C_y)$. Check $\operatorname{CV}(\bar x)$ to make sure the bias is negligible.
  • Linear relation with an intercept → regression estimator: always at least as good as both, to first order. Use the difference estimator when the slope is known to be about $1$ (e.g. this year versus last year).
  • Negative correlation → the product estimator $\bar y\,\bar x/\bar X$ (Exercise III.6), or the regression estimator.
  • Small $n$ and unbiasedness required → the Midzuno scheme, or the Hartley–Ross estimator (Exercise IV.5).
✏️ Example 5 — Ratio and regression estimates from a farm survey
An SRSWOR of $n=10$ farms is drawn from $N=200$. For each farm, the area $x$ (hectares, known for all farms from the land register, $\bar X=46.5$) and this season's output $y$ (quintals) are recorded:
$x$30524160384755354448
$y$7810496120741081108486100
Estimate the mean output per farm by $\bar y$, $\bar y_R$ and $\bar y_{lr}$, with estimated standard errors, and comment.
Summaries $\bar x=45$, $\bar y=96$, $s_x^2=86.44$, $s_y^2=227.56$, $s_{xy}=127.33$; sample correlation $r=0.908$. The factor is $\frac{1-f}{n}=\frac{0.95}{10}=0.095$.
Mean per unit $\bar y=96$ with $v=0.095\times227.56=21.62$, SE $4.65$.
Ratio $\hat R=96/45=2.1333$ and $\bar y_R=2.1333\times46.5=99.20$. The residuals $y_i-\hat Rx_i$ have $\sum(y_i-\hat Rx_i)^2=699.16$, so $$v(\bar y_R)=0.095\times\frac{699.16}{9}=7.38,$$ SE $2.72$.
Regression $b=127.33/86.44=1.473$ and $\bar y_{lr}=96+1.473\times1.5=98.21$. The residual sum of squares is $359.93$, so $$v(\bar y_{lr})=0.095\times\frac{359.93}{8}=4.27,$$ SE $2.07$.
Comment Both estimates move up from $96$ because the sample farms are smaller than average ($\bar x=45\lt46.5$). The ratio estimate moves more, since it assumes a line through the origin with slope $2.13$, while the fitted slope is only $1.47$. The criterion $r=0.908\gt\hat C_x/(2\hat C_y)=0.657$ says that the ratio estimator beats $\bar y$. The fitted line has a clear intercept ($96-1.473\cdot45=29.7$), which is why the regression estimator is better still. The total output is estimated as $200\times98.21\approx19\,640$ quintals.
✏️ Example 6 — Prove or disprove: "if $\rho\gt0$, the ratio estimator beats the sample mean"
A population has $\bar Y=\bar X=50$ and $S_y=S_x=10$. Compare the first-order MSEs of $\bar y$, $\bar y_R$ and $\bar y_{lr}$ (in units of $\frac{1-f}{n}$) for $\rho=0.4$ and for $\rho=0.7$, and decide the statement.
Setup $R=1$ and $C_x=C_y=0.2$, so the threshold is $C_x/(2C_y)=0.5$.
$\rho=0.4$ $\operatorname{Var}\bar y=100$, $\operatorname{MSE}\bar y_R=100+100-2\cdot0.4\cdot100=120$, $V_{lr}=100(1-0.16)=84$. The ratio estimator is worse than the sample mean.
$\rho=0.7$ $\operatorname{MSE}\bar y_R=200-140=60\lt100$, and $V_{lr}=51$. Check Theorem 8.20(3): $(RS_x-\rho S_y)^2=(10-7)^2=9=60-51$ ✓.
Verdict False. The correlation must exceed $C_x/(2C_y)$, not $0$. When $x$ is relatively more variable than $y$, the ratio adjustment injects more noise than it removes.
✏️ Example 7 — The Midzuno scheme makes the ratio estimator unbiased
A population has $N=4$ units with $X=(2,3,5,10)$ and $Y=(5,7,9,24)$; $n=2$. Compute $E[\bar y_R]$ (a) under SRSWOR and (b) under the Midzuno scheme, and compare with $\bar Y$.
Population $\bar X=5$, $\bar Y=11.25$, $p=(0.1,0.15,0.25,0.5)$.
Sample values For $\{1,2\},\{1,3\},\{1,4\},\{2,3\},\{2,4\},\{3,4\}$: $\bar y_R=5\,\frac{\bar y}{\bar x}=12,\ 10,\ 12.083,\ 10,\ 11.923,\ 11$.
(a) SRSWOR Each sample has probability $\frac16$: $E\bar y_R=\frac{67.006}{6}=11.168\ne11.25$. The bias is $-0.082$.
(b) Midzuno $P(s)=(p_i+p_j)/\binom31$: $\frac{0.25}3,\frac{0.35}3,\frac{0.6}3,\frac{0.4}3,\frac{0.65}3,\frac{0.75}3$. Then $E\bar y_R=\frac13(0.25\cdot12+0.35\cdot10+0.6\cdot12.083+0.4\cdot10+0.65\cdot11.923+0.75\cdot11)=11.25$ exactly ✓, as Theorem 8.19 says.
Interactive 2 — Ratio or Regression? Adjusting the Sample Mean with x

A population of $N=400$ points $(X_i,Y_i)$ with $Y=a+2X+\text{noise}$ (grey). Each step draws an SRSWOR of $n=12$ (dots) and shows the ratio line $y=\hat Rx$ through the origin and the regression line. Each line is read off at $x=\bar X$ (dashed vertical) to give $\bar y_R$ and $\bar y_{lr}$, while $\bar y$ ignores $x$. The readouts give the first-order MSEs of Theorems 8.17 and 8.20 and the MSEs observed over the draws. Move the intercept $a$ away from $0$ and watch the ratio estimator lose. Raise the noise and watch it lose to $\bar y$ once $\rho\lt C_x/(2C_y)$.

012
ρ vs C_x/(2C_y)—
MSE ȳ: theory / observed—
MSE ȳ_R: theory / observed—
MSE ȳ_lr: theory / observed—
⚠️ Counterexamples and pitfalls
  • Unbiased does not mean sensible. Basu's elephants (Example 4): the HT estimator is unbiased but gives $4040$ kg for fifty elephants with probability $0.99$.
  • The HT estimator is not location-invariant. If every $Y_i=c$, then $\hat Y_{\mathrm{HT}}=c\sum_{i\in s}1/\pi_i$, which equals $Nc$ for every sample only if $\sum_{i\in s}1/\pi_i=N$ always. Under the design of Example 3 it ranges from $3.04c$ to $6.53c$ (Exercise II.8).
  • The SYG variance estimator can be negative. Take $N=4$, $n=2$, $P\{1,2\}=P\{3,4\}=0.4$ and probability $0.05$ for each other pair. Then every $\pi_i=\frac12$ and $\pi_{12}=0.4\gt\pi_1\pi_2$, so on $s=\{1,2\}$ we get $v_{\mathrm{SYG}}=-1.5(y_1-y_2)^2\lt0$ (E4). The estimator is unbiased on average, but it can be negative on a single sample.
  • PPS with an irrelevant size measure can be disastrous. If $Y$ is constant but sizes vary, SRS is exact while $\hat Y_{\mathrm{HH}}$ varies wildly. With $X=(1,2,3,10)$, $Y=(10,12,9,11)$ and $n=1$: $\operatorname{Var}\hat Y_{\mathrm{HH}}=1613.6$ against $20$ for $N\bar y$ (Exercise IV.3).
  • Draw-by-draw PPS without replacement does not give $\pi_i=np_i$ (Example 3). Using $\sum y_i/(np_i)$ as an "HT" estimator is then biased.
  • Lahiri's method with $M\lt\max X_i$ is wrong. A unit with $X_i\gt M$ is accepted with probability $1$ instead of $X_i/M$. It is therefore selected in proportion to $M$, not to $X_i$, so the largest units are under-selected.
  • Positive correlation is not enough for the ratio estimator. It needs $\rho\gt C_x/(2C_y)$ (Example 6, where $\rho=0.4$ makes it worse than $\bar y$).
  • The ratio and regression estimators are biased under SRSWOR. For the regression estimator take $N=3$, $n=2$ and the points $(1,1),(2,3),(4,4)$. The three samples give $\bar y_{lr}=\frac{11}3,\frac73,\frac{19}6$, with mean $3.056\ne\bar Y=2.667$. The bias is of order $1/n$ and is usually negligible, but not for tiny samples.
  • Ratio versus regression. The ratio estimator is optimal for a line through the origin (Theorem 8.18). With an intercept it can be beaten by a wide margin (Interactive 2, $a\ne0$), whereas the regression estimator is never worse to first order.

Exercises

I
PPS sampling with replacement.
  • 1 Seven shops have sizes (employees) $X=12,30,8,25,45,10,20$. Using the cumulative total method and the random numbers $37,140,66,101$ from $1$–$150$, select a PPSWR sample of four shops.
  • 2 The four shops selected in I.1 report monthly sales (thousand rupees) of $95$ (shop 2), $70$ (shop 7), $80$ (shop 4) and $140$ (shop 5). Compute $\hat Y_{\mathrm{HH}}$ and its estimated standard error.
  • 3 For the shops of I.1, apply Lahiri's method with $M=45$ to the pairs $(3,12),(6,9),(1,40),(5,44),(2,31),(4,25)$.
  • 4 In I.3, find the probability that a pair is rejected and the expected number of pairs needed per selected unit.
  • 5 Prove that $\operatorname{Var}(\hat Y_{\mathrm{HH}})=\frac1n\left(\sum Y_i^2/p_i-Y^2\right)$, and that by Cauchy–Schwarz $\sum Y_i^2/p_i\ge Y^2$ for all $p$ with equality iff $p_i\propto Y_i$ (for $Y_i\gt0$).
  • 6 A population has $X=(1,2,3,4)$ and $Y=(3,5,9,11)$. For $n=2$ draws, compare $\operatorname{Var}(\hat Y_{\mathrm{HH}})$ with $\operatorname{Var}(N\bar y)$ under SRSWR and under SRSWOR.
  • 7 Show that PPSWR with all $X_i$ equal reduces Theorem 8.12 to Theorem 8.3.
  • 8 (Applied.) A health department wants the number of children under five in a district of $120$ villages. Population figures from the last census are known for each village. Describe a PPSWR design, the estimator, how you would compute its standard error, and one situation in which the design could perform badly.
1 The cumulative totals are $12,42,50,75,120,130,150$. So $37\to$ shop 2, $140\to$ shop 7, $66\to$ shop 4 and $101\to$ shop 5.
2 $p=0.2,\ 0.1333,\ 0.1667,\ 0.3$, so $z=475,\ 525,\ 480,\ 466.67$ and $\hat Y_{\mathrm{HH}}=486.67$. The variance estimate is $v=\frac{\sum(z_j-\bar z)^2}{4\cdot3}=170.8$, so SE $=13.07$ thousand rupees.
3 $(3,12)$: $8\lt12$, reject. $(6,9)$: $10\ge9$, accept shop 6. $(1,40)$: reject. $(5,44)$: $45\ge44$, accept shop 5. $(2,31)$: $30\lt31$, reject. $(4,25)$: $25\ge25$, accept shop 4. The sample is shops $6,5,4$.
4 $\theta=\frac{150}{7\cdot45}=0.476$, so a pair is rejected with probability $0.524$, and on average $1/\theta=2.1$ pairs are needed per selected unit.
5 Expand $\sum p_i(Y_i/p_i-Y)^2=\sum Y_i^2/p_i-2Y\sum Y_i+Y^2\sum p_i$. By Cauchy–Schwarz, $Y^2=\left(\sum\frac{Y_i}{\sqrt{p_i}}\sqrt{p_i}\right)^2\le\sum\frac{Y_i^2}{p_i}\sum p_i$, with equality iff $Y_i/\sqrt{p_i}\propto\sqrt{p_i}$, i.e. $p_i\propto Y_i$.
6 $p=(0.1,0.2,0.3,0.4)$ and $Y=28$: $\operatorname{Var}\hat Y_{\mathrm{HH}}=\frac12(\frac{9}{0.1}+\frac{25}{0.2}+\frac{81}{0.3}+\frac{121}{0.4}-784)=1.75$. Next $\sigma^2=10$, so SRSWR gives $16\cdot10/2=80$, and SRSWOR gives $16\cdot\frac12\cdot\frac{40}3\cdot\frac12=53.3$. PPS is about $30$ times as efficient.
7 $p_i=1/N$ gives $\hat Y_{\mathrm{HH}}=N\bar y$ and $\frac1n\sum\frac1N(NY_i-Y)^2=\frac{N^2\sigma^2}{n}$, which is $N^2$ times Theorem 8.3.
8 Draw $n$ villages with probabilities $p_i=\text{census population}_i/\text{total}$ (cumulative totals or Lahiri). Count the under-fives $y_j$ in each, estimate $\frac1n\sum y_j/p_j$, and compute the SE from the spread of the $z_j=y_j/p_j$. It performs badly if the proportion of young children varies a lot between villages, for instance a new settlement of young families recorded with an old, small census population: its $p_i$ is tiny and its $z_j$ huge.
II
Inclusion probabilities and the Horvitz–Thompson estimator.
  • 1 For draw-by-draw PPS without replacement with $n=2$, derive $\pi_{ij}=p_ip_j\left(\frac1{1-p_i}+\frac1{1-p_j}\right)$ and the formula for $\pi_i$.
  • 2 With $N=3$ and $p=(0.2,0.3,0.5)$, compute all $\pi_i$ and $\pi_{ij}$ for II.1 and check both identities of Theorem 8.14.
  • 3 Show that for SRSWOR $\hat Y_{\mathrm{HT}}=N\bar y$, and that Theorem 8.15(2) reduces to $N^2(1-f)S^2/n$.
  • 4 Derive Theorem 8.15(2) directly from the indicator covariances, without the SYG form.
  • 5 Prove that the SYG condition $\pi_i\pi_j\ge\pi_{ij}$ holds for SRSWOR, so that $v_{\mathrm{SYG}}\ge0$ there.
  • 6 Poisson sampling: each unit is included independently with probability $\pi_i$. Find $\operatorname{Var}(\hat Y_{\mathrm{HT}})$, show it has the form $\sum\frac{1-\pi_i}{\pi_i}Y_i^2$, and explain why the sample size is random.
  • 7 Show that if $\pi_i=0$ for some unit $i$, no estimator can be unbiased for $Y$ for every population.
  • 8 For the design of Example 3, compute $\sum_{i\in s}1/\pi_i$ for every sample, and deduce the range of $\hat Y_{\mathrm{HT}}$ when every $Y_i=c$.
1 $\{i,j\}$ arises as ($i$ first, then $j$), with probability $p_i\frac{p_j}{1-p_i}$, or ($j$ first, then $i$), with probability $p_j\frac{p_i}{1-p_j}$. Adding gives $\pi_{ij}$. Then $\pi_i=\sum_{j\ne i}\pi_{ij}=p_i\sum_{j\ne i}\frac{p_j}{1-p_i}+p_i\sum_{j\ne i}\frac{p_j}{1-p_j}=p_i\left(1+\sum_{j\ne i}\frac{p_j}{1-p_j}\right)$.
2 $\pi_{12}=0.06(\frac1{0.8}+\frac1{0.7})=0.1607$, $\pi_{13}=0.1(\frac1{0.8}+\frac1{0.5})=0.325$ and $\pi_{23}=0.15(\frac1{0.7}+2)=0.5143$. So $\pi=(0.4857,0.675,0.8393)$, with sum $2=n$ ✓. Also $\sum_{j\ne i}\pi_{ij}=\pi_i=(n-1)\pi_i$ ✓.
3 $\pi_i=n/N$ gives $\hat Y_{\mathrm{HT}}=\frac Nn\sum_sy_i=N\bar y$. In (2), $\frac{1-f}{f}\sum Y_i^2+\frac{\pi_{ij}-f^2}{f^2}\sum_{i\ne j}Y_iY_j$. Now $\pi_{ij}-f^2=-\frac{f(1-f)}{N-1}$, so this is $\frac{1-f}{f}\left[\sum Y_i^2-\frac{1}{N-1}\sum_{i\ne j}Y_iY_j\right]=\frac{1-f}{f}\cdot\frac{N}{N-1}\sum(Y_i-\bar Y)^2=\frac{N^2(1-f)S^2}{n}$.
4 $\operatorname{Var}\left(\sum Z_ia_i\right)=\sum_ia_i^2\operatorname{Var}Z_i+\sum_{i\ne j}a_ia_j\operatorname{Cov}(Z_i,Z_j)$ with $\operatorname{Var}Z_i=\pi_i-\pi_i^2$ and $\operatorname{Cov}=\pi_{ij}-\pi_i\pi_j$.
5 $\pi_i\pi_j-\pi_{ij}=\frac{n^2}{N^2}-\frac{n(n-1)}{N(N-1)}=\frac{f(1-f)}{N-1}\gt0$ for $1\le n\lt N$.
6 Independence gives $\pi_{ij}=\pi_i\pi_j$, so the cross terms vanish and $\operatorname{Var}=\sum\frac{1-\pi_i}{\pi_i}Y_i^2$. The sample size $\sum Z_i$ is a sum of independent Bernoulli variables, with mean $\sum\pi_i$ and variance $\sum\pi_i(1-\pi_i)$.
7 Every sample of positive probability excludes unit $i$, so any estimator $t(s)$ takes the same values for two populations that differ only in $Y_i$. Its expectation is then the same for both, but the totals differ.
8 With $\pi=(0.2345,0.4413,0.6083,0.7159)$: $\{1,2\}:6.53$, $\{1,3\}:5.91$, $\{1,4\}:5.66$, $\{2,3\}:3.91$, $\{2,4\}:3.66$, $\{3,4\}:3.04$. So $\hat Y_{\mathrm{HT}}$ ranges over $[3.04c,6.53c]$ although the true total is $4c$. It is still unbiased on average.
III
Ratio and regression estimators.
  • 1 From the $N=120$ blocks of a town, an SRSWOR of $8$ blocks gives the number of households $x$ (known for every block from the census, $\bar X=20.5$) and this month's electricity consumption $y$ (MWh): $x=12,20,15,25,18,30,22,10$; $y=33,41,42,52,44,66,45,29$. Compute $\bar y$, $\bar y_R$ and their estimated standard errors.
  • 2 For the data of III.1, compute $\bar y_{lr}$ and its estimated standard error.
  • 3 A population has $\bar Y=80$, $\bar X=40$, $S_y=16$, $S_x=10$, $\rho=0.6$, $N=1000$; $n=50$. Compute the first-order MSEs of $\bar y$, $\bar y_R$ and $\bar y_{lr}$, and explain the ranking with the criterion of Theorem 8.17.
  • 4 Derive the first-order bias $\frac{1-f}{n}\bar Y(C_x^2-\rho C_xC_y)$ of $\bar y_R$ and show that it vanishes to first order iff $B=R$.
  • 5 For the population of III.3, use Theorem 8.16 to bound $|\text{bias}|/\text{SE}$ of $\bar y_R$, and compare with the first-order bias.
  • 6 The product estimator is $\bar y_P=\bar y\,\bar x/\bar X$. Show that $\operatorname{MSE}(\bar y_P)\approx\frac{1-f}{n}(S_y^2+R^2S_x^2+2R\rho S_xS_y)$, that it beats $\bar y$ iff $\rho\lt-C_x/(2C_y)$, and evaluate it for III.3 with $\rho=-0.8$.
  • 7 Show that the difference-type estimator with $b_0=0$, $b_0=R$ and $b_0=B$ gives, respectively, $\bar y$, an estimator with the same first-order MSE as $\bar y_R$, and the minimum-variance member of the family.
  • 8 Prove Theorem 8.18 without matrices, by minimising $\sum_{i\in s}(y_i-\beta x_i)^2/x_i$.
  • 9 With $X=(1,2,3,4)$, $Y=(2,5,5,9)$ and $n=2$, compute the six Midzuno probabilities $P(s)$ and verify that $E\bar y_R=\bar Y$ and $\pi_i=p_i+(1-p_i)\frac{n-1}{N-1}$.
1 $\bar x=19$, $\bar y=44$, $s_y^2=129.71$ and $\frac{1-f}{n}=0.11667$. So $v(\bar y)=15.13$ (SE $3.89$). Next $\hat R=2.3158$ and $\bar y_R=47.47$; $\sum(y_i-\hat Rx_i)^2=229.84$, so $v(\bar y_R)=0.11667\cdot\frac{229.84}{7}=3.83$ (SE $1.96$).
2 $s_{xy}=72.86$ and $s_x^2=44.86$, so $b=1.624$ and $\bar y_{lr}=44+1.624\cdot1.5=46.44$. The residual SS is $79.66$, so $v=0.11667\cdot\frac{79.66}{6}=1.549$ (SE $1.24$).
3 $\frac{1-f}{n}=0.019$ and $R=2$. Then $\operatorname{Var}\bar y=0.019\cdot256=4.864$; $\operatorname{MSE}\bar y_R=0.019(256+400-384)=5.168$; $V_{lr}=4.864(1-0.36)=3.113$. The threshold is $C_x/(2C_y)=0.25/0.4=0.625\gt\rho=0.6$, so the ratio estimator loses to $\bar y$, while the regression estimator wins.
4 From the expansion in the proof of Theorem 8.17, the bias is $\bar Y(E\varepsilon_1^2-E\varepsilon_0\varepsilon_1)=\frac{1-f}{n}\bar Y C_x(C_x-\rho C_y)$. This is zero iff $C_x=\rho C_y$, i.e. $\frac{S_x}{\bar X}=\rho\frac{S_y}{\bar Y}$, i.e. $R=\frac{\bar Y}{\bar X}=\rho\frac{S_y}{S_x}=B$.
5 $\operatorname{CV}(\bar x)=\sqrt{0.019}\cdot0.25=0.0345$, so $|\text{bias}|\le0.0345\cdot\text{SE}$. First order: bias $\approx0.019\cdot80(0.0625-0.6\cdot0.25\cdot0.2)=0.0494$, against SE $\approx\sqrt{5.168}=2.27$, a ratio of $0.022$, within the bound.
6 $\bar y_P=\bar Y(1+\varepsilon_0)(1+\varepsilon_1)$, so to first order $\operatorname{MSE}=\bar Y^2E(\varepsilon_0+\varepsilon_1)^2$, which gives the formula. It is below $\frac{1-f}nS_y^2$ iff $R^2S_x^2+2R\rho S_xS_y\lt0$ iff $\rho\lt-\frac{RS_x}{2S_y}$. With $\rho=-0.8$: $0.019(256+400-512)=2.736\lt4.864$ (and $V_{lr}=0.019\cdot256\cdot0.36=1.751$).
7 $b_0=0$ gives $\bar y$. For $b_0=R$, the formula of Theorem 8.20(1) is $\frac{1-f}{n}(S_y^2-2RS_{xy}+R^2S_x^2)$, exactly the first-order MSE of $\bar y_R$ (Theorem 8.17); indeed $\bar y_R-\bar y_d=(\hat R-R)(\bar X-\bar x)$ is of order $1/n$. $b_0=B$ is the minimiser.
8 $\frac{d}{d\beta}\sum(y_i-\beta x_i)^2/x_i=-2\sum(y_i-\beta x_i)=0$ gives $\hat\beta=\sum y_i/\sum x_i$. Weighted least squares with weights $1/\operatorname{Var}(\varepsilon_i)$ gives the BLUE (Aitken).
9 $p=(0.1,0.2,0.3,0.4)$ and $P(s)=(p_i+p_j)/3$: $0.1,\ 0.1333,\ 0.1667,\ 0.1667,\ 0.2,\ 0.2333$ for $\{1,2\},\dots,\{3,4\}$. The values $\bar y_R=2.5\bar y/\bar x$ are $5.833,\ 4.375,\ 5.5,\ 5,\ 5.833,\ 5$, and $\sum P(s)\bar y_R=5.25=\bar Y$ ✓. Also $\pi=(0.4,0.4667,0.5333,0.6)=p_i+\frac{1-p_i}{3}$ ✓.
IV
Further results.
  • 1 In stratified sampling, compare the separate ratio estimator $\bar y_{RS}=\sum_hW_h\frac{\bar y_h}{\bar x_h}\bar X_h$ with the combined one $\bar y_{RC}=\frac{\bar y_{\mathrm{st}}}{\bar x_{\mathrm{st}}}\bar X$. Write their first-order MSEs and explain why the separate estimator is risky when the $n_h$ are small.
  • 2 Des Raj's ordered estimator for draw-by-draw PPS with $n=2$: $t_1=y_1/p_1$, $t_2=y_1+y_2(1-p_1)/p_2$, $t=\frac12(t_1+t_2)$, where $y_1$ is the first unit drawn. Prove that $t$ is unbiased for $Y$, and check it on the population of Example 7.
  • 3 Give an example in which PPSWR with $n=1$ is much worse than SRSWR with $n=1$, and explain it with Theorem 8.12(2).
  • 4 Show that for $n=1$ the HT estimator under PPS sampling coincides with the Hansen–Hurwitz estimator.
  • 5 Hartley–Ross. Let $r_i=y_i/x_i$ and $\bar r$ be their sample mean. Show that $\bar r\bar X$ is biased for $\bar Y$, and that $t_{HR}=\bar r\bar X+\frac{n(N-1)}{N(n-1)}(\bar y-\bar r\bar x)$ is exactly unbiased under SRSWOR. Verify it on the population of Example 7.
1 Separate: $\sum_hW_h^2\frac{1-f_h}{n_h}(S_{yh}^2+R_h^2S_{xh}^2-2R_h\rho_hS_{xh}S_{yh})$. Combined: $\sum_hW_h^2\frac{1-f_h}{n_h}(S_{yh}^2+R^2S_{xh}^2-2R\rho_hS_{xh}S_{yh})$. The separate one is better when the $R_h$ differ, but its bias is a sum of $L$ stratum biases of order $1/n_h$ each. With many small strata this accumulates and may no longer be small relative to the SE, which is of order $1/\sqrt n$. The combined estimator has a single bias of order $1/n$.
2 $E[t_1]=\sum_ip_i\frac{Y_i}{p_i}=Y$. Given the first unit $i$, the second is $j$ with probability $p_j/(1-p_i)$, so $E[y_2(1-p_1)/p_2\mid i]=\sum_{j\ne i}Y_j=Y-Y_i$. Hence $E[t_2\mid i]=Y$, and $E[t]=Y$. For $X=(2,3,5,10)$ and $Y=(5,7,9,24)$, exact enumeration gives $E[t]=45=Y$ ✓.
3 $X=(1,2,3,10)$ and $Y=(10,12,9,11)$, so $Y=42$ and $p=(\frac1{16},\frac18,\frac3{16},\frac58)$. Then $\operatorname{Var}\hat Y_{\mathrm{HH}}=\sum Y_i^2/p_i-Y^2=1600+1152+432+193.6-1764=1613.6$, whereas $\operatorname{Var}(N y)=16\sigma^2=20$. The ratios $Y_i/p_i=160,96,48,17.6$ vary enormously because $Y$ does not grow with $X$.
4 With $n=1$, $\pi_i=p_i$, so $\hat Y_{\mathrm{HT}}=y/p=\hat Y_{\mathrm{HH}}$.
5 $E\bar r=\bar R_{\cdot}=\frac1N\sum Y_i/X_i$, so $E[\bar r\bar X]-\bar Y=\bar X\bar R_\cdot-\frac1N\sum r_iX_i=-\frac{N-1}{N}S_{rx}$, where $S_{rx}$ is the population covariance of $r_i$ and $X_i$. Under SRSWOR, $s_{rx}=\frac{n}{n-1}(\bar y-\bar r\bar x)$ is unbiased for $S_{rx}$ (because $\frac1n\sum r_ix_i=\bar y$), so adding $\frac{N-1}{N}s_{rx}$ removes the bias. For Example 7: $E[\bar r\bar X]=11.292$, while $E[t_{HR}]=11.25=\bar Y$ ✓.
🎯 Exam corner CSIR Part BCSIR Part CGATE ST

E1. CSIR Part B GATE ST From a population of $N=10$ units, two draws are made with replacement with probabilities proportional to size. The first draw gives a unit with $p=0.10$ and $y=30$; the second gives a unit with $p=0.25$ and $y=60$. The Hansen–Hurwitz estimate of the population total is (A) $270$; (B) $257.1$; (C) $450$; (D) $540$.

Idea $\hat Y_{\mathrm{HH}}=\frac1n\sum y_j/p_j$.
Compute $\frac12(300+240)=270$. (B) is $\frac{30+60}{0.35}$, (C) is $N\bar y$ and (D) forgets the factor $\frac1n$.
Answer (A).

E2. CSIR Part B In Lahiri's method a population has $N=10$ units with total size $X=250$, and $M=50$. The expected number of pairs $(i,R)$ that must be drawn to select one unit is (A) $0.5$; (B) $2$; (C) $5$; (D) $25$.

Idea Theorem 8.13: success probability $\theta=X/(NM)$ per pair.
Compute $\theta=250/500=0.5$, so the expected number of pairs is $1/\theta=2$. (A) is $\theta$ itself.
Answer (B).

E3. CSIR Part B GATE ST An SRSWOR of $n=20$ is drawn from $N=500$ units. For the population $S_y=12$, $S_x=10$, $\rho=0.8$ and $R=\bar Y/\bar X=1.5$. To first order, the mean squared error of the ratio estimator $\bar y_R$ is (A) $3.888$; (B) $6.912$; (C) $2.488$; (D) $31.536$.

Idea $\frac{1-f}{n}(S_y^2+R^2S_x^2-2R\rho S_xS_y)$ with $\frac{1-f}{n}=\frac{0.96}{20}=0.048$.
Compute $0.048(144+225-288)=0.048\cdot81=3.888$. (B) is $\operatorname{Var}\bar y$, (C) is the regression estimator $0.048\cdot144\cdot0.36$, and (D) uses $+2R\rho S_xS_y$.
Answer (A).

E4. CSIR Part C Consider a sampling design without replacement in which every unit has $\pi_i\gt0$, and the Horvitz–Thompson estimator $\hat Y_{\mathrm{HT}}$. Which of the following are true? (One or more options may be correct.) (A) If the design has fixed size $n$, then $\sum_i\pi_i=n$. (B) $\hat Y_{\mathrm{HT}}$ is unbiased for the population total. (C) The Sen–Yates–Grundy variance estimator is non-negative for every fixed-size design with all $\pi_{ij}\gt0$. (D) Under PPS sampling with replacement, the Hansen–Hurwitz estimator has zero variance when $Y_i\propto X_i$.

(A) True Theorem 8.14.
(B) True Theorem 8.15(1).
(C) False Non-negativity needs $\pi_i\pi_j\ge\pi_{ij}$. Take $N=4$, $n=2$ with $P\{1,2\}=P\{3,4\}=0.4$ and probability $0.05$ for each of the other four pairs. Every $\pi_{ij}\gt0$ and every $\pi_i=0.5$, but $\pi_{12}=0.4\gt0.25=\pi_1\pi_2$. On the sample $\{1,2\}$, $$v_{\mathrm{SYG}}=\frac{0.25-0.4}{0.4}(2y_1-2y_2)^2=-1.5(y_1-y_2)^2\lt0$$ whenever $y_1\ne y_2$.
(D) True Theorem 8.12(4).
Answer (A), (B) and (D).

E5. CSIR Part C Let $\bar y_R=(\bar y/\bar x)\bar X$ under SRSWOR, with all $X_i\gt0$. Which of the following are true? (A) $\bar y_R$ is unbiased for $\bar Y$ for every population. (B) $|\text{bias}(\bar y_R)|\le\operatorname{CV}(\bar x)\cdot\operatorname{SD}(\bar y_R)$. (C) To first order, $\bar y_R$ has smaller MSE than $\bar y$ if and only if $\rho\gt C_x/(2C_y)$. (D) If $Y_i=cX_i$ for all $i$, then $\bar y_R=\bar Y$ for every sample.

(A) False Example 7(a): $E\bar y_R=11.168\ne11.25$.
(B) True The Hartley–Ross bound (Theorem 8.16).
(C) True Theorem 8.17.
(D) True Then $\bar y=c\bar x$, so $\bar y_R=c\bar X=\bar Y$.
Answer (B), (C) and (D).

E6. CSIR Part C Under SRSWOR with $\bar X$ known, consider $\bar y_d=\bar y+b_0(\bar X-\bar x)$ for a constant $b_0$ and the regression estimator $\bar y_{lr}$ with the sample slope. Which of the following are true? (A) $\bar y_d$ is unbiased for $\bar Y$ for every $b_0$. (B) $\min_{b_0}\operatorname{Var}(\bar y_d)=\frac{1-f}{n}S_y^2(1-\rho^2)$. (C) To first order, $\operatorname{MSE}(\bar y_{lr})\le\operatorname{MSE}(\bar y_R)$, with equality iff the population regression line passes through the origin. (D) $\bar y_{lr}$ is exactly unbiased for every population.

(A) True It is the mean of $Y_i-b_0X_i$ plus the constant $b_0\bar X$ (Theorem 8.20(1)).
(B) True The minimum is at $b_0=B$.
(C) True The difference is $\frac{1-f}{n}(RS_x-\rho S_y)^2$, which is zero iff $B=R$.
(D) False Take $N=3$, $n=2$ and the points $(1,1),(2,3),(4,4)$, with $\bar X=\frac73$ and $\bar Y=\frac83$. The three samples give $\bar y_{lr}=\frac{11}3,\frac73,\frac{19}6$, whose mean is $3.056\ne2.667$.
Answer (A), (B) and (C).
🧑‍🏫 For the teacher

Suggested plan — 3 lectures × 60 min.

  1. PPS with replacement. Motivate with villages of very different sizes. Prove Theorem 8.12 (the iid trick) and emphasise part (4). Cumulative totals and Lahiri (Theorem 8.13 with the geometric argument), then Examples 1–2 on the board. Project Interactive 1, first "proportional", then "decreasing".
  2. Inclusion probabilities and HT. Theorem 8.14 from indicators, the HT theorem with the SYG proof in full, Example 3 (assign the six samples to six groups of students), and Basu's elephants as a discussion. Finish with the first three bullets of the ⚠️ box.
  3. Ratio and regression. The Hartley–Ross identity (two lines), the first-order expansion, the criterion $\rho\gt C_x/(2C_y)$, the model view (Theorem 8.18), Midzuno (Theorem 8.19), and then the regression estimator and the comparison theorem. Example 5 with a calculator, Example 6 as a poll. Project Interactive 2: $a=0$, then $a=100$, then a large $\sigma$. End with E4–E6.

Discussion questions. (1) The HT estimator is unbiased for every design, so why do we not simply use it with any convenient design? (2) Under what model would you prefer the ratio estimator to the regression estimator, even with an intercept present (hint: small $n$, heteroscedasticity)? (3) A size measure is ten years old. How does its age affect PPS and ratio estimation differently?

Misconceptions to watch for. Using $y_i/(np_i)$ with draw-by-draw $\pi_i$; believing unbiasedness implies a good estimator; thinking any positive correlation helps the ratio estimator; treating the ratio estimator as unbiased; forgetting that $\bar X$ must be known exactly; using $n-1$ instead of $n-2$ in the residual variance of the regression estimator; applying Lahiri with $M\lt\max X_i$.

Suggested homework. I(1, 2, 4, 6), II(1, 2, 3, 5, 8), III(1, 2, 3, 6, 9), IV(2, 5); Exam corner E1, E4, E6.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. Satellite vegetation indices (such as NDVI) are available for every field at almost no cost, while yields come from expensive crop-cutting experiments. How much can the satellite index improve a crop-yield estimate, and which estimator — PPS, ratio or regression — makes the best use of it?

Why it matters. National agricultural statistics in India, the United States, the European Union and many African countries combine field measurements with remote sensing. The choice of estimator decides how many costly field plots are needed for a given precision, and the wrong choice can introduce bias in years when the yield–NDVI relationship shifts (drought, floods).

What to do.

  1. Simulate a population of $N=2000$ fields with $x_i=\text{NDVI}_i\sim U(0.2,0.8)$ and yield $y_i=a+\beta x_i+\varepsilon_i$, with $\operatorname{Var}\varepsilon_i\propto x_i$. Take three scenarios: $a=0$; $a=1.5$ t/ha (a yield floor); and a "drought year" in which $\beta$ is halved.
  2. For $n=30$, run $5000$ replications of each design: SRS with $\bar y$, SRS with $\bar y_R$ and $\bar y_{lr}$, and PPSWR by $x$ with Hansen–Hurwitz. Record the bias and the MSE.
  3. Compare the simulated MSEs with the first-order formulas of Theorems 8.12, 8.17 and 8.20. Where does the approximation break down? Try $n=8$.
  4. Compute the sample size each method needs for a $2\%$ relative standard error, and translate it into crop-cutting cost.

Tools: Python (numpy) or R; any plotting library.

Hand in: A table of bias and MSE for the four estimators in the three scenarios, a plot of simulated versus first-order MSE, and a recommendation for the agency.

Going further. Read about the generalised regression (GREG) estimator and calibration estimators, which extend $\bar y_{lr}$ to several auxiliary variables and arbitrary designs, and about small-area estimation, where the same idea borrows strength across districts.

Instructor note: with $a=0$ and variance proportional to $x$, ratio, regression and PPS are all close, and PPS is often best. With an intercept the ratio and PPS estimators lose heavily while regression keeps its gain. In the drought year all model-assisted estimators remain nearly unbiased under the design (they are design-consistent), but their gains shrink because $\rho$ falls. At $n=8$ the first-order formulas underestimate the MSE of $\bar y_R$ by noticeably more than for $n=30$.

🧪 Self-check quiz

0/5 correct

Q1. Under PPS sampling with replacement ($n$ draws, normed sizes $p_i$), the Hansen–Hurwitz estimator of the total is

Q2. In Lahiri's method a pair $(i,R)$ with $R$ uniform on $\{1,\dots,M\}$ leads to selection of unit $i$ when

Q3. The Horvitz–Thompson estimator $\sum_{i\in s}y_i/\pi_i$ is unbiased for the total

Q4. To first order, the ratio estimator $\bar y_R$ is more efficient than $\bar y$ if and only if

Q5. The minimum over $b_0$ of $\operatorname{Var}\big(\bar y+b_0(\bar X-\bar x)\big)$ under SRSWOR equals