8.1 Simple Random, Stratified and Systematic Sampling
How to estimate a population mean, total or proportion from a probability sample: simple random sampling with and without replacement and the finite population correction, stratification and the allocation of the sample to strata, and systematic sampling with its strengths (trends) and its weaknesses (periodicity).
A statistical office that wants the average monthly expenditure of the households of a state cannot visit them all. It chooses a few thousand households by a random mechanism, measures them, and publishes an estimate with a standard error. Two features make this different from the iid sampling of Chapter 4. The population is finite and completely specified: there are $N$ households with fixed (unknown) values $Y_1,\dots,Y_N$, and the only randomness is the one we introduce when we draw the sample. And the statistician designs that randomness: she can sample without replacement, split the population into strata and sample each separately, or take every $k$-th household from a list. Each choice changes the variance of the estimator, sometimes dramatically, and the whole subject consists of computing these variances exactly and choosing the design that makes them small for a given cost.
This section covers the three basic designs. Simple random sampling gives the finite population correction $1-n/N$ and the formulae for sample size. Stratified sampling gives the allocation problem, solved by Neyman's allocation and its cost version, together with an exact account of when stratification helps. Systematic sampling is the cheapest design in the field. It is excellent for populations with a trend and disastrous for periodic ones, and the reason is one variance identity. CSIR NET asks for these variance formulae in Part B, and in Part C it asks which statements about unbiasedness, allocation and efficiency are true. The traps are listed in the ⚠️ box.
Finite Populations and Simple Random Sampling
Throughout, the population is a finite set of labelled units $U=\{1,2,\dots,N\}$, and the characteristic of interest takes the fixed value $Y_i$ on unit $i$. Following the standard convention of survey sampling, population quantities are written in capitals and sample quantities in lower case ($\bar y$, $s^2$, $y_1,\dots,y_n$), even though the sample quantities are random variables. The only randomness comes from the selection of the sample.
The population total, mean and the two population variances are
$$Y=\sum_{i=1}^NY_i,\qquad \bar Y=\frac YN,$$ $$\sigma^2=\frac1N\sum_{i=1}^N(Y_i-\bar Y)^2,\qquad S^2=\frac1{N-1}\sum_{i=1}^N(Y_i-\bar Y)^2=\frac{N}{N-1}\sigma^2 .$$A sampling design is a probability distribution $p(\cdot)$ on a collection of possible samples (ordered sequences or unordered subsets of $U$). The sampling frame is the list of units from which the sample is drawn. With $n$ the sample size and $f=n/N$ the sampling fraction:
- Simple random sampling without replacement (SRSWOR): each of the $\binom Nn$ subsets of size $n$ has probability $1/\binom Nn$. Equivalently, draw units one at a time, each time choosing with equal probability among the units not yet drawn.
- Simple random sampling with replacement (SRSWR): $n$ independent draws, each choosing every unit with probability $1/N$; each of the $N^n$ ordered sequences has probability $N^{-n}$.
From the sample values $y_1,\dots,y_n$ we form the sample mean $\bar y=\frac1n\sum_{j=1}^ny_j$ and the sample variance $s^2=\frac1{n-1}\sum_{j=1}^n(y_j-\bar y)^2$. The inclusion probability of unit $i$ is $\pi_i=P(i\in s)$, and $\pi_{ij}=P(i\in s,\ j\in s)$ for $i\ne j$.
The divisor $N-1$ in $S^2$ is not a matter of taste. We will see that $S^2$, not $\sigma^2$, is the quantity that $s^2$ estimates without bias under SRSWOR, and the formulae for variances come out cleanest in terms of $S^2$.
Under SRSWOR of size $n$ ($1\le n\le N$, $N\ge2$) let $Z_i=1$ if unit $i$ is in the sample and $Z_i=0$ otherwise. Then $\sum_{i=1}^NZ_i=n$ and, for $i\ne j$,
$$\pi_i=E[Z_i]=\frac nN=f,\qquad \pi_{ij}=E[Z_iZ_j]=\frac{n(n-1)}{N(N-1)},$$ $$\operatorname{Var}(Z_i)=f(1-f),\qquad \operatorname{Cov}(Z_i,Z_j)=-\frac{f(1-f)}{N-1}.$$Proof. The samples containing unit $i$ are obtained by choosing the other $n-1$ members from the remaining $N-1$ units, so $\pi_i=\binom{N-1}{n-1}\big/\binom Nn=\frac nN$. Similarly the samples containing both $i$ and $j$ number $\binom{N-2}{n-2}$, and $\binom{N-2}{n-2}\big/\binom Nn=\frac{n(n-1)}{N(N-1)}$. Since $Z_i^2=Z_i$, $\operatorname{Var}(Z_i)=f-f^2$. Finally
$$\operatorname{Cov}(Z_i,Z_j)=\frac{n(n-1)}{N(N-1)}-\frac{n^2}{N^2}=\frac nN\cdot\frac{N(n-1)-n(N-1)}{N(N-1)}=-\frac nN\cdot\frac{N-n}{N(N-1)},$$which is $-f(1-f)/(N-1)$. ∎
The negative covariance has a plain meaning: if unit $i$ takes one of the $n$ places, fewer places are left for unit $j$. It is this negative dependence that makes sampling without replacement more precise than sampling with replacement.
Under SRSWOR of size $n$ with $2\le n\le N$,
- $\bar y$ is unbiased for $\bar Y$, and $\hat Y=N\bar y$ is unbiased for the total $Y$;
- the variance of the sample mean carries the finite population correction (fpc) $1-f$: $$\operatorname{Var}(\bar y)=\frac{(1-f)\,S^2}{n}=\Big(\frac1n-\frac1N\Big)S^2,\qquad \operatorname{Var}(N\bar y)=\frac{N^2(1-f)S^2}{n};$$
- $E[s^2]=S^2$, so $v(\bar y)=\dfrac{(1-f)\,s^2}{n}$ is an unbiased estimator of $\operatorname{Var}(\bar y)$.
Proof. Write $\bar y=\frac1n\sum_{i=1}^NZ_iY_i$, a sum over the whole population in which only the sampled units count. (1) $E[\bar y]=\frac1n\sum_i\frac nNY_i=\bar Y$.
(2) Put $D_i=Y_i-\bar Y$, so that $\sum_iD_i=0$. Because $\sum_iZ_i=n$, we have $\bar y-\bar Y=\frac1n\sum_iZ_iD_i$. By Theorem 8.1,
$$n^2\operatorname{Var}(\bar y)=f(1-f)\sum_iD_i^2-\frac{f(1-f)}{N-1}\sum_{i\ne j}D_iD_j .$$Since $0=\left(\sum_iD_i\right)^2=\sum_iD_i^2+\sum_{i\ne j}D_iD_j$, the cross sum equals $-\sum_iD_i^2$. Hence $n^2\operatorname{Var}(\bar y)=f(1-f)\frac{N}{N-1}\sum_iD_i^2=f(1-f)NS^2$, and dividing by $n^2$ (with $fN=n$) gives $\operatorname{Var}(\bar y)=(1-f)S^2/n$.
(3) Expanding about $\bar Y$ gives the identity $(n-1)s^2=\sum_iZ_iD_i^2-n(\bar y-\bar Y)^2$. Taking expectations, $E\sum_iZ_iD_i^2=\frac nN(N-1)S^2$ and $nE(\bar y-\bar Y)^2=(1-f)S^2=\frac{N-n}{N}S^2$, so
$$(n-1)E[s^2]=\frac{n(N-1)-(N-n)}{N}S^2=\frac{(n-1)N}{N}S^2=(n-1)S^2 .\ ∎$$Under SRSWR of size $n\ge2$ the draws $y_1,\dots,y_n$ are iid, each uniform on $\{Y_1,\dots,Y_N\}$. Hence $E[\bar y]=\bar Y$, $E[s^2]=\sigma^2$ and
$$\operatorname{Var}_{\mathrm{WR}}(\bar y)=\frac{\sigma^2}{n}=\frac{N-1}{N}\cdot\frac{S^2}{n}.$$For the same $n$, $\dfrac{\operatorname{Var}_{\mathrm{WOR}}(\bar y)}{\operatorname{Var}_{\mathrm{WR}}(\bar y)}=\dfrac{N-n}{N-1}\le1$, with equality only when $n=1$.
Proof. A single draw takes the value $Y_i$ with probability $1/N$, so it has mean $\bar Y$ and variance $\frac1N\sum(Y_i-\bar Y)^2=\sigma^2$. The draws are independent, so the moments of the mean of an iid sample (§4.1) give $E\bar y=\bar Y$, $\operatorname{Var}\bar y=\sigma^2/n$ and $Es^2=\sigma^2$. The ratio is $\frac{(1-f)S^2/n}{(N-1)S^2/(Nn)}=\frac{N-n}{N-1}$. ∎
So the fpc matters only when the sampling fraction is appreciable. For $N=10^6$ and $n=1000$ it is $0.999$. The precision of $\bar y$ then depends on $n$ and on $S^2$, almost not at all on $N$. A sample of $1000$ is about as informative about a country as about a city with the same variability. This surprises almost everyone the first time.
Confidence intervals. For large $n$ and $N-n$, a finite-population central limit theorem (due to Erdős–Rényi and Hájek; we quote it) makes $(\bar y-\bar Y)/\sqrt{\operatorname{Var}(\bar y)}$ approximately $N(0,1)$, provided no small group of units dominates $S^2$. With $s^2$ in place of $S^2$, the approximate $100(1-\alpha)\%$ interval is
$$\bar y\pm z_{\alpha/2}\,s\sqrt{\frac{1-f}{n}}.$$Here $z_{\alpha/2}=\Phi^{-1}(1-\alpha/2)$, so $z_{0.025}=1.960$.
Suppose $Y_i\in\{0,1\}$, so that $\bar Y=P$ is the proportion of units with the attribute; put $Q=1-P$, and let $p=\bar y$ be the sample proportion and $q=1-p$. Under SRSWOR with $2\le n\le N$:
$$S^2=\frac{NPQ}{N-1},\qquad E[p]=P,\qquad \operatorname{Var}(p)=\frac{N-n}{N-1}\cdot\frac{PQ}{n},$$and $v(p)=\dfrac{(1-f)\,pq}{n-1}$ is unbiased for $\operatorname{Var}(p)$. The number $np$ of sampled units with the attribute is hypergeometric (§2.1).
Proof. With $0/1$ values $\sum Y_i^2=\sum Y_i=NP$, so $(N-1)S^2=NP-NP^2=NPQ$. Theorem 8.2(2) gives $\operatorname{Var}(p)=\frac{N-n}{Nn}\cdot\frac{NPQ}{N-1}$. In the same way $(n-1)s^2=np-np^2=npq$, so $v(p)=(1-f)s^2/n=(1-f)pq/(n-1)$, which is unbiased by Theorem 8.2(3). The count $np$ is the number of "successes" in $n$ draws without replacement from $NP$ successes and $NQ$ failures. ∎
Under SRSWOR with the normal approximation, $P(|\bar y-\bar Y|\le d)\approx1-\alpha$ holds exactly when $z_{\alpha/2}^2\left(\frac1n-\frac1N\right)S^2=d^2$, that is, when
$$n=\frac{n_0}{1+n_0/N},\qquad n_0=\Big(\frac{z_{\alpha/2}\,S}{d}\Big)^2 .$$Here $n_0$ is the size needed if $N$ were infinite (or with replacement). For a proportion, with $S^2=NPQ/(N-1)$, the solution is
$$n=\frac{n_0}{1+(n_0-1)/N},\qquad n_0=\frac{z_{\alpha/2}^2\,PQ}{d^2}.$$Since $PQ\le\frac14$, the choice $P=\frac12$ gives a size that works whatever $P$ is. For a relative margin $|\bar y-\bar Y|\le r\bar Y$, replace $S/d$ by $C/r$, where $C=S/\bar Y$ is the coefficient of variation.
Proof. Solving $\frac1n=\frac1N+\frac{d^2}{z^2S^2}=\frac1N+\frac1{n_0}$ gives $n=\frac{Nn_0}{N+n_0}$. For a proportion, $\frac1n=\frac1N+\frac{d^2(N-1)}{z^2NPQ}=\frac1N+\frac{N-1}{Nn_0}=\frac{n_0+N-1}{Nn_0}$. In both cases $n$ increases with $S$ (or with $PQ$), which proves the worst-case remark. The relative version is the same equation with $d=r\bar Y$. ∎
In practice $S$ is unknown at the planning stage. It is taken from a pilot survey, a previous round of the same survey, or a rough guess such as "range$/4$". Then $n$ is rounded up.
Stratified Random Sampling and Allocation
If the population splits naturally into groups that are internally alike — urban and rural blocks, small and large farms, age bands — we can sample each group separately. The variability between groups then never enters the error, and we may also spend more of the sample where the variability is larger or the interviews are cheaper.
The population is partitioned into $L$ strata $U_1,\dots,U_L$ of sizes $N_1,\dots,N_L$ ($\sum N_h=N$), with stratum weights $W_h=N_h/N$, stratum means $\bar Y_h$ and stratum variances $S_h^2=\frac1{N_h-1}\sum_{i\in U_h}(Y_i-\bar Y_h)^2$. In stratified random sampling an SRSWOR of size $n_h$ is drawn from each stratum, independently across strata, with $n=\sum n_h$. With $\bar y_h$ and $s_h^2$ the sample mean and variance in stratum $h$ and $f_h=n_h/N_h$, the stratified mean is
$$\bar y_{\mathrm{st}}=\sum_{h=1}^LW_h\,\bar y_h .$$Under stratified random sampling, $\bar y_{\mathrm{st}}$ is unbiased for $\bar Y$ and
$$\operatorname{Var}(\bar y_{\mathrm{st}})=\sum_{h=1}^LW_h^2\,\frac{(1-f_h)S_h^2}{n_h}=\sum_hW_h^2\frac{S_h^2}{n_h}-\frac1N\sum_hW_hS_h^2 .$$If every $n_h\ge2$, then $v(\bar y_{\mathrm{st}})=\sum_hW_h^2(1-f_h)s_h^2/n_h$ is unbiased for this variance. The total is estimated by $N\bar y_{\mathrm{st}}=\sum_hN_h\bar y_h$.
Proof. By Theorem 8.2 applied inside stratum $h$, $E\bar y_h=\bar Y_h$, $\operatorname{Var}\bar y_h=(1-f_h)S_h^2/n_h$ and $Es_h^2=S_h^2$. Hence $E\bar y_{\mathrm{st}}=\sum W_h\bar Y_h=\frac1N\sum_h N_h\bar Y_h=\bar Y$. The $\bar y_h$ are independent, so the variance of the weighted sum is $\sum W_h^2\operatorname{Var}\bar y_h$. The second form uses $W_h^2/N_h=W_h/N$. ∎
Note that $\bar y_{\mathrm{st}}$ weights each stratum mean by the population share $W_h$. It is not, in general, the plain average of all $n$ sampled values.
Let the cost of the survey be $C=c_0+\sum_hc_hn_h$, where $c_h$ is the cost of observing one unit in stratum $h$. An allocation is a choice of $(n_1,\dots,n_L)$. The standard ones are
- proportional allocation: $n_h=nW_h$, i.e. $f_h=f$ for every $h$;
- Neyman allocation: $n_h=n\,\dfrac{N_hS_h}{\sum_kN_kS_k}$;
- optimum allocation (with costs): $n_h\propto\dfrac{N_hS_h}{\sqrt{c_h}}$.
Treat the $n_h$ as positive real numbers and ignore the constraints $n_h\le N_h$.
- For a fixed budget $C$, $\operatorname{Var}(\bar y_{\mathrm{st}})$ is minimised exactly when $n_h\propto W_hS_h/\sqrt{c_h}$, that is, when $$n_h=(C-c_0)\,\frac{N_hS_h/\sqrt{c_h}}{\sum_kN_kS_k\sqrt{c_k}} .$$
- For a fixed variance $V$, the cost is minimised by the same allocation, and the total sample size is $$n=\frac{\left(\sum_hW_hS_h\sqrt{c_h}\right)\left(\sum_hW_hS_h/\sqrt{c_h}\right)}{V+\frac1N\sum_hW_hS_h^2}.$$
- If all $c_h$ are equal (or $n$ is fixed), the optimum is Neyman allocation, with $$V_{\mathrm{Ney}}=\frac{\left(\sum_hW_hS_h\right)^2}{n}-\frac1N\sum_hW_hS_h^2 .$$
Proof. Write $V'=\operatorname{Var}(\bar y_{\mathrm{st}})+\frac1N\sum W_hS_h^2=\sum_hW_h^2S_h^2/n_h$, which differs from the variance by a constant, and $C'=C-c_0=\sum c_hn_h$. By the Cauchy–Schwarz inequality, applied to $a_h=\sqrt{c_hn_h}$ and $b_h=W_hS_h/\sqrt{n_h}$,
$$V'C'=\Big(\sum_h b_h^2\Big)\Big(\sum_h a_h^2\Big)\ge\Big(\sum_ha_hb_h\Big)^2=\Big(\sum_hW_hS_h\sqrt{c_h}\Big)^2 .$$The right side does not depend on the allocation, and equality holds if and only if $b_h/a_h=W_hS_h/(n_h\sqrt{c_h})$ is the same for all $h$, i.e. $n_h\propto W_hS_h/\sqrt{c_h}$. So for fixed $C'$ the product, hence $V'$, is smallest exactly at this allocation; for fixed $V'$ the same argument minimises $C'$. This proves the optimality claims in (1) and (2). The constant of proportionality in (1) comes from $\sum c_hn_h=C'$. In (2), if $n_h=KW_hS_h/\sqrt{c_h}$, then $V'=\sum W_hS_h\sqrt{c_h}/K$. This fixes $K$, and $n=K\sum W_hS_h/\sqrt{c_h}$. For (3) put $c_h\equiv1$, so $n_h=nW_hS_h/\sum W_kS_k$, and $\sum W_h^2S_h^2/n_h=(\sum W_hS_h)^2/n$. ∎
The rule is easy to remember: sample more in a stratum that is larger, more variable, or cheaper to observe — in proportion to size, in proportion to the standard deviation, and inversely to the square root of the unit cost.
Let $V_{\mathrm{ran}}=(1-f)S^2/n$ (SRSWOR of size $n$), $V_{\mathrm{prop}}$ the variance under proportional allocation, and $V_{\mathrm{Ney}}$ that under Neyman allocation, all with the same $n$. Then
$$V_{\mathrm{prop}}=\frac{1-f}{n}\sum_hW_hS_h^2,\qquad V_{\mathrm{prop}}-V_{\mathrm{Ney}}=\frac1n\sum_hW_h\big(S_h-\bar S\big)^2\ge0,$$where $\bar S=\sum_hW_hS_h$. Moreover
$$V_{\mathrm{ran}}-V_{\mathrm{prop}}=\frac{1-f}{n(N-1)}\Big[\sum_hN_h(\bar Y_h-\bar Y)^2-\frac1N\sum_h(N-N_h)S_h^2\Big].$$This can be negative. If every $N_h$ is large, so that terms of order $1/N_h$ are negligible, it is approximately $\frac{1-f}{n}\sum_hW_h(\bar Y_h-\bar Y)^2\ge0$. Then $V_{\mathrm{ran}}\ge V_{\mathrm{prop}}\ge V_{\mathrm{Ney}}$.
Proof. With $n_h=nW_h$ we have $f_h=f$ and $W_h^2/n_h=W_h/n$, which gives $V_{\mathrm{prop}}$. Subtracting the formula of Theorem 8.7(3),
$$V_{\mathrm{prop}}-V_{\mathrm{Ney}}=\frac1n\Big[\sum_hW_hS_h^2-\Big(\sum_hW_hS_h\Big)^2\Big]=\frac1n\sum_hW_h(S_h-\bar S)^2,$$which is a "variance of the $S_h$" and is therefore non-negative. For the comparison with SRS we use the analysis-of-variance identity for the population (§6.2):
$$(N-1)S^2=\sum_h(N_h-1)S_h^2+\sum_hN_h(\bar Y_h-\bar Y)^2 .$$It gives
$$(N-1)\Big[S^2-\sum_h W_hS_h^2\Big]=\sum_h N_h(\bar Y_h-\bar Y)^2+\sum_hd_hS_h^2,$$with $d_h=N_h-1-\frac{(N-1)N_h}N=-\frac{N-N_h}{N}$. Multiplying by $(1-f)/(n(N-1))$ gives the stated difference. For large $N_h$ the term $\frac1N\sum(N-N_h)S_h^2$ is of smaller order than $\sum N_h(\bar Y_h-\bar Y)^2$ unless the stratum means are (nearly) equal, and $\frac{N_h}{N-1}\approx W_h$. ∎
So stratification gains through two channels. Proportional allocation removes the between-strata variation $\sum W_h(\bar Y_h-\bar Y)^2$. Neyman allocation adds a further gain when the $S_h$ differ. A stratification is worth making when the strata means differ, or when the strata variances differ.
- Tabulate $N_h$, $W_h$, a guess of $S_h$ (pilot or past survey) and, if relevant, $c_h$.
- Compute the allocation: $W_h$ (proportional), $N_hS_h$ (Neyman) or $N_hS_h/\sqrt{c_h}$ (cost). Normalise to the total $n$ or to the budget $C-c_0$.
- If some $n_h\gt N_h$, take a census of that stratum ($n_h=N_h$, contributing zero variance) and re-allocate the rest of the sample among the other strata by the same rule.
- Round to integers keeping $\sum n_h=n$, and make sure that $n_h\ge2$ wherever a variance estimate is needed.
- Evaluate $\operatorname{Var}(\bar y_{\mathrm{st}})=\sum W_h^2(1-f_h)S_h^2/n_h$ and compare with $(1-f)S^2/n$ to report the gain.
| stratum $h$ | small | medium | large |
|---|---|---|---|
| $N_h$ | 600 | 300 | 100 |
| $\bar Y_h$ (quintals) | 20 | 40 | 100 |
| $S_h$ | 5 | 10 | 40 |
| $c_h$ (cost units) | 1 | 4 | 16 |
Systematic Sampling
In the field the easiest way to pick $40$ houses out of $400$ on a list is to start at a random house among the first ten and then take every tenth. This is systematic sampling. It needs a single random number, it is easy to supervise, and it spreads the sample evenly over the list. Its precision depends entirely on how the list is ordered.
Let $N=nk$ with integers $n,k\ge2$, and let the units be in a fixed order $1,\dots,N$. A (linear) systematic sample with sampling interval $k$ chooses a random start $r$ uniformly from $\{1,\dots,k\}$ and takes the units $r,\ r+k,\ r+2k,\ \dots,\ r+(n-1)k$. Write $y_{rj}=Y_{r+(j-1)k}$ ($j=1,\dots,n$) for the values of the $r$-th possible sample and $\bar y_r$ for its mean. Then $\bar y_{\mathrm{sys}}=\bar y_r$ with probability $1/k$ each. Define the within-sample variance
$$S^2_{\mathrm{wsy}}=\frac1{k(n-1)}\sum_{r=1}^k\sum_{j=1}^n(y_{rj}-\bar y_r)^2 .$$Define also the intraclass correlation between pairs of units in the same systematic sample:
$$\rho_w=\frac{\sum_{r=1}^k\sum_{j\ne j'}(y_{rj}-\bar Y)(y_{rj'}-\bar Y)}{N(n-1)\,\sigma^2}.$$If $N$ is not a multiple of $n$, circular systematic sampling takes $k$ to be the integer nearest to $N/n$ and a random start $r\in\{1,\dots,N\}$. It then takes the units $r+jk$ reduced modulo $N$ to $\{1,\dots,N\}$, for $j=0,\dots,n-1$; these are distinct when $(n-1)k\lt N$.
The $k$ possible samples are the clusters of a partition of the population, and systematic sampling selects exactly one cluster at random. Every unit has $\pi_i=1/k=n/N$, as in SRSWOR. Most pairs, however, have $\pi_{ij}=0$: two units in different clusters are never sampled together.
For linear systematic sampling with $N=nk$, $\bar y_{\mathrm{sys}}$ is unbiased for $\bar Y$ and
$$\operatorname{Var}(\bar y_{\mathrm{sys}})=\frac1k\sum_{r=1}^k(\bar y_r-\bar Y)^2,$$ $$\operatorname{Var}(\bar y_{\mathrm{sys}})=\frac{N-1}{N}S^2-\frac{k(n-1)}{N}S^2_{\mathrm{wsy}},$$ $$\operatorname{Var}(\bar y_{\mathrm{sys}})=\frac{\sigma^2}{n}\big[1+(n-1)\rho_w\big].$$Proof. $E\bar y_{\mathrm{sys}}=\frac1k\sum_r\bar y_r=\frac1{nk}\sum_r\sum_jy_{rj}=\bar Y$, since the $k$ samples together contain every unit once. The first formula is then the definition of the variance of a variable taking the values $\bar y_r$ with equal probabilities. For the second, decompose the total sum of squares over the $k$ clusters as in one-way analysis of variance:
$$\sum_{r,j}(y_{rj}-\bar Y)^2=n\sum_r(\bar y_r-\bar Y)^2+\sum_{r,j}(y_{rj}-\bar y_r)^2 .$$The left side is $(N-1)S^2$, and the two terms on the right are $nk\operatorname{Var}(\bar y_{\mathrm{sys}})$ and $k(n-1)S^2_{\mathrm{wsy}}$; divide by $N=nk$. For the third, square $n(\bar y_r-\bar Y)=\sum_j(y_{rj}-\bar Y)$:
$$n^2(\bar y_r-\bar Y)^2=\sum_j(y_{rj}-\bar Y)^2+\sum_{j\ne j'}(y_{rj}-\bar Y)(y_{rj'}-\bar Y).$$Summed over $r$, the first part gives $N\sigma^2$ and the second gives $N(n-1)\rho_w\sigma^2$. Dividing by $kn^2$ and using $N/k=n$ gives $\frac{\sigma^2}{n}[1+(n-1)\rho_w]$. ∎
With $N=nk$ and $V_{\mathrm{ran}}=(1-f)S^2/n$ the variance of the SRSWOR mean of the same size,
$$\operatorname{Var}(\bar y_{\mathrm{sys}})-V_{\mathrm{ran}}=\frac{k(n-1)}{N}\big(S^2-S^2_{\mathrm{wsy}}\big).$$Hence systematic sampling is more precise than SRSWOR if and only if $S^2_{\mathrm{wsy}}\gt S^2$, equivalently if and only if $\rho_w\lt-\frac1{N-1}$. In words, it is better exactly when the units within a systematic sample are more heterogeneous than the population as a whole.
Proof. Here $1-f=\frac{k-1}{k}$, so $V_{\mathrm{ran}}=\frac{(k-1)S^2}{nk}=\frac{(k-1)S^2}{N}$. Subtract this from the second form in Theorem 8.9 and use $N-1-(k-1)=N-k=k(n-1)$. For the $\rho_w$ form, substitute $\sigma^2=\frac{N-1}NS^2$ into the condition
$$\frac{\sigma^2}n\big[1+(n-1)\rho_w\big]\lt\frac{N-n}{Nn}S^2 .$$It becomes
$$(N-1)(n-1)\rho_w\lt(N-n)-(N-1)=-(n-1),$$i.e. $\rho_w\lt-\frac1{N-1}$. ∎
Let $Y_i=a+bi$ for $i=1,\dots,N$ with $N=nk$. Let $V_{\mathrm{st}}$ be the variance of the stratified mean when the population is cut into $n$ strata of $k$ consecutive units and one unit is drawn from each. Then
$$V_{\mathrm{ran}}=\frac{b^2(k-1)(N+1)}{12},\qquad V_{\mathrm{sys}}=\frac{b^2(k^2-1)}{12},\qquad V_{\mathrm{st}}=\frac{b^2(k^2-1)}{12\,n}.$$So $V_{\mathrm{st}}\le V_{\mathrm{sys}}\le V_{\mathrm{ran}}$, with $V_{\mathrm{sys}}/V_{\mathrm{ran}}=\frac{k+1}{N+1}\approx\frac1n$.
Proof. The values $a+bi$, $i=1,\dots,m$, have variance (divisor $m-1$) $b^2\frac{m(m+1)}{12}$, because $\sum_{i=1}^m(i-\frac{m+1}2)^2=\frac{m(m^2-1)}{12}$. With $m=N$ this gives $S^2=b^2N(N+1)/12$ and $V_{\mathrm{ran}}=\frac{(k-1)S^2}{N}$, the first formula. The $r$-th systematic sample has mean $\bar y_r=a+b\left(r+\frac{(n-1)k}{2}\right)$, which runs over an arithmetic progression with step $b$ as $r=1,\dots,k$. So $V_{\mathrm{sys}}=b^2\cdot\frac1k\sum_r(r-\frac{k+1}2)^2=b^2\frac{k^2-1}{12}$. Each stratum of $k$ consecutive values has $S_h^2=b^2\frac{k(k+1)}{12}$, $W_h=\frac1n$, $n_h=1$, $f_h=\frac1k$. Theorem 8.6 gives $V_{\mathrm{st}}=n\cdot\frac1{n^2}(1-\frac1k)\,b^2\frac{k(k+1)}{12}=\frac{b^2(k^2-1)}{12n}$. Finally $\frac{V_{\mathrm{sys}}}{V_{\mathrm{ran}}}=\frac{k+1}{N+1}\le1$ because $k\le N$. ∎
Under a trend, a systematic sample is in effect a stratified sample with one unit per stratum. But it always takes the units at the same relative position in every stratum. If $r$ is small, every unit sits low in its stratum, and these errors add up. Yates' end corrections remove this error exactly for a linear trend. One gives the first and the last sampled units the weights $\frac1n\pm\frac{2r-k-1}{2(n-1)k}$ instead of $\frac1n$ (Exercise III.7).
- $s^2$ estimates different things under the two SRS schemes. It is unbiased for $S^2$ under SRSWOR, but for $\sigma^2=\frac{N-1}{N}S^2$ under SRSWR (Example 3(b)).
- $s$ is biased for $S$. For the population $\{0,0,3\}$ with $n=2$, $E[s]=\sqrt2\approx1.414$ but $S=\sqrt3\approx1.732$. Jensen's inequality gives $E[s]\le\sqrt{E[s^2]}=S$, with equality only when $s$ is constant.
- Proportional stratification can lose (slightly). With equal stratum means, $V_{\mathrm{prop}}\gt V_{\mathrm{ran}}$ (Example 5). The inequality $V_{\mathrm{prop}}\le V_{\mathrm{ran}}$ holds only up to terms of order $1/N_h$.
- The plain sample mean is biased under disproportionate allocation. With Neyman allocation in Example 4, the average of all $100$ farms gives the large farms weight $0.4$ instead of $W_3=0.1$; its expectation is $0.3\cdot20+0.3\cdot40+0.4\cdot100=58\ne34$. Only proportional allocation is self-weighting.
- Neyman allocation can ask for more units than a stratum has. With $N_h=(50,950)$, $S_h=(100,2)$ and $n=100$, the rule gives $n_1=72.5\gt50$. Take a census of stratum 1 and give the remaining $50$ units to stratum 2.
- "Optimum" allocation computed with wrong $S_h$ can be worse than proportional. If in truth all $S_h$ are equal, proportional allocation is the Neyman allocation. Any allocation based on unequal guessed $S_h$ is then strictly worse, by the equality case in Theorem 8.7.
- Systematic sampling has no unbiased variance estimator. Units in different clusters have $\pi_{ij}=0$, and a single cluster gives no information about the spread of the $\bar y_r$ (Exercise IV.2). Treating the sample as SRS overstates the variance under a trend and badly understates it under periodicity.
- Linear systematic sampling is biased when $N\ne nk$. For $N=7$, $k=3$ and values $1,1,1,1,1,1,8$, the samples $\{1,4,7\},\{2,5\},\{3,6\}$ have means $\frac{10}3,1,1$, which average to $1.78\ne\bar Y=2$. Circular systematic sampling repairs this.
- Systematic is not "SRS in disguise". Each unit has the same $\pi_i=n/N$ as in SRS, but the pairwise probabilities differ completely. Two designs with the same $\pi_i$ can have very different variances (Example 7).
Exercises
- 1 A population has values $3,5,8,12$. List all SRSWOR samples of size $2$, and verify $E\bar y=\bar Y$ and $\operatorname{Var}(\bar y)=(1-f)S^2/n$ by enumeration.
- 2 For the population of I.1, find $\operatorname{Var}(\bar y)$ under SRSWR with $n=2$ and check the ratio of Theorem 8.3.
- 3 Using Theorem 8.1, show directly that $\operatorname{Var}\left(\sum_{i=1}^NZ_i\right)=0$. Why must this be so?
- 4 A sample of $n=60$ from $N=1200$ shops gives mean daily sales $\bar y=48.5$ and $s=15.2$ (thousand rupees). Estimate the total daily sales of all shops and give an approximate $95\%$ confidence interval.
- 5 In an SRSWOR of $200$ households out of $2000$, $46$ own a two-wheeler. Estimate the proportion with its estimated standard error and a $95\%$ interval.
- 6 How large an SRSWOR from $N=3000$ is needed to estimate a mean to within $\pm2$ units with $95\%$ confidence if $S\approx15$?
- 7 The coefficient of variation of household income in a town of $N=10000$ households is about $0.8$. What sample size estimates the mean income within $5\%$ of its true value with $95\%$ confidence?
- 8 Prove Theorem 8.4 directly from the hypergeometric distribution of $a=np$, without using Theorem 8.2.
- 9 Under SRSWR with $n$ draws, show that the number $\nu$ of distinct units has $E[\nu]=N\left[1-(1-\frac1N)^n\right]$. Evaluate it for $N=10$, $n=5$.
- 1 Stratum 1 ($N_1=300$) gave the sample $12,15,9,14,10,18$ and stratum 2 ($N_2=200$) gave $40,52,47,37$. Compute $\bar y_{\mathrm{st}}$, $v(\bar y_{\mathrm{st}})$ and the estimated standard error.
- 2 Strata of sizes $1500,900,600$. Find the proportional allocation of $n=60$.
- 3 For the strata of II.2 with $S_h=8,20,40$, find the Neyman allocation of $n=54$.
- 4 In II.3 compute $V_{\mathrm{prop}}$ and $V_{\mathrm{Ney}}$ for $n=54$, and check that their difference is $\frac1n\sum W_h(S_h-\bar S)^2$.
- 5 In II.3 the unit costs are $c_h=4,9,16$. Find the cheapest allocation achieving $\operatorname{Var}(\bar y_{\mathrm{st}})\le4$ and its cost.
- 6 Two strata have $N_h=(50,950)$ and $S_h=(100,2)$, and $n=100$. Show that the Neyman rule is infeasible and find the optimal feasible allocation.
- 7 Prove that $V_{\mathrm{prop}}-V_{\mathrm{Ney}}=\frac1n\sum W_h(S_h-\bar S)^2$ and decide when it is zero.
- 8 Prove: if the stratum means are all equal and all $S_h$ are equal, then $V_{\mathrm{prop}}\ge V_{\mathrm{ran}}$, with equality only if $L=1$ or $f=1$.
- 9 (Applied.) A household expenditure survey has an urban stratum ($W=0.3$, mean $15000$, $S=4000$) and a rural one ($W=0.7$, mean $8000$, $S=2000$). Ignoring the fpc, with $n=500$, compute the variances under SRS, proportional and Neyman allocation, the two design effects, and the Neyman allocation.
- 1 $N=12$ values in list order: $4,7,9,6,8,11,13,10,12,15,17,14$, with $n=3$ ($k=4$). List the four systematic samples, compute $V_{\mathrm{sys}}$, $V_{\mathrm{ran}}$ and $S^2_{\mathrm{wsy}}$, and check Theorem 8.10.
- 2 For $Y_i=i$, $N=15$, $n=3$, compute $V_{\mathrm{ran}}$, $V_{\mathrm{sys}}$ and $V_{\mathrm{st}}$ from Theorem 8.11.
- 3 Show that $V_{\mathrm{sys}}\lt V_{\mathrm{ran}}$ is equivalent to $\rho_w\lt-1/(N-1)$, and interpret the sign of $\rho_w$.
- 4 Circular systematic sampling with $N=11$, $n=3$, $k=4$: list the sample for $r=9$ and show that every unit has inclusion probability $3/11$.
- 5 Show that linear systematic sampling with $N=7$, $k=3$ gives a biased sample mean for the values $1,1,1,1,1,1,8$. Is it biased for $Y_i=i$?
- 6 A population of $N=36$ repeats the pattern $1,3,5,5,3,1$. Find $V_{\mathrm{sys}}$ and $V_{\mathrm{ran}}$ for $k=6$, $k=3$ and $k=4$.
- 7 Show that for $Y_i=a+bi$, Yates' corrected estimator $\bar y_r+\frac{2r-k-1}{2(n-1)k}\left(y_{r1}-y_{rn}\right)$ equals $\bar Y$ for every start $r$.
- 8 Suppose the list order is a uniformly random permutation of the population (independent of the start). Show that the systematic sample is then an SRSWOR, and deduce that $V_{\mathrm{sys}}$, averaged over all orderings, equals $V_{\mathrm{ran}}$.
- 1 Prove that under SRSWOR $s^2=\frac1{n(n-1)}\sum_{j\ne j'}\frac{(y_j-y_{j'})^2}{2}$ and $S^2=\frac1{N(N-1)}\sum_{i\ne i'}\frac{(Y_i-Y_{i'})^2}{2}$, and deduce $E s^2=S^2$ from Theorem 8.1.
- 2 Prove that in linear systematic sampling no estimator $v$ (a function of the sample) can be unbiased for $\operatorname{Var}(\bar y_{\mathrm{sys}})$ for every population $(Y_1,\dots,Y_N)$, when $n\ge2$ and $k\ge2$.
- 3 Under SRSWOR of pairs $(y_i,x_i)$, prove $\operatorname{Cov}(\bar y,\bar x)=\frac{(1-f)S_{xy}}{n}$, where $S_{xy}=\frac1{N-1}\sum(Y_i-\bar Y)(X_i-\bar X)$. (This is used in §8.2.)
- 4 Post-stratification. An SRSWOR of size $n$ is drawn and afterwards classified into strata with known $W_h$. Show that, conditional on the realised stratum counts $(n_1,\dots,n_L)$ with all $n_h\ge1$, the estimator $\sum W_h\bar y_h$ is unbiased and its conditional variance is $\sum W_h^2(1-n_h/N_h)S_h^2/n_h$.
- 5 Give an example of a population, a stratification and an allocation for which $V(\bar y_{\mathrm{st}})\gt V_{\mathrm{ran}}$ by a large factor (not only by a term of order $1/N$).
- 6 Research-flavoured. For cost $C=c_0+c_1\sqrt{n}$ (travel between sampled units grows like $\sqrt n$), find the $n$ that minimises $\operatorname{Var}(\bar y)+\lambda C$ under SRS with negligible fpc, and discuss how it differs from the linear-cost answer.
E1. CSIR Part B GATE ST An SRSWOR of size $10$ is drawn from a population of $50$ units with $S^2=\frac1{49}\sum(Y_i-\bar Y)^2=40$. The variance of the sample mean is (A) $4.00$; (B) $3.92$; (C) $3.20$; (D) $0.80$.
E2. CSIR Part B A population has two strata with $N_1=400$, $S_1=5$ and $N_2=100$, $S_2=30$. A stratified sample of total size $50$ is to be allocated by Neyman allocation. Then $(n_1,n_2)$ is (A) $(40,10)$; (B) $(25,25)$; (C) $(20,30)$; (D) $(10,40)$.
E3. CSIR Part B GATE ST The population values are $Y_i=i$, $i=1,\dots,30$. A linear systematic sample of size $5$ is drawn. The variance of the sample mean is (A) $35/60$; (B) $35/12$; (C) $3$; (D) $155/12$.
E4. CSIR Part C Under SRSWOR of size $n$ ($2\le n\lt N$) from a population with $S^2\gt0$, which of the following are true? (One or more options may be correct.) (A) $(1-f)s^2/n$ is unbiased for $\operatorname{Var}(\bar y)$. (B) The sample standard deviation $s$ is unbiased for $S$. (C) $\operatorname{Var}(\bar y)$ is no larger than under SRSWR with the same $n$. (D) The probability that two given units are both in the sample is $n^2/N^2$.
E5. CSIR Part C In stratified random sampling with fixed total size $n$, which of the following are true? (A) Under proportional allocation, $\bar y_{\mathrm{st}}$ equals the mean of all $n$ sampled values. (B) For every population and every stratification, proportional allocation gives a variance no larger than SRSWOR of size $n$. (C) Whenever the Neyman sizes satisfy $n_h\le N_h$ (treated as real numbers), Neyman allocation gives a variance no larger than proportional allocation. (D) If all $S_h$ are equal, Neyman allocation coincides with proportional allocation.
E6. CSIR Part C Let $N=nk$ with $n,k\ge2$, and consider linear systematic sampling with interval $k$. Which of the following are true? (A) $\bar y_{\mathrm{sys}}$ is unbiased for $\bar Y$. (B) If the units are listed in a uniformly random order, then the average of $\operatorname{Var}(\bar y_{\mathrm{sys}})$ over all orderings equals the SRSWOR variance $(1-f)S^2/n$. (C) There is an estimator, computed from the single systematic sample, that is unbiased for $\operatorname{Var}(\bar y_{\mathrm{sys}})$ for every population. (D) If the population is periodic with period exactly $k$ and not constant, then $\operatorname{Var}(\bar y_{\mathrm{sys}})\gt(1-f)S^2/n$.
Suggested plan — 3 lectures × 60 min.
- Simple random sampling. The finite-population viewpoint (fixed $Y_i$, random selection). Theorem 8.1 by counting, then the indicator proof of Theorem 8.2 in full. Do Example 1 on the board (students compute two or three of the ten samples each). Theorem 8.3 and the "a sample of 1000 suffices for a country" discussion, proportions, and the sample-size formula with Example 2.
- Stratification. Definition 8.2 and Theorem 8.6. Derive the optimum allocation by Cauchy–Schwarz (Theorem 8.7), then Example 4 with the class filling in the table. Theorem 8.8 and Example 5 as the "surprise". Project Interactive 1: first $\delta=0$ (proportional ≈ SRS, Neyman still better), then $\delta=12$.
- Systematic sampling. Definition 8.4. Theorem 8.9 via the ANOVA decomposition, and Theorems 8.10 and 8.11. Examples 6 and 7. Project Interactive 2 with the period-6 population at $k=6$, $3$ and $4$, then the trend. Finish with the ⚠️ box and E4–E6.
Discussion questions. (1) Why does the precision of a national opinion poll hardly depend on the size of the country? When does it start to matter? (2) A colleague proposes stratifying by an outcome recorded last year. Is that allowed, and what does Theorem 8.8 say about the gain? (3) You must estimate the average waiting time at a clinic from a list of appointments, sorted by time of day. Systematic or simple random sampling, and why?
Misconceptions to watch for. Using $\sigma^2/n$ under SRSWOR; thinking the fpc is about the size of $N$ instead of $f=n/N$; averaging all stratified observations equally under Neyman allocation; believing stratification can never hurt; computing a "variance" from a single systematic sample as if it were SRS; choosing $k$ equal to a known period (weekly sales sampled every 7th day).
Suggested homework. I(1, 4, 5, 6, 9), II(1, 3, 4, 5, 9), III(1, 2, 6, 8), IV(1, 3); Exam corner E2, E4, E6.
Research question. An air-quality agency can afford only a few laboratory analyses per month. Should it sample the hourly PM2.5 record systematically (every $k$-th hour) or at random? For which intervals $k$ does systematic sampling become dangerous?
Why it matters. Pollution, electricity demand, hospital admissions and road traffic all have daily and weekly cycles. Monitoring networks in Delhi, Beijing, London and Mexico City sample filters on fixed schedules. A schedule in phase with a cycle can bias a monthly average by more than the regulatory margin, while a well-chosen interval can beat random sampling.
What to do.
- Build an hourly series for $30$ days ($N=720$). Either download one month of hourly PM2.5 for a city of your choice from an open air-quality portal, or simulate $Y_t=60+25\cos(2\pi t/24)+10\cos(4\pi t/24)+0.02t+\varepsilon_t$ with $\varepsilon_t\sim N(0,8^2)$ from a fixed seed.
- For every divisor $k$ of $720$ with $2\le k\le 72$, compute $V_{\mathrm{sys}}$ exactly from the $k$ possible samples (Theorem 8.9) and $V_{\mathrm{ran}}=(k-1)S^2/N$. Plot the ratio $V_{\mathrm{sys}}/V_{\mathrm{ran}}$ against $k$.
- Explain the spikes using Theorem 8.10: what happens to $S^2_{\mathrm{wsy}}$ when $k$ is a multiple of $24$? Of $12$?
- Propose a design with the same number of analyses that is safe against unknown cycles, for example stratified sampling with one random hour per day, and compare its variance.
Tools: Python with numpy (or a spreadsheet); an open air-quality data portal if you use real data.
Hand in: The ratio plot, a two-paragraph explanation of the spikes, and your recommended sampling schedule with its variance.
Going further. Read about Cochran's "autocorrelated populations" model, under which the expected $V_{\mathrm{sys}}$ can be compared with stratified and random sampling when the correlogram is convex. Relate it to the sign of $\rho_w$.
🧪 Self-check quiz
Q1. Under SRSWOR of size $n$ from $N$ units, $\operatorname{Var}(\bar y)$ equals
Q2. Under SRSWR, the sample variance $s^2$ (divisor $n-1$) is unbiased for
Q3. With equal unit costs and fixed $n$, the stratified variance is minimised by
Q4. For a population with an exact linear trend ($N=nk$), the variances of the mean satisfy
Q5. Systematic sampling is more precise than SRSWOR of the same size exactly when