13.3 Variance and Standard Deviation
Square the deviations instead of dropping their signs: the mean of the squared deviations is the variance $\sigma^2$, and its square root, the standard deviation $\sigma$, is the measure of spread used across science, finance and industry.
The mean deviation fixed the problem of cancelling deviations by taking absolute values. It works, but the modulus is awkward to handle in algebra: $|x-a|$ has a corner at $x=a$, the sum $\sum|x_i-a|$ has to be split into cases, and there is no neat formula that turns $\sum|x_i-\bar x|$ into something simpler. There is a second way to make every deviation positive: square it. Squares are never negative, they are smooth, and $(x_i-\bar x)^2$ can be expanded by ordinary algebra.
Averaging the squared deviations gives the variance, and taking its square root brings us back to the units of the data: the standard deviation. It is the number a pharmaceutical company quotes for the variation in the amount of medicine in each tablet, that an engineer uses to set tolerances on machined parts, that a weather scientist uses to say how unusual a monsoon was, and that an investor uses to measure how risky a share is. In this section we compute it for lists, for discrete frequency tables and for class-interval tables, and we develop the shortcut that makes large tables manageable.
Variance and Standard Deviation of a List
The variance of the observations $x_1,x_2,\dots,x_n$ with mean $\bar x$ is the mean of the squared deviations from the mean: $$\sigma^2=\frac1n\sum_{i=1}^{n}(x_i-\bar x)^2.$$ The standard deviation is its non-negative square root, $$\sigma=\sqrt{\frac1n\sum_{i=1}^{n}(x_i-\bar x)^2}.$$ $\sigma$ has the same unit as the data ($\sigma^2$ has the square of that unit); $\sigma=0$ exactly when all the observations are equal.
Squaring gives large deviations extra weight: a deviation of $4$ contributes $16$, four times as much as a deviation of $2$. So $\sigma$ reacts more strongly than the mean deviation to observations far from the mean, and for any data $\sigma\ge\text{M.D.}(\bar x)$. (We always divide by $n$ in this course. Some calculators and spreadsheets also offer a version that divides by $n-1$, used when the data are a sample from a larger population; you will meet it in later statistics courses.)
| $x_i$ | 6 | 9 | 3 | 12 | 10 |
|---|---|---|---|---|---|
| $x_i-\bar x$ | โ2 | 1 | โ5 | 4 | 2 |
| $(x_i-\bar x)^2$ | 4 | 1 | 25 | 16 | 4 |
When the mean is not a whole number, the squared deviations become messy decimals. Expanding the square gives a formula that needs only $\sum x_i$ and $\sum x_i^2$.
For observations $x_1,\dots,x_n$ with mean $\bar x$, $$\sigma^2=\frac1n\sum_{i=1}^{n}x_i^2-\bar x^{\,2},\qquad\text{equivalently}\qquad \sigma=\frac1n\sqrt{n\sum x_i^2-\left(\sum x_i\right)^2}.$$
Proof. Expand each square: $(x_i-\bar x)^2=x_i^2-2\bar x\,x_i+\bar x^{\,2}$. Adding over $i$, and using $\sum x_i=n\bar x$, $$\sum(x_i-\bar x)^2=\sum x_i^2-2\bar x\sum x_i+n\bar x^{\,2}=\sum x_i^2-2n\bar x^{\,2}+n\bar x^{\,2}=\sum x_i^2-n\bar x^{\,2}.$$ Divide by $n$ for the first form. For the second, multiply $\sigma^2=\tfrac1n\sum x_i^2-\tfrac{1}{n^2}\left(\sum x_i\right)^2$ by $n^2$ and take square roots. $\blacksquare$
Two consequences are used constantly: $\sum x_i^2=n\left(\sigma^2+\bar x^{\,2}\right)$, which recovers the sum of squares from a given mean and variance; and $\sigma^2\ge0$ gives $\tfrac1n\sum x_i^2\ge\bar x^{\,2}$ โ the mean of the squares is never less than the square of the mean.
Frequency Distributions and the Shortcut Method
For a frequency table we weight each squared deviation by its frequency, exactly as we weighted the distances in Section 13.2: with $N=\sum f_i$, $$\sigma^2=\frac1N\sum f_i(x_i-\bar x)^2=\frac1N\sum f_ix_i^2-\bar x^{\,2},$$ where for class intervals $x_i$ is the class mark. (The second form follows by the same expansion as Theorem 13.5, with every sum weighted by $f_i$.) For a table with large class marks the squares $x_i^2$ are huge; coding the class marks as small integers makes the work light.
Let the class marks (or values) $x_i$ be equally spaced with common difference $h$, let $A$ be any one of them, and put $u_i=\dfrac{x_i-A}{h}$. Then $$\sigma_x^2=h^2\,\sigma_u^2=h^2\left[\frac{\sum f_iu_i^2}{N}-\left(\frac{\sum f_iu_i}{N}\right)^2\right],$$ that is, $$\sigma_x=\frac hN\sqrt{N\sum f_iu_i^2-\left(\sum f_iu_i\right)^2}.$$
Proof. $x_i=A+hu_i$, and by Theorem 13.3 $\bar x=A+h\bar u$, where $\bar u=\tfrac1N\sum f_iu_i$. Subtracting, $x_i-\bar x=h(u_i-\bar u)$, so $(x_i-\bar x)^2=h^2(u_i-\bar u)^2$. Multiply by $f_i$, add and divide by $N$: $\sigma_x^2=h^2\sigma_u^2$. Finally apply Theorem 13.5 (weighted by $f_i$) to the $u_i$: $\sigma_u^2=\tfrac1N\sum f_iu_i^2-\bar u^{\,2}$. $\blacksquare$
The proof shows why the method works: subtracting $A$ slides every value and the mean by the same amount, which changes no deviation; dividing by $h$ shrinks every deviation by the factor $h$, so the variance shrinks by $h^2$ and we multiply back at the end. Whatever $A$ you choose, the answer is the same (try it in Interactive 2).
| Marks | 0โ10 | 10โ20 | 20โ30 | 30โ40 | 40โ50 | 50โ60 | 60โ70 |
|---|---|---|---|---|---|---|---|
| Students | 3 | 6 | 11 | 14 | 8 | 5 | 3 |
| Class | $f_i$ | $x_i$ | $u_i$ | $f_iu_i$ | $f_iu_i^2$ |
|---|---|---|---|---|---|
| 0โ10 | 3 | 5 | โ3 | โ9 | 27 |
| 10โ20 | 6 | 15 | โ2 | โ12 | 24 |
| 20โ30 | 11 | 25 | โ1 | โ11 | 11 |
| 30โ40 | 14 | 35 | 0 | 0 | 0 |
| 40โ50 | 8 | 45 | 1 | 8 | 8 |
| 50โ60 | 5 | 55 | 2 | 10 | 20 |
| 60โ70 | 3 | 65 | 3 | 9 | 27 |
| Total | 50 | โ5 | 117 |
1. $\left(\sum x_i\right)^2$ for $\sum x_i^2$. $\sum x_i^2$ squares first and then adds ($1^2+2^2=5$); $\left(\sum x_i\right)^2$ adds first ($3^2=9$).
2. Forgetting to subtract $\bar x^{\,2}$ in Theorem 13.5, or subtracting $\bar x$ instead of $\bar x^{\,2}$.
3. Not multiplying back by $h^2$ (or $h$ for $\sigma$) in the shortcut method. $\sigma_u$ is the spread of the coded values, not of the data.
4. Reporting the variance as the spread. If marks are out of $70$, $\sigma^2=233$ "marksยฒ" is not a typical distance; $\sigma=15.26$ marks is.
5. A negative variance. $\sigma^2\ge0$ always. A negative answer from $\tfrac1n\sum x_i^2-\bar x^{\,2}$ means an arithmetic slip โ often a rounded $\bar x$ squared too early. Keep full accuracy until the end.
6. Correcting only $\sum x_i$. When an observation is corrected, update both $\sum x_i$ and $\sum x_i^2$ (Example 4).
Not in the CBSE syllabus, and not in JEE Main (which lists only mean deviation, variance and standard deviation); JEE Advanced lists the analysis of frequency distributions with the same mean but different variances, and the coefficient of variation is the tool for it. A standard deviation of $10$ runs means a lot for a batsman who averages $15$ and little for one who averages $80$. To compare the variability of data with different means (or different units), divide the standard deviation by the mean: $$\text{C.V.}=\frac{\sigma}{\bar x}\times100\%\qquad(\bar x\gt0).$$ The series with the smaller C.V. is the more consistent. (The proof of Theorem 13.6 also shows that if $y_i=ax_i+b$ then $\bar y=a\bar x+b$ and $\sigma_y=|a|\,\sigma_x$: adding a constant changes neither $\sigma$ nor the variance.)
Worked example. Over a season batsman P scores a mean of $50$ runs per innings with $\sigma=15$; batsman Q a mean of $40$ with $\sigma=10$. Who is more consistent?
Solution. C.V.(P) $=\tfrac{15}{50}\times100=30\%$; C.V.(Q) $=\tfrac{10}{40}\times100=25\%$. Q is more consistent, although P scores more on average. Check: P's typical swing is $\tfrac{3}{10}$ of his average, Q's only $\tfrac14$ of his โ.
Exercises
- 1 $2, 4, 6, 8, 10$
- 2 $5, 12, 3, 18, 6, 8, 2, 10$
- 3 the first ten multiples of $3$
- 4 $1.2, 2.4, 1.8, 3.0, 2.1$
- 5 $3, 7, 8, 12, 15$, using $\sum x_i$ and $\sum x_i^2$
- 6 $10, 20, 30, 40, 50$
- 7 Six observations have mean $7$ and variance $7$. Four of them are $3, 6, 8, 11$. Find the other two.
- 8 The mean and variance of $10$ observations were found to be $15$ and $9$. Later it was noticed that the observation $21$ had been read as $12$. Find the correct mean and standard deviation.
- 1
$x_i$ 4 8 12 16 20 $f_i$ 3 6 10 7 4 - 2
$x_i$ 1 2 3 4 5 $f_i$ 4 8 12 10 6 - 3
Class 0โ10 10โ20 20โ30 30โ40 40โ50 $f_i$ 5 8 15 16 6 - 4 Daily wages of $50$ workers:
Wage (โน) 100โ200 200โ300 300โ400 400โ500 500โ600 Workers 8 12 20 6 4 - 5 Heights of $50$ students:
Height (cm) 140โ145 145โ150 150โ155 155โ160 160โ165 165โ170 Students 4 9 15 12 7 3 - 6
Class 0โ20 20โ40 40โ60 60โ80 80โ100 $f_i$ 5 12 20 9 4 - 7 For $20$ observations, $\sum(x_i-10)=12$ and $\sum(x_i-10)^2=180$.
- 8 For $40$ observations, $\sum x_i=200$ and $\sum x_i^2=1160$.
- 1 The daily screen times (in hours) of eight students were $2, 3, 3, 4, 5, 6, 6, 7$. Find the mean and the standard deviation.
- 2 For $1, 2, 3, 4, 5$ find the standard deviation and the mean deviation about the mean, and check that $\sigma\ge\text{M.D.}(\bar x)$.
- 3 Prove that $\sigma=0$ if and only if all the observations are equal.
- 4 In a test the mean mark is $60$ and the standard deviation is $8$. How many standard deviations above the mean is a mark of $76$? Which mark is $1.5$ standard deviations below the mean?
- 5 For $15$ observations the sum of the squares of the deviations from their mean is $240$. Find the standard deviation.
- 6 Find the standard deviation of the volumes (ml) filled by machine P in Section 13.1: $498, 503, 500, 497, 502, 501, 499, 500$.
- 7 For the screen-time data of part 1, how many of the eight observations lie within one standard deviation of the mean, i.e. in $[\bar x-\sigma,\ \bar x+\sigma]$?
- 8 Show that $\sum x_i^2\ge n\bar x^{\,2}$ for any observations, with equality only when they are all equal.
- 9 Data on rainfall are in millimetres. In what unit is the variance? Why do weather reports quote the standard deviation instead?
How this is examined: the Board's long question is usually the mean and standard deviation of a class-interval table by the shortcut method (5 marks); shorter questions correct a wrongly recorded observation or work back from a given mean and variance. JEE Main asks for missing observations from $\bar x$ and $\sigma^2$; JEE Advanced, whose syllabus includes comparing distributions with the same mean but different variances, can also ask for coefficients of variation (see the ๐ box).
E1. Board ยท 1 mark The variance of $1, 3, 5, 7$ is (A) $\sqrt5$ (B) $5$ (C) $20$ (D) $4$
E2. Board ยท 2 marks The mean and variance of $10$ observations are $12$ and $4$. Find $\sum x_i$ and $\sum x_i^2$.
E3. Board ยท 3 marks The mean and standard deviation of $10$ observations were found to be $50$ and $5$. On checking, it was found that the observation $45$ had been copied as $54$. Find the correct mean and standard deviation.
E4. Board ยท 5 marks Find the mean and standard deviation of the following distribution of the weights (kg) of $50$ adults.
| Weight | 30โ40 | 40โ50 | 50โ60 | 60โ70 | 70โ80 | 80โ90 |
|---|---|---|---|---|---|---|
| Adults | 3 | 7 | 12 | 15 | 8 | 5 |
E5. JEE Main Six observations have mean $5$ and variance $6$. Four of them are $1, 4, 6, 9$. The other two are (A) $4$ and $6$ (B) $3$ and $7$ (C) $2$ and $8$ (D) $5$ and $5$
E6. JEE Advanced Only one option correct. The mean of a distribution is $20$ and its coefficient of variation is $25\%$. Its variance is (A) $25$ (B) $5$ (C) $4$ (D) $16$
Research question. Has the monsoon rainfall in your region become more variable over the last few decades?
Why it matters. Farmers, reservoir engineers and disaster-management agencies plan for the spread of rainfall, not just its average: a district that receives its average every year can plan crops easily, while one that swings between drought and flood cannot. Climate scientists in India and worldwide track the standard deviation of seasonal rainfall to detect such changes.
What to do.
- Obtain the JuneโSeptember (monsoon) rainfall totals for your state or district for at least $30$ consecutive years โ public meteorological data portals publish them, or your teacher can provide a table.
- Split the years into three decades. For each decade find the mean and the standard deviation (Theorem 13.5 in a spreadsheet: columns $x$ and $x^2$).
- Mark on a time-series graph the band $\bar x\pm\sigma$ for the whole period, and list the years outside it. Are they spread evenly, or bunched in one decade?
- Group all the years into classes of width $100\ \text{mm}$ and find $\sigma$ by the shortcut method (Theorem 13.6). How close is it to the value from the raw data?
Tools: A spreadsheet, a public rainfall data table, graph paper or a plotting app
Hand in: The data table, the decade-by-decade means and standard deviations, the time-series graph with the $\bar x\pm\sigma$ band, the grouped calculation, and a paragraph answering the research question with the evidence and its limits.
Going further. A decade with a smaller mean and a smaller $\sigma$ may still be relatively more variable. If you are preparing for JEE Advanced, read the ๐ box and compare the decades using the relative measure introduced there.
๐งช Self-check quiz
Q1. The standard deviation of $7, 7, 7, 7, 7$ is
Q2. The variance of $1, 3, 5$ is
Q3. For $5$ observations, $\sum x_i^2=50$ and $\bar x=3$. The variance is
Q4. In the shortcut method with class width $h=5$, the coded values $u_i$ have standard deviation $1.2$. The standard deviation of the data is
Q5. Heights are measured in cm. The variance of the heights is measured in