13.2 Probability and Bayes' Theorem
Probability measures how likely an event is. From sample spaces and events we build the addition theorem, conditional probability and the multiplication theorem, independent events, the theorem of total probability and Bayes' theorem — the rule for reasoning backwards from evidence to its causes.
A screening test for a disease detects $95\%$ of people who have it, and wrongly flags $10\%$ of people who do not. One person in a hundred in the population has the disease. Your test comes back positive. How worried should you be? Most people — including, in published surveys, many doctors — answer "about $95\%$". The correct answer is under $9\%$. Picture $1000$ people: about $10$ have the disease and about $9.5$ of them test positive, but $10\%$ of the $990$ healthy people, about $99$, test positive too. Of roughly $108.5$ positives, only $9.5$ are ill: $\tfrac{9.5}{108.5}\approx8.8\%$.
The mistake is to confuse "the probability of a positive test given disease" with "the probability of disease given a positive test". Keeping the two apart is the business of conditional probability, and turning one into the other is Bayes' theorem. In this section we build the whole toolkit from the beginning: sample spaces and events, the probability of an event, the addition theorem, conditional probability and the multiplication theorem, independent events, the theorem of total probability and Bayes' theorem. JEE Main sets two or three probability questions in most papers; counting from Chapter 4 does much of the work in many of them.
Sample Spaces, Events and the Probability of an Event
A random experiment has more than one possible outcome, all known in advance, but which one occurs cannot be predicted. The set $S$ of all its outcomes is the sample space; an event is a subset of $S$. An event with one outcome is elementary; $S$ itself is the sure event and $\phi$ the impossible event. The event "$A$ does not occur" is the complement $A'=S-A$; "$A$ or $B$" is $A\cup B$; "$A$ and $B$" is $A\cap B$.
Events $A$ and $B$ are mutually exclusive if $A\cap B=\phi$ (they cannot occur together). Events $E_1,\dots,E_n$ are exhaustive if $E_1\cup\dots\cup E_n=S$ (at least one must occur).
For two coins, $S=\{HH,HT,TH,TT\}$; for two dice, $S$ is the $36$ ordered pairs $(a,b)$. The set language of Section 1.1 is exactly what we need: De Morgan's law, for example, says "not ($A$ or $B$)" is "not $A$ and not $B$", $(A\cup B)'=A'\cap B'$.
Classical definition. If $S$ is finite and all its outcomes are equally likely, then $P(A)=\dfrac{n(A)}{n(S)}$ — favourable outcomes over all outcomes.
Axiomatic definition. In general a probability is a rule $P$ that gives each event $A$ a number with (i) $P(A)\ge0$, (ii) $P(S)=1$, and (iii) $P(A\cup B)=P(A)+P(B)$ whenever $A$ and $B$ are mutually exclusive. The classical rule satisfies all three.
The odds in favour of $A$ are $P(A):P(A')$; odds of $a:b$ in favour mean $P(A)=\tfrac{a}{a+b}$.
For all events $A$, $B$: $\ P(\phi)=0$; $\ P(A')=1-P(A)$; $\ 0\le P(A)\le1$; and if $A\subseteq B$ then $P(A)\le P(B)$. Also $P(A\cap B')=P(A)-P(A\cap B)$.
Proof. $A$ and $A'$ are mutually exclusive with union $S$, so $P(A)+P(A')=P(S)=1$; with $A=S$ this gives $P(\phi)=0$, and $P(A')\ge0$ gives $P(A)\le1$. If $A\subseteq B$ then $B$ is the disjoint union of $A$ and $B\cap A'$, so $P(B)=P(A)+P(B\cap A')\ge P(A)$. Finally $A$ is the disjoint union of $A\cap B$ and $A\cap B'$. $\blacksquare$ The complement rule is the most useful line in this section: "at least one" is almost always easier as $1-P(\text{none})$.
Two fair coins are tossed. A student says: "There are three outcomes — two heads, two tails, one of each — so $P(\text{one head and one tail})=\tfrac13$." What is wrong?
The Addition Theorem
Axiom (iii) handles "or" for mutually exclusive events. When $A$ and $B$ can occur together, adding $P(A)$ and $P(B)$ counts the overlap twice — exactly the problem we solved for counting in Section 1.1.
For any events $A$, $B$, $C$:
$$P(A\cup B)=P(A)+P(B)-P(A\cap B),$$ $$\begin{aligned}P(A\cup B\cup C)&=P(A)+P(B)+P(C)-P(A\cap B)\\&\quad-P(B\cap C)-P(C\cap A)+P(A\cap B\cap C).\end{aligned}$$Also $P(\text{exactly one of }A,B)=P(A)+P(B)-2P(A\cap B)$ and $P(\text{neither})=1-P(A\cup B)$.
Proof. $A\cup B$ is the disjoint union of $A$ and $B\cap A'$, and $P(B\cap A')=P(B)-P(A\cap B)$ by Theorem 13.6; add. The three-event form follows by applying the two-event form to $A\cup B$ and $C$ and then using the distributive law $(A\cup B)\cap C=(A\cap C)\cup(B\cap C)$. "Exactly one" is $A\cup B$ without $A\cap B$. $\blacksquare$ Because $P(A\cup B)\le1$, the theorem also gives the bounds
$$P(A)+P(B)-1\le P(A\cap B)\le\min\{P(A),P(B)\}.$$$P(A)=0.7$ and $P(B)=0.6$. The least possible value of $P(A\cap B)$ is
Conditional Probability, the Multiplication Theorem and Independence
Knowing that one event has happened changes the chances of another. If a die shows an even number, the chance that it is a $6$ rises from $\tfrac16$ to $\tfrac13$: the sample space has shrunk to $\{2,4,6\}$.
For events $A$, $B$ with $P(B)\gt0$, the probability of $A$ given $B$ is
$$P(A\,|\,B)=\frac{P(A\cap B)}{P(B)}.$$For equally likely outcomes this is $\tfrac{n(A\cap B)}{n(B)}$: count inside the new, smaller sample space $B$. For fixed $B$, $P(\,\cdot\,|\,B)$ obeys all the rules of probability; for example $P(A'|B)=1-P(A|B)$.
whenever the conditioning events have positive probability.
Proof. The first line is Definition 13.9 multiplied out. For three events apply it twice: $P(A\cap B\cap C)=P(A\cap B)\,P(C|A\cap B)$ and $P(A\cap B)=P(A)P(B|A)$. $\blacksquare$ This is the rule for probability trees: along a path, multiply the branch probabilities (each one conditional on the path so far); for an event made of several paths, add the paths.
$A$ and $B$ are independent if $P(A\cap B)=P(A)\,P(B)$; equivalently (when $P(B)\gt0$) $P(A|B)=P(A)$ — knowing $B$ does not change the chance of $A$. Three events $A$, $B$, $C$ are mutually independent if they are pairwise independent and $P(A\cap B\cap C)=P(A)P(B)P(C)$.
If $A$ and $B$ are independent, so are $A$ and $B'$, $A'$ and $B$, and $A'$ and $B'$. Hence, for mutually independent $A_1,\dots,A_n$,
$$P(\text{at least one } A_i)=1-P(A_1')\,P(A_2')\cdots P(A_n').$$Proof. $P(A\cap B')=P(A)-P(A\cap B)$ $=P(A)-P(A)P(B)$ $=P(A)\,P(B')$. Applying this twice gives the other pairs. "At least one" is the complement of "none", and $P(\text{none})=P(A_1'\cap\dots\cap A_n')=\prod P(A_i')$. $\blacksquare$ Do not confuse independent with mutually exclusive: if $P(A),P(B)\gt0$ and $A\cap B=\phi$, then $P(A\cap B)=0\ne P(A)P(B)$, so mutually exclusive events of positive probability are never independent — if one happens, the other certainly does not.
$P(A)=0.3$, $P(B\,|\,A)=0.8$ and $P(B\,|\,A')=0.2$. Event $B$ has occurred. What is $P(A\,|\,B)$, approximately?
$A$ and $B$ are events with $P(A)=0.4$, $P(B)=0.5$ and $P(A\cap B)=0.2$. Which statement is true?
Total Probability and Bayes' Theorem
Many experiments happen in two stages: a factory chooses a machine, then the machine makes a good or a faulty item; a person is ill or well, then the test is positive or negative. The first stage splits the sample space into exhaustive, non-overlapping cases $E_1,\dots,E_n$ (mutually exclusive and exhaustive events with positive probabilities). The theorem of total probability runs the experiment forwards; Bayes' theorem runs it backwards.
If $E_1,\dots,E_n$ are mutually exclusive and exhaustive events with $P(E_i)\gt0$, then for any event $A$
$$P(A)=\sum_{i=1}^{n}P(E_i)\,P(A\,|\,E_i).$$Under the same hypotheses, and if $P(A)\gt0$, for each $k=1,\dots,n$
$$P(E_k\,|\,A)=\frac{P(E_k)\,P(A\,|\,E_k)}{\sum_{i=1}^{n}P(E_i)\,P(A\,|\,E_i)}.$$$P(E_k)$ is the prior probability of the cause $E_k$ and $P(E_k|A)$ its posterior probability after $A$ has been observed.
Proof. Since the $E_i$ are exhaustive and mutually exclusive, $A$ is the disjoint union of the events $A\cap E_i$, so $P(A)=\sum P(A\cap E_i)=\sum P(E_i)P(A|E_i)$ by the multiplication theorem. Then $P(E_k|A)=\tfrac{P(E_k\cap A)}{P(A)}=\tfrac{P(E_k)P(A|E_k)}{P(A)}$. $\blacksquare$ On a tree, Bayes' theorem says: the posterior probability of a cause is the probability of its path to $A$, divided by the total probability of all paths to $A$. In the screening test, the cause "ill" has a strong link to "positive" but a tiny prior, and the cause "well" has a weak link but a huge prior, so the healthy path can easily carry more probability.
A disease affects $1\%$ of people. A test has sensitivity $95\%$ ($P(+\,|\,\text{ill})=0.95$) and specificity $90\%$ ($P(-\,|\,\text{well})=0.90$). A randomly chosen person tests positive. The probability that they are ill is about
1. $P(A|B)$ is not $P(B|A)$. The test's accuracy $P(+|\text{ill})$ and the patient's worry $P(\text{ill}|+)$ can be wildly different. Bayes' theorem is the only bridge between them, and it needs the prior.
2. Independent is not mutually exclusive. Independence is $P(A\cap B)=P(A)P(B)$; mutual exclusion is $P(A\cap B)=0$. For events of positive probability they can never both hold.
3. Three events. Pairwise independence does not imply mutual independence; you also need $P(A\cap B\cap C)=P(A)P(B)P(C)$ (Exercise III.8).
4. With or without replacement. Without replacement the second draw is conditional on the first ($\tfrac47$, not $\tfrac58$); with replacement the draws are independent. Read the question.
5. "At least one" and alternating games. Use $1-P(\text{none})$, not a sum of cases; and for "first to succeed wins" games, sum the geometric series $\tfrac{p}{1-q^k}$ for the player who goes first among $k$ players.
Urn I has $1$ red and $1$ white ball; urn II has $3$ red and $1$ white. An urn is chosen at random and a ball drawn from it is red. The probability that urn II was chosen is
Exercises
Dice and coins are fair and cards come from a well-shuffled pack of $52$ unless stated otherwise. Give answers as fractions in lowest terms or as decimals.
- 1 A coin is tossed three times. Find the probability of at least two heads.
- 2 Two dice are thrown. Find the probability that the sum is at least $10$.
- 3 Two dice are thrown. Find the probability that the product of the numbers is odd.
- 4 A card is drawn. Find the probability that it is a face card (jack, queen or king).
- 5 The letters of EQUATION are arranged at random. Find the probability that the five vowels are together.
- 6 Four bulbs are chosen at random from $10$, of which $3$ are defective. Find the probability that none is defective.
- 7 The odds in favour of an event are $3:5$. Find its probability and the odds against it.
- 8 Two different numbers are chosen at random from $1, 2, \dots, 20$. Find the probability that their sum is even.
- 1 $P(A)=0.42$, $P(B)=0.48$, $P(A\cap B)=0.16$. Find $P(A\cup B)$, $P(A'\cap B')$ and $P(A\cap B')$.
- 2 A card is drawn. Find the probability that it is a king or a heart.
- 3 Two dice are thrown. Find the probability of a doublet or a sum of $4$.
- 4 $P(A\cup B)=0.8$, $P(A)=0.5$, $P(B)=0.6$. Find $P(A\cap B)$ and the probability that exactly one of $A$, $B$ occurs.
- 5 A number is chosen at random from $1, \dots, 200$. Find the probability that it is divisible by $6$ or by $8$.
- 6 $P(A)=P(B)=P(C)=0.4$, each pair has $P(\text{both})=0.15$ and $P(A\cap B\cap C)=0.05$. Find the probability that at least one of the three occurs.
- 7 If $P(A)=0.7$ and $P(B)=0.6$, find the least and greatest possible values of $P(A\cap B)$.
- 1 $P(A)=0.6$, $P(B)=0.5$, $P(A\cap B)=0.3$. Find $P(A|B)$ and $P(B|A)$. Are $A$ and $B$ independent?
- 2 A die shows an even number. Find the probability that it is greater than $3$.
- 3 Two cards are drawn without replacement. Find the probability that both are aces.
- 4 $A$ and $B$ are independent with $P(A)=0.3$, $P(B)=0.4$. Find $P(A\cup B)$ and $P(A'\cap B')$.
- 5 Prove that if $A$ and $B$ are independent, so are $A'$ and $B'$.
- 6 Three marksmen hit a target with probabilities $0.5$, $0.6$, $0.8$, independently. Find the probability that the target is hit, and that exactly two of them hit it.
- 7 How many times must a fair coin be tossed so that the probability of at least one head is at least $0.99$?
- 8 Two coins are tossed. $A$: first coin head; $B$: second coin head; $C$: exactly one head. Show that $A$, $B$, $C$ are pairwise independent but not mutually independent.
- 9 $A$ and $B$ take turns throwing two dice, $A$ first; the first to throw a total of $9$ wins. Find the probability that $A$ wins.
- 1 Bag I has $3$ red and $4$ black balls; bag II has $5$ red and $6$ black. A bag is chosen at random and a ball drawn from it. Find the probability that the ball is red.
- 2 In part 1, the ball is red. Find the probability that it came from bag I.
- 3 An insurer covers $2000$ scooter riders, $4000$ car drivers and $6000$ truck drivers, whose accident probabilities are $0.01$, $0.03$ and $0.15$. An insured person has an accident. Find the probability that they ride a scooter.
- 4 Plant A makes $60\%$ of the phones, $1\%$ of them faulty; plant B makes the rest, $2\%$ faulty. Find the probability that a phone is faulty, and that a faulty phone came from B.
- 5 In a four-option MCQ test a student knows the answer with probability $\tfrac34$ and otherwise guesses. Given that the answer is correct, find the probability that the student knew it.
- 6 A disease affects $0.5\%$ of a population. A test detects it with probability $0.99$ and gives a false positive with probability $0.02$. Find the probability that a person who tests positive has the disease.
- 7 A box has three coins: one two-headed, one fair, and one that shows heads with probability $0.75$. A coin is chosen at random and tossed; it shows heads. Find the probability that it is the two-headed coin.
- 8 Modelling. $40\%$ of the emails reaching an inbox are spam. The word "offer" appears in $25\%$ of spam and in $2\%$ of genuine emails. An email contains "offer". Find the probability that it is spam.
E1. JEE Main · MCQ $A$ and $B$ are independent events with $P(A)=0.5$ and $P(A\cup B)=0.8$. Then $P(B)$ is (A) $0.3$ (B) $0.5$ (C) $0.6$ (D) $0.4$
E2. JEE Main · Numerical Two different numbers are chosen at random from $\{1,2,\dots,10\}$. If $p$ is the probability that their product is even, find $45p$. Give the answer to the nearest integer.
E3. JEE Main · MCQ Bag A has $2$ white and $3$ red balls; bag B has $4$ white and $1$ red ball. A die is thrown: if it shows $1$ or $2$, a ball is drawn from bag A, otherwise from bag B. The ball is red. The probability that it came from bag A is (A) $\tfrac25$ (B) $\tfrac35$ (C) $\tfrac13$ (D) $\tfrac34$
E4. JEE Main · Numerical $A$, $B$ and $C$, in that order, throw a fair die in turn, round after round; the first to throw a six wins. If the probability that $C$ wins is $\tfrac mn$ in lowest terms, find $m+n$. Give the answer to the nearest integer.
E5. JEE Main · MCQ For events $A$ and $B$, $P(A)=\tfrac14$, $P(B\,|\,A)=\tfrac12$ and $P(A\,|\,B)=\tfrac14$. Which statement is true? (A) $A$ and $B$ are mutually exclusive (B) $P(A'\,|\,B)=\tfrac12$ (C) $P(B)=\tfrac14$ (D) $A$ and $B$ are independent
E6. JEE Main · Numerical A box contains $5$ coins: one is two-headed and four are fair. A coin is chosen at random and tossed three times, showing heads every time. If $p$ is the probability that the coin is two-headed, find $100p$. Give the answer to the nearest integer.
Research question. When a population is screened for a rare condition, how many of the positive results are false alarms — and what is the most effective way to reduce them: a better test, or a second test?
Why it matters. Screening programmes — newborn screening, cancer screening, airport security, fraud detection in banking — all face the base-rate problem of Interactive 2. Each false positive costs money and anxiety and may lead to unnecessary procedures; each false negative misses a real case. Health agencies decide whom to screen and how to confirm results using exactly the calculation of Theorem 13.11.
What to do.
- Choose a real screening setting and look up (from a public health agency's website) a typical prevalence and the sensitivity and specificity of the test. If you cannot find figures, use prevalence $0.5\%$, sensitivity $90\%$, specificity $95\%$.
- Compute $P(\text{ill}\,|\,+)$ with Bayes' theorem and check it in Interactive 2. Express it as "out of every $100$ positives, about … are real".
- Option 1: improve the specificity to $99\%$. Option 2: keep the test but retest every positive with an independent second test of the same quality. Compute $P(\text{ill}\,|\,{+}{+})$ for option 2.
- For each option estimate, per $100\,000$ people screened, the number of false positives and the number of tests used.
Tools: Interactive 2, a calculator or spreadsheet; optionally a short simulation in Python.
Hand in: A one-page memo to a health officer recommending one option, with your calculations and one sentence on what the "independent second test" assumption means in practice.
Going further. Screening only a high-risk group raises the base rate. Recompute $P(\text{ill}|+)$ when the prevalence in the screened group is $10$ times higher. Which lever — test quality or targeting — has the bigger effect?
🧪 Self-check quiz
Q1. $A$ and $B$ are mutually exclusive with $P(A)=0.3$ and $P(B)=0.5$. Then $P(A\cup B)$ is
Q2. $A$ and $B$ are mutually exclusive and both have positive probability. Then $A$ and $B$ are
Q3. $P(A\cap B)=0.1$ and $P(B)=0.4$. Then $P(A\,|\,B)$ is
Q4. Machine X makes $70\%$ of the items with $1\%$ faulty, machine Y makes $30\%$ with $3\%$ faulty. A faulty item is found. The probability that Y made it is
Q5. A fair die is thrown three times. The probability of at least one six is