Chapter 9 · Reliability, Linear Programming and Queues

9.1 Reliability: Hazard Rates, Censoring and Systems

How long does a component last, does it age, and how reliable is a system built from many of them? The reliability function, hazard rate and cumulative hazard; IFR and DFR laws; exponential, Weibull and gamma lifetimes; series, parallel and $k$-out-of-$n$ systems; and life tests that stop before every item has failed.

A turbine blade, a heart valve, a hard disk and a light bulb share one awkward property: the quantity we care about most is the time until they fail, and we can never wait to see it for every unit. Reliability theory is the part of probability that studies such lifetimes. It asks three questions. How should the risk of failure be described as a unit gets older — does an old unit fail more readily than a new one (wear-out), less readily (early failures weeded out), or neither (pure chance)? How reliable is a system built from many unreliable parts, connected in series, in parallel or with "$k$ out of $n$" voting? And how do we estimate a mean life from a test that is stopped after a few failures, when most items are still working? The answers use almost every tool of the earlier chapters: the tail formula for expectations (§1.3), the exponential, Weibull and gamma laws (§2.1), the Poisson process (§3.3), order statistics and Rényi's representation (§4.1), sufficiency, the Lehmann–Scheffé theorem and the Cramér–Rao bound (§4.2), and pivotal confidence intervals (§4.3).

The central object is the hazard rate $h(t)$, the instantaneous failure rate of a unit that has survived to age $t$. It determines the lifetime law completely, it adds up when components are put in series, and its shape — increasing, decreasing or constant — is the mathematical meaning of "ageing". CSIR NET asks for hazard rates and conditional survival probabilities, for the reliability and mean life of small systems of exponential or Weibull components, for the IFR/DFR status of standard laws and of mixtures, and for the maximum likelihood estimate of a mean life from a censored exponential test. Each of these has a standard trap, and the ⚠️ box collects them.

Lifetimes, Reliability and the Hazard Rate

A lifetime is a random variable $T$ with $P(T\ge0)=1$: the age at which a unit fails. Throughout this section $T$ has a density $f$ unless we say otherwise, so $P(T=t)=0$ for every $t$. We describe its law not by the cdf $F$ but by the probability of survival, because that is what an engineer, an actuary or a doctor actually reports.

📘 Definition 9.1 — Reliability function, hazard rate and cumulative hazard

Let $T$ be a lifetime with cdf $F$. Its reliability function (survival function) is $$R(t)=P(T\gt t)=1-F(t),\qquad t\ge0.$$ Let $\tau=\sup\{t:R(t)\gt0\}\le\infty$ be the right end of the support. If $T$ has a density $f$, its hazard rate (failure rate, force of mortality) and cumulative hazard are, for $0\le t\lt\tau$, $$h(t)=\frac{f(t)}{R(t)},\qquad H(t)=\int_0^t h(u)\,du.$$

A hazard rate is a rate, not a probability: it is measured in failures per unit time and may exceed $1$.

📐 Theorem 9.1 — The hazard rate determines the lifetime law

Let $T$ be a lifetime whose density $f$ is continuous on $[0,\tau)$. Then for $0\le t\lt\tau$:

  1. $h(t)=-\dfrac{d}{dt}\log R(t)$, and $$R(t)=e^{-H(t)},\qquad f(t)=h(t)\,e^{-H(t)}.$$
  2. $P(t\lt T\le t+\Delta\mid T\gt t)=h(t)\Delta+o(\Delta)$ as $\Delta\downarrow0$.
  3. $H(t)\to\infty$ as $t\uparrow\tau$; in particular $\int_0^\infty h(u)\,du=\infty$ when $\tau=\infty$.
  4. $H(T)\sim\operatorname{Exp}(1)$.

Conversely, if $h\ge0$ is continuous on $[0,\infty)$ and $\int_0^\infty h=\infty$, then $R(t)=\exp\left(-\int_0^th\right)$ is the reliability function of a lifetime whose hazard rate is $h$, and it is the only such law.

Proof. (1) $F(t)=\int_0^tf$ with $f$ continuous, so $R$ is differentiable with $R'=-f$ (fundamental theorem of calculus). Where $R\gt0$, $\frac{d}{dt}\log R(t)=-f(t)/R(t)=-h(t)$. Integrating from $0$ to $t$ and using $R(0)=1-P(T=0)=1$ gives $\log R(t)=-H(t)$; then $f=hR=he^{-H}$.

(2) By the definition of conditional probability, $$P(t\lt T\le t+\Delta\mid T\gt t)=\frac{R(t)-R(t+\Delta)}{R(t)},$$ and the numerator is $\int_t^{t+\Delta}f(u)\,du=f(t)\Delta+o(\Delta)$ by continuity of $f$.

(3) $R$ is continuous and nonincreasing. If $\tau\lt\infty$ then $R(\tau)=0$ and $R(t)\to0$ as $t\uparrow\tau$; if $\tau=\infty$ then $R(t)\to P(T=\infty)=0$. In both cases $H(t)=-\log R(t)\to\infty$.

(4) $H$ is continuous and nondecreasing on $[0,\tau)$, with $H(0)=0$ and $H(t)\to\infty$. Fix $y\ge0$. The set $\{t:H(t)=y\}$ is nonempty (intermediate value theorem), closed in $[0,\tau)$ and bounded away from $\tau$, so it has a largest element $t_y$. Then $H(t)\le y$ for $t\le t_y$ and $H(t)\gt y$ for $t\gt t_y$, so $\{H(T)\gt y\}=\{T\gt t_y\}$ and $P(H(T)\gt y)=R(t_y)=e^{-H(t_y)}=e^{-y}$.

For the converse, $R(t)=e^{-\int_0^th}$ is continuous, nonincreasing, equal to $1$ at $0$ and tends to $0$; so $F=1-R$ is the cdf of a lifetime, with continuous density $-R'=hR$ and hazard rate $hR/R=h$. Any lifetime with hazard rate $h$ has reliability $e^{-H}$ by (1), so the law is unique. ∎

Part (2) is the working meaning of the hazard: among units that have survived to age $t$, a fraction of about $h(t)\Delta$ fails in the next short interval of length $\Delta$. Part (4) is the reliability version of the probability integral transform, and it gives a simulation recipe: if $E\sim\operatorname{Exp}(1)$ and $H$ is strictly increasing, then $T=H^{-1}(E)$ has cumulative hazard $H$. Part (3) rules out candidates such as $h(t)=e^{-t}$: its integral is $1$, and $e^{-\int_0^t h}$ decreases only to $e^{-1}$, which would leave a unit that never fails with probability $e^{-1}$ (Exercise I.3).

📘 Definition 9.2 — Mean time to failure and mean residual life

The mean time to failure (MTTF) is $\mu=E[T]$. For $t$ with $R(t)\gt0$, the residual life at age $t$ is $T-t$ given $T\gt t$; its reliability function is $x\mapsto R(t+x)/R(t)$, and its mean $$m(t)=E[T-t\mid T\gt t]$$ is the mean residual life (in demography, the life expectancy at age $t$). Note $m(0)=\mu$.

📐 Theorem 9.2 — MTTF and mean residual life from the reliability function

Let $T$ be any lifetime (no density is needed for (1)–(3)).

  1. $E[T]=\int_0^\infty R(t)\,dt$, both sides possibly $+\infty$; more generally $E[T^r]=r\int_0^\infty t^{r-1}R(t)\,dt$ for $r\gt0$.
  2. If $\mu\lt\infty$ and $R(t)\gt0$, then $m(t)=\dfrac{1}{R(t)}\displaystyle\int_t^\infty R(u)\,du$.
  3. If $\mu\lt\infty$, $m$ is finite on $\{R\gt0\}$; if moreover $T$ has a continuous density and $m$ is differentiable, then $$h(t)=\frac{1+m'(t)}{m(t)}.$$

Proof. (1) For $T\ge0$, $T^r=\int_0^T rt^{r-1}dt=\int_0^\infty rt^{r-1}\mathbf 1\{T\gt t\}\,dt$. The integrand is nonnegative, so Tonelli's theorem lets us take expectations inside: $E[T^r]=\int_0^\infty rt^{r-1}P(T\gt t)\,dt$. This is the tail formula of §1.3. (2) Apply (1) with $r=1$ to the residual life, whose reliability is $R(t+x)/R(t)$: $m(t)=\int_0^\infty\frac{R(t+x)}{R(t)}dx=\frac{1}{R(t)}\int_t^\infty R(u)\,du$. (3) Finiteness follows from $\int_t^\infty R\le\mu$. Differentiate $m(t)R(t)=\int_t^\infty R(u)\,du$: $m'R+mR'=-R$. Divide by $R$ and use $R'/R=-h$: $m'-mh=-1$, which rearranges to the formula. ∎

Part (3) shows that the mean residual life also determines the law: solving $h=(1+m')/m$ gives the inversion formula $R(t)=\dfrac{m(0)}{m(t)}\exp\left(-\displaystyle\int_0^t\frac{du}{m(u)}\right)$ (Exercise I.5).

✏️ Example 1 — A linearly increasing hazard
The lifetime $T$ (in years) of a pump has hazard rate $h(t)=0.1+0.02t$. Find $R$, $f$ and the median life; find the MTTF in terms of $\Phi$; find $P(T\gt15\mid T\gt10)$ and the mean residual life at age $10$.
Law $H(t)=0.1t+0.01t^2$, which tends to $\infty$, so by Theorem 9.1 $$R(t)=e^{-0.1t-0.01t^2},$$ $$f(t)=(0.1+0.02t)\,e^{-0.1t-0.01t^2},\qquad t\ge0.$$
Median $R(t)=\tfrac12\iff0.01t^2+0.1t-\log2=0$, whose positive root is $$t=\frac{-0.1+\sqrt{0.01+0.04\log2}}{0.02}=4.712\ \text{years}.$$
MTTF Complete the square: $0.01t^2+0.1t=0.01(t+5)^2-0.25$. By Theorem 9.2, $$\mu=e^{0.25}\int_5^\infty e^{-u^2/100}\,du.$$ The integrand is $\sqrt{2\pi\cdot50}$ times the $N(0,50)$ density, so $$\mu=10\sqrt\pi\,e^{0.25}\left(1-\Phi\left(\tfrac{5}{\sqrt{50}}\right)\right).$$ With $1-\Phi(0.7071)=0.23975$ this is $\mu=5.456$ years.
Conditional $H(15)=1.5+2.25=3.75$ and $H(10)=2$, so $$P(T\gt15\mid T\gt10)=e^{-(H(15)-H(10))},$$ which is $e^{-1.75}=0.1738$. A new pump survives $5$ years with probability $R(5)=e^{-0.75}=0.4724$: the hazard increases, so a used pump is worse than a new one.
Residual life By Theorem 9.2(2) and the same completion of the square, $$m(10)=\frac{10\sqrt\pi\,e^{0.25}\left(1-\Phi(15/\sqrt{50})\right)}{e^{-2}}.$$ With $1-\Phi(2.1213)=0.016947$ this gives $m(10)=2.850$ years, against $m(0)=\mu=5.456$ years.
Check Numerical integration of $R$ gives the same MTTF, $5.4564$. A constant hazard $0.1$ alone would give a mean of $10$ years; the wear-out term $0.02t$ roughly halves it.

Ageing: IFR and DFR Laws; Exponential, Weibull and Gamma Lifetimes

"The unit ages" can mean several things: its hazard rises, its chance of surviving a further $x$ years falls with age, its expected remaining life shrinks, or a used unit is worse than a new one. These notions are not equivalent, and the distinctions are exactly what Part C questions test.

📘 Definition 9.3 — Ageing classes

Let $T$ be a lifetime with reliability $R$, $R(0)=1$.

  • $T$ is IFR (increasing failure rate) if for each $x\gt0$ the conditional survival $R(t+x)/R(t)$ is nonincreasing in $t$ on $\{R(t)\gt0\}$; DFR (decreasing failure rate) if it is nondecreasing. For laws with a continuous density these mean that $h$ is nondecreasing, respectively nonincreasing (Theorem 9.3).
  • $T$ is NBU (new better than used) if $R(s+t)\le R(s)R(t)$ for all $s,t\ge0$, and NWU if $R(s+t)\ge R(s)R(t)$.
  • $T$ is DMRL (IMRL) if its mean residual life $m(t)$ is nonincreasing (nondecreasing).
  • $T$ is IFRA (increasing failure rate on average) if $H(t)/t$ is nondecreasing on $t\gt0$.

"Increasing" is meant in the weak sense, so the exponential law, with constant hazard, is both IFR and DFR.

📐 Theorem 9.3 — Properties of IFR and DFR laws

Let $T$ have a density, continuous on $[0,\tau)$, with hazard rate $h$ and cumulative hazard $H$.

  1. The following are equivalent: (a) $h$ is nondecreasing on $[0,\tau)$; (b) $H$ is convex, i.e. $\log R$ is concave, on $[0,\tau)$; (c) $T$ is IFR. The same holds with "nonincreasing", "concave" ("$\log R$ convex") and "DFR".
  2. IFR implies NBU and, if $\mu\lt\infty$, DMRL. DFR implies NWU and, if $\mu\lt\infty$, IMRL.
  3. $T$ is both IFR and DFR if and only if $T\sim\operatorname{Exp}(\lambda)$ for some $\lambda\gt0$.
  4. If $T_1,\dots,T_n$ are independent with hazard rates $h_1,\dots,h_n$, then $\min_iT_i$ has hazard rate $h_1+\cdots+h_n$; hence a minimum of independent IFR (DFR) lifetimes is IFR (DFR).
  5. For $c\gt0$, $cT$ has hazard rate $\frac1c h\left(\frac tc\right)$, so $cT$ is IFR (DFR) iff $T$ is.

Proof. (1) $H'=h$ on $[0,\tau)$, and a differentiable function is convex exactly when its derivative is nondecreasing; this is (a)$\iff$(b), and $\log R=-H$. For (a)$\Rightarrow$(c) write $$\frac{R(t+x)}{R(t)}=\exp\left(-\int_0^xh(t+v)\,dv\right),$$ which is nonincreasing in $t$ when $h$ is nondecreasing. For (c)$\Rightarrow$(a): if $t\lt s$ then $H(t+x)-H(t)\le H(s+x)-H(s)$ for every $x\gt0$; divide by $x$ and let $x\downarrow0$ to get $h(t)\le h(s)$. The DFR case reverses every inequality.

(2) If $T$ is IFR, the conditional survival $R(s+t)/R(s)$ is nonincreasing in $s$, so it is at most its value at $s=0$, which is $R(t)$: that is NBU. For DMRL, by Theorem 9.2(2), $m(t)=\int_0^\infty\frac{R(t+x)}{R(t)}dx$, and the integrand is nonincreasing in $t$ for every $x$. The DFR statements are proved in the same way, the integrals being finite because $\mu\lt\infty$.

(3) If $h$ is both nondecreasing and nonincreasing it is a constant $\lambda$, and $\lambda\gt0$ because $\int_0^\infty h=\infty$ (Theorem 9.1(3)); so $R(t)=e^{-\lambda t}$. Conversely $\operatorname{Exp}(\lambda)$ has $h\equiv\lambda$.

(4) By independence $P(\min T_i\gt t)=\prod R_i(t)=\exp\left(-\sum H_i(t)\right)$, so the cumulative hazard of the minimum is $\sum H_i$ and its hazard rate is $\sum h_i$. A sum of nondecreasing (nonincreasing) functions is nondecreasing (nonincreasing).

(5) $P(cT\gt t)=R(t/c)$; differentiate $-\log R(t/c)$. ∎

📐 Theorem 9.4 — Mixtures of exponential laws are DFR
Let $\Lambda$ be a random rate with any distribution $G$ on $(0,\infty)$, and given $\Lambda=\lambda$ let $T\sim\operatorname{Exp}(\lambda)$. Then $R(t)=\int e^{-\lambda t}dG(\lambda)$, and for $t\gt0$ $$h(t)=E[\Lambda\mid T\gt t],$$ $$h'(t)=-\operatorname{Var}(\Lambda\mid T\gt t)\le0.$$ So $T$ is DFR, strictly unless $\Lambda$ is constant, although every individual unit has a constant hazard.

Proof. Conditioning on $\Lambda$, $R(t)=\int e^{-\lambda t}dG(\lambda)$ and $f(t)=\int\lambda e^{-\lambda t}dG(\lambda)$. By Bayes' theorem the conditional law of $\Lambda$ given $T\gt t$ is $dG_t(\lambda)=e^{-\lambda t}dG(\lambda)/R(t)$, so $h(t)=f(t)/R(t)=\int\lambda\,dG_t(\lambda)=E[\Lambda\mid T\gt t]$. For $t\ge t_0\gt0$ the functions $\lambda^je^{-\lambda t}$ ($j=0,1,2$) are bounded by constants depending only on $t_0$, so we may differentiate under the integral sign (dominated convergence): $R'=-\int\lambda e^{-\lambda t}dG$, $f'=-\int\lambda^2e^{-\lambda t}dG$. Then $$h'=\frac{f'R-fR'}{R^2}=-\int\lambda^2dG_t+\left(\int\lambda\,dG_t\right)^2,$$ which is $-\operatorname{Var}(\Lambda\mid T\gt t)$. ∎

The interpretation is selection: the units with large rates tend to fail first, so the survivors at age $t$ are a progressively more reliable sub-population. This is why pooling data from several production batches can make a population of non-ageing units look DFR. More generally, any mixture of DFR laws is DFR (we only state this); mixtures of IFR laws need not be IFR — the exponentials themselves are IFR.

📐 Theorem 9.5 — A lower bound for IFR reliability
If $T$ is IFR (with a continuous density) and has mean $\mu$, then $$R(t)\ge e^{-t/\mu}\qquad\text{for }0\le t\lt\mu.$$ In particular an IFR unit survives to (just below) its mean life with probability at least $e^{-1}\approx0.368$.

Proof. Since $T\lt\tau$ with probability one and $T$ is not constant, $\mu\lt\tau$, so $H(\mu)$ is defined. By Theorem 9.3(1), $H$ is convex, and $H(0)=0$. By Theorem 9.1(4), $H(T)\sim\operatorname{Exp}(1)$, so $E[H(T)]=1$, and Jensen's inequality (§2.2) gives $H(\mu)=H(E[T])\le E[H(T)]=1$. For $0\le t\lt\mu$, convexity on $[0,\mu]$ gives $H(t)\le\left(1-\frac t\mu\right)H(0)+\frac t\mu H(\mu)\le\frac t\mu$. Hence $R(t)=e^{-H(t)}\ge e^{-t/\mu}$. ∎

So among IFR laws with a given mean, the exponential law is the least reliable before the mean. The DFR analogue reverses the inequality (Exercise II.5).

📐 Theorem 9.6 — Exponential, Weibull and gamma lifetimes
  1. (Exponential) $T\sim\operatorname{Exp}(\lambda)$ has $h\equiv\lambda$, $R(t)=e^{-\lambda t}$, $\mu=1/\lambda$ and $m(t)\equiv1/\lambda$: a used unit is as good as new.
  2. (Weibull) $T\sim\operatorname{Weib}(k,\lambda)$ (shape $k$, scale $\lambda$) has $H(t)=(t/\lambda)^k$ and $h(t)=\frac k\lambda\left(\frac t\lambda\right)^{k-1}$. It is IFR iff $k\ge1$ and DFR iff $k\le1$. Its moments are $E[T^r]=\lambda^r\Gamma\left(1+\frac rk\right)$, so $\mu=\lambda\Gamma\left(1+\frac1k\right)$. Moreover $$\log\left(-\log R(t)\right)=k\log t-k\log\lambda,$$ and the minimum of $n$ iid $\operatorname{Weib}(k,\lambda)$ lifetimes is $\operatorname{Weib}(k,\lambda n^{-1/k})$.
  3. (Gamma) $T\sim\operatorname{Gamma}(\alpha,\lambda)$ (shape $\alpha$, rate $\lambda$) has an increasing hazard rate if $\alpha\gt1$ and a decreasing one if $\alpha\lt1$, and $h(t)\to\lambda$ as $t\to\infty$. For $\alpha=2$, $h(t)=\dfrac{\lambda^2t}{1+\lambda t}$.

Proof. (1) Direct, and the lack of memory is the characterisation of the exponential law in §2.1. (2) $R(t)=e^{-(t/\lambda)^k}$ (§2.1), so $H=(t/\lambda)^k$ and $h=H'$; $t^{k-1}$ is nondecreasing iff $k\ge1$ and nonincreasing iff $k\le1$. For moments substitute $u=(t/\lambda)^k$ in Theorem 9.2(1): $$E[T^r]=\int_0^\infty rt^{r-1}e^{-(t/\lambda)^k}dt,$$ $$E[T^r]=\lambda^r\int_0^\infty\tfrac rk u^{r/k-1}e^{-u}du,$$ which is $\lambda^r\frac rk\Gamma\left(\frac rk\right)=\lambda^r\Gamma\left(1+\frac rk\right)$. Taking logarithms twice in $-\log R(t)=(t/\lambda)^k$ gives the straight line. By Theorem 9.3(4) the minimum has cumulative hazard $n(t/\lambda)^k=\left(t/(\lambda n^{-1/k})\right)^k$.

(3) With $f(s)=\frac{\lambda^\alpha}{\Gamma(\alpha)}s^{\alpha-1}e^{-\lambda s}$ and the substitution $s=t+u$, $$\frac1{h(t)}=\frac{R(t)}{f(t)}=\int_t^\infty\frac{f(s)}{f(t)}ds,$$ $$\frac1{h(t)}=\int_0^\infty\left(1+\frac ut\right)^{\alpha-1}e^{-\lambda u}du.$$ For fixed $u\gt0$, $(1+u/t)^{\alpha-1}$ is strictly decreasing in $t$ if $\alpha\gt1$ and strictly increasing if $\alpha\lt1$; so $1/h$ is decreasing (increasing) and $h$ increasing (decreasing). As $t\to\infty$ the integrand tends to $e^{-\lambda u}$, dominated by $(1+u)^{\alpha-1}e^{-\lambda u}$ for $t\ge1$ when $\alpha\gt1$ and by $e^{-\lambda u}$ when $\alpha\lt1$; so $1/h(t)\to1/\lambda$. For $\alpha=2$: $R(t)=(1+\lambda t)e^{-\lambda t}$ and $f(t)=\lambda^2te^{-\lambda t}$. ∎

The straight line in (2) is the basis of the Weibull probability plot: plot $\log(-\log\hat R(t))$ against $\log t$ for the observed failure times; a roughly straight plot supports a Weibull model, and its slope estimates the shape $k$ (Exercise II.6 and the Research–Teaching Nexus). Real populations often show all three regimes: a decreasing hazard while manufacturing defects show up (burn-in), a roughly constant hazard during the useful life, and an increasing hazard during wear-out. The resulting bathtub curve is not IFR, DFR or exponential; one simple model is a series system of a Weibull component with $k\lt1$, an exponential one and a Weibull one with $k\gt1$, whose hazards add (Theorem 9.3(4), Exercise II.4). The lognormal hazard of §2.1, which rises and then falls, is another non-monotone shape.

✏️ Example 2 — Weibull bearings
Bearings have $\operatorname{Weib}(2.5,8)$ lifetimes, in years. (a) Find the MTTF and the standard deviation of the life. (b) Find $R(5)$ and $h(5)$. (c) Find the "$B_{10}$ life", the age by which $10\%$ of bearings have failed. (d) Find the probability that a bearing which is working at age $5$ is still working at age $8$. (e) Compare $R(5)$ with the reliability of an exponential unit with the same MTTF.
(a) By Theorem 9.6(2), $\mu=8\,\Gamma(1.4)=8(0.88726)=7.098$ years, and $$\operatorname{SD}=8\sqrt{\Gamma(1.8)-\Gamma(1.4)^2},$$ which is $8\sqrt{0.93138-0.78723}=3.037$ years.
(b) $(5/8)^{2.5}=0.30881$, so $R(5)=e^{-0.30881}=0.7343$, and $$h(5)=\frac{2.5}{8}\left(\frac58\right)^{1.5}=0.3125\times0.49411,$$ that is, $0.1544$ failures per year.
(c) $R(t)=0.9\iff(t/8)^{2.5}=-\log0.9=0.10536$, so $t=8(0.10536)^{0.4}=3.252$ years.
(d) $\dfrac{R(8)}{R(5)}=\exp\left(-\left(1-0.30881\right)\right)$, which is $e^{-0.69119}=0.5010$. A new bearing survives its first $3$ years with probability $e^{-(3/8)^{2.5}}=0.917$: wear-out (IFR, $k=2.5\gt1$).
(e) An exponential unit with mean $7.098$ has $R(5)=e^{-5/7.098}=0.4944$, far below $0.7343$. This is Theorem 9.5 in action: $5\lt\mu$, so the IFR bearing satisfies $R(5)\ge e^{-5/\mu}$. Two lifetimes with the same mean can have very different reliabilities — the MTTF alone does not describe a lifetime.
✏️ Example 3 — Ageing: prove or disprove
Prove or disprove: (a) the minimum of two independent IFR lifetimes is IFR; (b) the lifetime with $R(t)=\frac12e^{-t}+\frac12e^{-3t}$ is DFR; (c) if $T_1\sim\operatorname{Exp}(1)$ and $T_2\sim\operatorname{Exp}(3)$ are independent, then $\max(T_1,T_2)$ is IFR; (d) if $T$ is IFR with mean $\mu$, then $P(T\gt\mu/2)\ge e^{-1/2}$.
(a) True By Theorem 9.3(4) the hazard of the minimum is $h_1+h_2$, a sum of nondecreasing functions.
(b) True This is a mixture of $\operatorname{Exp}(1)$ and $\operatorname{Exp}(3)$, so Theorem 9.4 applies. Explicitly, $$h(t)=\frac{e^{-t}+3e^{-3t}}{e^{-t}+e^{-3t}}=1+\frac{2}{e^{2t}+1},$$ which decreases from $2$ to $1$: the survivors are more and more likely to be the "rate $1$" units.
(c) False $R(t)=1-(1-e^{-t})(1-e^{-3t})$, that is, $R(t)=e^{-t}+e^{-3t}-e^{-4t}$, and $f(t)=e^{-t}+3e^{-3t}-4e^{-4t}$. So $h(0)=0$, and $h(t)\to1$ as $t\to\infty$ (the $e^{-t}$ terms dominate). But $f(t)-R(t)=2e^{-3t}-3e^{-4t}$, so $$f(t)-R(t)=e^{-4t}\left(2e^{t}-3\right)\gt0$$ for $t\gt\log1.5$, so $h(t)\gt1$ there. A nondecreasing function with limit $1$ never exceeds $1$, so $h$ is not monotone: it rises to about $1.119$ (near $t=0.83$) and falls back to $1$. Once the fast unit has probably failed, the system behaves like the slow unit alone.
(d) True Theorem 9.5 with $t=\mu/2$ gives $R(\mu/2)\ge e^{-1/2}=0.607$. For the pump of Example 1, $\mu=5.456$ and $R(2.728)=0.707$, consistent with the bound.

Series, Parallel and $k$-out-of-$n$ Systems

A system is built from $n$ components, each either working or failed. Its reliability depends on two things: how reliable the components are, and how they are connected. A chain fails when its first link breaks (series); a set of redundant pumps fails only when the last one stops (parallel); an aircraft with three hydraulic circuits may need any two of them (2-out-of-3). The structure function separates the wiring from the probabilities.

📘 Definition 9.4 — Structure functions and coherent systems

Let $x=(x_1,\dots,x_n)\in\{0,1\}^n$ record the states of $n$ components ($x_i=1$: component $i$ works). A structure function is a map $\varphi:\{0,1\}^n\to\{0,1\}$, with $\varphi(x)=1$ when the system works. Write $(1_i,x)$ and $(0_i,x)$ for $x$ with its $i$-th coordinate set to $1$ or $0$, and $x\amalg y$ for the coordinatewise maximum. The system is coherent if $\varphi$ is nondecreasing in each coordinate and every component is relevant: for each $i$ there is an $x$ with $\varphi(1_i,x)\ne\varphi(0_i,x)$. The standard structures are

Series$\varphi(x)=\min_ix_i=\prod_{i=1}^nx_i$: all components must work.
Parallel$\varphi(x)=\max_ix_i=1-\prod_{i=1}^n(1-x_i)$: one working component suffices.
$k$-out-of-$n$$\varphi(x)=\mathbf 1\{x_1+\cdots+x_n\ge k\}$: at least $k$ must work. $k=n$ is series, $k=1$ is parallel.

If the component states $X_i$ are independent with $p_i=P(X_i=1)$, the reliability of the system is $r(p)=P(\varphi(X)=1)=E[\varphi(X)]$. With component lifetimes $T_i$ and $X_i(t)=\mathbf 1\{T_i\gt t\}$, the system lifetime is $T_S=\sup\{t:\varphi(X(t))=1\}$ and $R_S(t)=r\big(R_1(t),\dots,R_n(t)\big)$.

📐 Theorem 9.7 — Reliability of series, parallel and $k$-out-of-$n$ systems

Let the component lifetimes $T_1,\dots,T_n$ be independent with reliability functions $R_i$ and hazard rates $h_i$.

  1. (Series) $T_S=\min_iT_i$, $R_S(t)=\prod_iR_i(t)$ and $h_S(t)=\sum_ih_i(t)$.
  2. (Parallel) $T_S=\max_iT_i$ and $R_S(t)=1-\prod_i\big(1-R_i(t)\big)$.
  3. ($k$-out-of-$n$, iid components with reliability $R$) $T_S=T_{(n-k+1)}$, the $(n-k+1)$-th order statistic, and $$R_S(t)=\sum_{j=k}^n\binom nj R(t)^j\big(1-R(t)\big)^{n-j}.$$

Proof. (1) A series system works at time $t$ iff every component does, i.e. iff $\min T_i\gt t$; independence gives the product, and Theorem 9.3(4) gives the hazard. (2) A parallel system has failed by time $t$ iff every component has: $P(T_S\le t)=\prod_iP(T_i\le t)=\prod_i(1-R_i(t))$. (3) The number of components working at time $t$ is $N(t)=\sum_i\mathbf 1\{T_i\gt t\}\sim\operatorname{Bin}(n,R(t))$, and the system works iff $N(t)\ge k$, which gives the sum. The system stops when the number of failed components first reaches $n-k+1$, i.e. at the $(n-k+1)$-th smallest lifetime. ∎

📐 Theorem 9.8 — Systems of exponential components
  1. A series system of independent $\operatorname{Exp}(\lambda_i)$ components has lifetime $\operatorname{Exp}(\lambda_1+\cdots+\lambda_n)$ and MTTF $1/\sum_i\lambda_i$.
  2. A parallel system of independent $\operatorname{Exp}(\lambda_1)$ and $\operatorname{Exp}(\lambda_2)$ components has MTTF $\dfrac1{\lambda_1}+\dfrac1{\lambda_2}-\dfrac1{\lambda_1+\lambda_2}$.
  3. A $k$-out-of-$n$ system of iid $\operatorname{Exp}(\lambda)$ components has MTTF $$\mu_S=\frac1\lambda\sum_{j=k}^n\frac1j.$$ In particular a parallel system has MTTF $\frac1\lambda\left(1+\frac12+\cdots+\frac1n\right)$ and a series system $\frac{1}{n\lambda}$.
  4. (Cold standby) If one unit works and $n-1$ identical spares, which do not fail while idle, are switched in perfectly one after another, the system lifetime is $\operatorname{Gamma}(n,\lambda)$: $$R_S(t)=e^{-\lambda t}\sum_{j=0}^{n-1}\frac{(\lambda t)^j}{j!},\qquad\mu_S=\frac n\lambda.$$

Proof. (1) Theorem 9.7(1) with $R_i(t)=e^{-\lambda_it}$ gives $R_S(t)=e^{-(\sum\lambda_i)t}$. (2) $R_S(t)=e^{-\lambda_1t}+e^{-\lambda_2t}-e^{-(\lambda_1+\lambda_2)t}$; integrate term by term (Theorem 9.2(1)), using $\int_0^\infty e^{-ct}dt=1/c$.

(3) By Theorem 9.7(3), $T_S=T_{(m)}$ with $m=n-k+1$. Rényi's representation of exponential order statistics (§4.1) writes $T_{(m)}=\sum_{i=1}^m\frac{D_i}{n-i+1}$ with $D_1,\dots,D_n$ iid $\operatorname{Exp}(\lambda)$. Hence $$E[T_{(m)}]=\frac1\lambda\sum_{i=1}^{n-k+1}\frac{1}{n-i+1}=\frac1\lambda\sum_{j=k}^n\frac1j.$$ The intuition is the memoryless property: while $j$ components are working, the time to the next failure is the minimum of $j$ fresh exponential lifetimes, an $\operatorname{Exp}(j\lambda)$ variable with mean $1/(j\lambda)$, and the system passes through $j=n,n-1,\dots,k$ before it fails.

(4) By the memoryless property each unit, once switched in, lives an $\operatorname{Exp}(\lambda)$ time independently of the past, so $T_S$ is a sum of $n$ iid $\operatorname{Exp}(\lambda)$ variables, which is $\operatorname{Gamma}(n,\lambda)$ (§2.1). Its reliability $P(T_S\gt t)$ is the probability of fewer than $n$ events by time $t$ in a Poisson process of rate $\lambda$ (§3.3). ∎

Two consequences are worth remembering. Adding parallel units helps less and less: the MTTF grows like $\frac1\lambda\log n$, without bound but very slowly. And cold standby beats hot (parallel) redundancy, $\frac2\lambda$ against $\frac{3}{2\lambda}$ for two units, because an idle spare does not use up its life.

📐 Theorem 9.9 — Pivotal decomposition and the redundancy principle

Let $\varphi$ be a coherent structure whose components are independent, with reliabilities $p=(p_1,\dots,p_n)$.

  1. (Pivotal decomposition) For each $i$, $\varphi(x)=x_i\varphi(1_i,x)+(1-x_i)\varphi(0_i,x)$, so $$r(p)=p_i\,r(1_i,p)+(1-p_i)\,r(0_i,p).$$ Hence $r$ is a polynomial of degree at most one in each $p_i$, and it is strictly increasing in each $p_i$ on $(0,1)^n$.
  2. (Redundancy principle) $\varphi(x\amalg y)\ge\varphi(x)\amalg\varphi(y)$ for all $x,y$. Consequently, duplicating every component (a spare in parallel with each one, the spares independent with reliabilities $p'$) is at least as good as duplicating the whole system: writing $a\amalg b=1-(1-a)(1-b)$ for numbers, and coordinatewise for vectors, $$r(p\amalg p')\ge r(p)\amalg r(p').$$

Proof. (1) If $x_i=1$ the right side is $\varphi(1_i,x)=\varphi(x)$; if $x_i=0$ it is $\varphi(0_i,x)=\varphi(x)$. Take expectations with $X$ random: $\varphi(1_i,X)$ does not depend on $X_i$, which is independent of the other coordinates, so $E[X_i\varphi(1_i,X)]=p_i\,r(1_i,p)$, and similarly for the second term. Applying the identity to each coordinate in turn shows that $r$ is multilinear. Finally $\frac{\partial r}{\partial p_i}=r(1_i,p)-r(0_i,p)=E[\varphi(1_i,X)-\varphi(0_i,X)]$; the difference is $\ge0$ by monotonicity, and it equals $1$ at some state $x$ by relevance — a state that has positive probability when every $p_j\in(0,1)$.

(2) $x\amalg y\ge x$ and $x\amalg y\ge y$ coordinatewise, and $\varphi$ is nondecreasing, so $\varphi(x\amalg y)\ge\max\left(\varphi(x),\varphi(y)\right)$. Now let $X$ (the original components) and $Y$ (the spares) be independent. The coordinates of $X\amalg Y$ are independent with $P\big((X\amalg Y)_i=1\big)=p_i\amalg p'_i$, so the left side of the inequality is $E[\varphi(X\amalg Y)]$. The right side is $E[\max(\varphi(X),\varphi(Y))]$, because $\varphi(X)$ and $\varphi(Y)$ are independent with means $r(p)$ and $r(p')$. ∎

The pivotal decomposition computes the reliability of structures that are neither series nor parallel. The classical example is the bridge: two paths $1\to2$ and $3\to4$ joined by a cross-link $5$. With $x_5=1$ the bridge is $(1\parallel3)$ in series with $(2\parallel4)$; with $x_5=0$ it is $(1\text{–}2)\parallel(3\text{–}4)$. For iid components with reliability $p$ this gives $$r(p)=p(2p-p^2)^2+(1-p)\big(1-(1-p^2)^2\big),$$ $$r(p)=2p^2+2p^3-5p^4+2p^5$$ (Exercise III.2).

🧭 Problem-solving strategy — reliability and MTTF of a system
  1. Draw the block diagram and reduce it: blocks in series multiply reliabilities; blocks in parallel multiply unreliabilities $1-R$. For identical components with "at least $k$ working", use the binomial sum of Theorem 9.7(3).
  2. If it does not reduce (a bridge, a shared component), condition on one component: $r=p_i\,r(1_i,p)+(1-p_i)\,r(0_i,p)$.
  3. For the MTTF, write $R_S(t)$ as a combination of terms $e^{-ct}$ and use $\int_0^\infty e^{-ct}dt=\frac1c$; for $k$-out-of-$n$ exponential systems use $\frac1\lambda\sum_{j=k}^n\frac1j$ directly. Never add or average component MTTFs.
  4. Check: $R_S(0)=1$, $R_S(\infty)=0$; series $\le$ every component $\le$ parallel; for identical components, $r(p)=p$ only at $p=0,1$ unless the system is a single component.
✏️ Example 4 — A pump and two valves
A cooling line consists of a pump $A$ in series with two valves $B$, $C$ in parallel. Lifetimes are independent exponential with rates $\lambda_A=0.001$ and $\lambda_B=\lambda_C=0.002$ per hour. (a) Find $R_S(t)$, $R_S(200)$ and the MTTF. (b) The budget allows one extra component: a second pump in parallel with $A$, or a third valve in parallel with $B$ and $C$. Which raises the MTTF more?
Reliability The valve pair has reliability $1-(1-e^{-0.002t})^2=2e^{-0.002t}-e^{-0.004t}$, so $$R_S(t)=e^{-0.001t}\left(2e^{-0.002t}-e^{-0.004t}\right),$$ $$R_S(t)=2e^{-0.003t}-e^{-0.005t}.$$ At $t=200$: $R_S=2e^{-0.6}-e^{-1}=1.0976-0.3679=0.7297$.
MTTF $\mu_S=\dfrac{2}{0.003}-\dfrac{1}{0.005}=666.7-200=466.7$ hours, less than the pump's own MTTF of $1000$ hours: the series pump limits the system.
Option 1 Two pumps in parallel have reliability $2e^{-0.001t}-e^{-0.002t}$, and $R_1(t)$ is this times the valve pair's $2e^{-0.002t}-e^{-0.004t}$. Expanding, $R_1(t)$ equals $$4e^{-0.003t}-2e^{-0.005t}-2e^{-0.004t}+e^{-0.006t},$$ so $$\mu_1=\frac{4}{0.003}-\frac{2}{0.005}-\frac{2}{0.004}+\frac1{0.006},$$ which is $600$ hours.
Option 2 Three valves: $R_2(t)=e^{-0.001t}\left(1-(1-e^{-0.002t})^3\right)$, which expands to $$R_2(t)=3e^{-0.003t}-3e^{-0.005t}+e^{-0.007t},$$ so $\mu_2=1000-600+142.9=542.9$ hours.
Answer Duplicate the pump: $600$ h against $542.9$ h (and $R(200)=0.862$ against $0.789$). Redundancy pays most at the weakest point of the system — here the single component in series.
✏️ Example 5 — Triple modular redundancy: prove or disprove
A 2-out-of-3 ("triple modular redundancy", TMR) system has three iid components. Prove or disprove: (a) if the components are $\operatorname{Exp}(\lambda)$, the TMR system has a larger MTTF than a single component; (b) with $\operatorname{Exp}(\lambda)$ components, $R_S(t)\gt e^{-\lambda t}$ exactly for $0\lt t\lt\frac{\log2}{\lambda}$; (c) with $\operatorname{Exp}(\lambda)$ components the TMR system is IFR; (d) for components of fixed reliability $p\in(0,1)$, the TMR system is more reliable than one component iff $p\gt\frac12$.
Set-up With component reliability $p$, Theorem 9.7(3) gives $r(p)=3p^2(1-p)+p^3=3p^2-2p^3$. For exponential components put $x=e^{-\lambda t}$: $R_S(t)=3x^2-2x^3$.
(a) False By Theorem 9.8(3), $\mu_S=\frac1\lambda\left(\frac12+\frac13\right)=\frac{5}{6\lambda}\lt\frac1\lambda$. (Directly: $\int_0^\infty(3e^{-2\lambda t}-2e^{-3\lambda t})dt=\frac{3}{2\lambda}-\frac{2}{3\lambda}$.) Redundancy can shorten the mean life.
(b) True For $x\gt0$, $3x^2-2x^3\gt x$ is equivalent to $2x^2-3x+1\lt0$, i.e. to $(2x-1)(x-1)\lt0$, i.e. to $\frac12\lt x\lt1$; that is, $0\lt t\lt\frac{\log2}\lambda$. TMR is better for short missions and worse for long ones, which is why it suits spacecraft computers with a fixed mission time.
(c) True $f_S(t)=-R_S'(t)=6\lambda(x^2-x^3)$, so $$h_S(t)=\frac{6\lambda x^2(1-x)}{3x^2-2x^3}=6\lambda\,\frac{1-x}{3-2x}.$$ Now $\frac{d}{dx}\frac{1-x}{3-2x}=\frac{-1}{(3-2x)^2}\lt0$ and $x$ decreases in $t$, so $h_S$ increases, from $0$ to $2\lambda$. A system of non-ageing parts can age: as components fail, the remaining structure becomes more fragile.
(d) True $3p^2-2p^3-p=-p(2p-1)(p-1)$, which for $0\lt p\lt1$ has the sign of $2p-1$. Majority voting amplifies reliability above $\frac12$ and destroys it below.
Interactive 1 — Series, Parallel and $k$-out-of-$n$ Systems

A system of $n$ iid components works while at least $k$ of them work ($k=n$: series; $k=1$: parallel). Every component law has mean life $1$. The top lanes show the component lifetimes of one simulated system: a lane turns red when its component fails. Below, the blue curve is the system reliability $R_S(t)=\sum_{j\ge k}\binom nj R^j(1-R)^{n-j}$, the dashed grey curve is one component's $R(t)$, and the green steps are the fraction of $300$ simulated systems still working. Press Play to run the clock, or drag the time slider.

32
structure—
time t—
R_S(t), theory—
simulated fraction working—
system MTTF—
system hazard h_S(t)—

Censoring and Life Testing with Exponential Lifetimes

To estimate the mean life of a component, a manufacturer puts $n$ units on test. Waiting until all of them fail is rarely possible: good units last for years. So the test is stopped early, and the data consist of some failure times together with the information that the remaining units were still working when the test ended. Such observations are censored. Throwing the survivors away, or treating their running times as failure times, gives badly biased answers; the likelihood tells us how to use both kinds of information correctly.

📘 Definition 9.5 — Life tests and censoring

At time $0$, $n$ items with iid lifetimes $X_1,\dots,X_n$ are put on test; $X_{(1)}\le\cdots\le X_{(n)}$ are the order statistics.

  • Type II censoring (failure-censored test): the test stops at the $r$-th failure, $r\le n$ fixed in advance. We observe $X_{(1)},\dots,X_{(r)}$; the other $n-r$ items are censored at $X_{(r)}$. The duration is random.
  • Type I censoring (time-censored test): the test stops at a fixed time $t_0$. We observe the number $D$ of failures in $[0,t_0]$ and their times; $D$ is random and may be $0$.
  • A test is run with replacement if every failed item is immediately replaced by a new one, so that $n$ items are always on test; otherwise without replacement.
  • The total time on test is the sum, over all items that were ever on test, of the time each spent on test. Without replacement, $$T_r=\sum_{i=1}^rX_{(i)}+(n-r)X_{(r)}\quad\text{(Type II)},$$ $$T=\sum_{i=1}^DX_{(i)}+(n-D)\,t_0\quad\text{(Type I)};$$ with replacement it is $n$ times the duration of the test.
  • Random censoring: item $i$ has a censoring time $C_i$, independent of $X_i$, and we observe $Y_i=\min(X_i,C_i)$ and $\delta_i=\mathbf 1\{X_i\le C_i\}$ (patients leaving a clinical study, for instance).
📐 Theorem 9.10 — Likelihoods under censoring

Let the lifetimes have density $f_\theta$, reliability $R_\theta$, hazard $h_\theta$ and cumulative hazard $H_\theta$.

  1. Type II: $L(\theta)=\dfrac{n!}{(n-r)!}\displaystyle\prod_{i=1}^rf_\theta(x_{(i)})\cdot R_\theta(x_{(r)})^{\,n-r}$.
  2. Type I: given $D=d$ failures at $x_{(1)}\lt\cdots\lt x_{(d)}\le t_0$, $L(\theta)=\dfrac{n!}{(n-d)!}\displaystyle\prod_{i=1}^df_\theta(x_{(i)})\cdot R_\theta(t_0)^{\,n-d}$; for $d=0$, $L(\theta)=R_\theta(t_0)^n$.
  3. Random censoring, when the law of the $C_i$ does not involve $\theta$: $L(\theta)\propto\displaystyle\prod_{i=1}^nf_\theta(y_i)^{\delta_i}R_\theta(y_i)^{1-\delta_i}$.

In every case, up to a factor free of $\theta$, $$L(\theta)\propto\prod_{\text{failures}}h_\theta(y_i)\cdot\exp\left(-\sum_{\text{all items}}H_\theta(y_i)\right),$$ where $y_i$ is the time item $i$ spent on test: failures contribute their density, survivors their reliability.

Proof. (1) Fix $x_1\lt\cdots\lt x_r$. The event "the first $r$ failures occur in $[x_i,x_i+dx_i]$, $i=1,\dots,r$" happens when some ordered choice of $r$ distinct items fails at those times — there are $n(n-1)\cdots(n-r+1)=\frac{n!}{(n-r)!}$ ordered choices — and the other $n-r$ items survive beyond $x_r$. Each such configuration has probability $\prod f(x_i)dx_i\cdot R(x_r)^{n-r}$ by independence, which gives the density. (It is also the joint density of order statistics in §4.1 with the $n-r$ largest integrated out.) (2) is the same argument with the $n-d$ survivors outliving the fixed time $t_0$. (3) If $C$ has density $g$ and reliability $G$, a failure observed at $y$ contributes $f_\theta(y)G(y)$ and a censoring at $y$ contributes $R_\theta(y)g(y)$; the factors $G$, $g$ do not involve $\theta$. The final form follows from $f=hR$ and $R=e^{-H}$ (Theorem 9.1). ∎

From now on the lifetimes are exponential with mean $\theta$, i.e. $\operatorname{Exp}(1/\theta)$, the standard parametrisation in life testing. Then $h_\theta=1/\theta$ and $H_\theta(y)=y/\theta$, so every likelihood above is $$L(\theta)\propto\theta^{-(\text{number of failures})}\,e^{-(\text{total time on test})/\theta}.$$ Only two numbers matter: how many failures, and how much total time on test.

📐 Theorem 9.11 — Exponential life test, Type II censoring

Let $n$ items with iid lifetimes of mean $\theta$ be tested without replacement until the $r$-th failure, and let $T_r$ be the total time on test.

  1. The MLE of the mean life is $\hat\theta=T_r/r$.
  2. $2T_r/\theta\sim\chi^2_{2r}$. $T_r$ is a complete sufficient statistic, $\hat\theta$ is unbiased with $\operatorname{Var}(\hat\theta)=\theta^2/r$, it is the UMVUE of $\theta$, and it attains the Cramér–Rao bound; the Fisher information in the test is $r/\theta^2$.
  3. An exact $100(1-\alpha)\%$ confidence interval for $\theta$ is $$\left[\frac{2T_r}{\chi^2_{2r,\alpha/2}},\ \frac{2T_r}{\chi^2_{2r,1-\alpha/2}}\right],$$ where $\chi^2_{m,\alpha}$ is the upper $\alpha$ point of $\chi^2_m$ (as in §4.3); $2T_r/\chi^2_{2r,\alpha}$ is a $100(1-\alpha)\%$ lower confidence bound.
  4. The expected duration of the test is $E[X_{(r)}]=\theta\displaystyle\sum_{i=1}^r\frac{1}{n-i+1}$.

Proof. (1) With $f_\theta(x)=\theta^{-1}e^{-x/\theta}$ and $R_\theta(x)=e^{-x/\theta}$, Theorem 9.10(1) gives $L(\theta)=\frac{n!}{(n-r)!}\theta^{-r}e^{-T_r/\theta}$. So $\ell(\theta)=c-r\log\theta-T_r/\theta$ and $\ell'(\theta)=\frac{T_r-r\theta}{\theta^2}$, which is positive for $\theta\lt T_r/r$ and negative after: the global maximum is at $\hat\theta=T_r/r$.

(2) By Rényi's representation (§4.1), the normalised spacings $D_i=(n-i+1)\left(X_{(i)}-X_{(i-1)}\right)$, $X_{(0)}=0$, are iid exponential with mean $\theta$. Summing, the coefficient of $X_{(j)}$ in $\sum_{i=1}^r(n-i+1)(X_{(i)}-X_{(i-1)})$ is $(n-j+1)-(n-j)=1$ for $j\lt r$ and $n-r+1$ for $j=r$, so $$T_r=\sum_{i=1}^rD_i\sim\operatorname{Gamma}\left(r,\tfrac1\theta\right),$$ and $2T_r/\theta\sim\operatorname{Gamma}(r,\tfrac12)=\chi^2_{2r}$. Hence $E[\hat\theta]=\theta$ and $\operatorname{Var}(\hat\theta)=r\theta^2/r^2=\theta^2/r$. The likelihood depends on the data only through $T_r$, so $T_r$ is sufficient (factorisation theorem, §4.2); its laws $\operatorname{Gamma}(r,1/\theta)$, $\theta\gt0$, form a one-parameter exponential family whose natural parameter $-1/\theta$ fills an open interval, so $T_r$ is complete (§4.2). An unbiased function of a complete sufficient statistic is the UMVUE (Lehmann–Scheffé, §4.2). Finally $-\ell''(\theta)=-\frac r{\theta^2}+\frac{2T_r}{\theta^3}$ has expectation $\frac r{\theta^2}$, so the Cramér–Rao bound is $\theta^2/r=\operatorname{Var}(\hat\theta)$.

(3) $2T_r/\theta$ is a pivot (§4.3): $P\left(\chi^2_{2r,1-\alpha/2}\le2T_r/\theta\le\chi^2_{2r,\alpha/2}\right)=1-\alpha$ for every $\theta$, and solving the inequalities for $\theta$ reverses them. The one-sided bound comes from $P(2T_r/\theta\le\chi^2_{2r,\alpha})=1-\alpha$. (4) $X_{(r)}=\sum_{i=1}^rD_i/(n-i+1)$ and $E[D_i]=\theta$. ∎

Part (2) contains a planning lesson: the information $r/\theta^2$ depends on the number of failures, not on the number of items. A Type II test with $r$ failures among $n$ items is exactly as informative as a complete test of $r$ items, but by (4) it ends much sooner when $n$ is large (Exercise IV.2). Putting more items on test buys time, not precision.

📐 Theorem 9.12 — Exponential life test, Type I censoring

Let $n$ items with iid lifetimes of mean $\theta$ be tested until a fixed time $t_0$, with $D$ failures and total time on test $T$.

  1. (Without replacement) If $D\ge1$, the MLE is $\hat\theta=T/D$. If $D=0$, which has probability $e^{-nt_0/\theta}$, the likelihood $e^{-nt_0/\theta}$ increases in $\theta$ and no MLE exists. Moreover $D\sim\operatorname{Bin}\left(n,1-e^{-t_0/\theta}\right)$, so the counts alone give the estimator $\tilde\theta=-t_0/\log(1-D/n)$ ($1\le D\lt n$).
  2. (With replacement) The failures form a Poisson process of rate $n/\theta$ on $[0,t_0]$, so $D\sim\operatorname{Poi}(nt_0/\theta)$, $T=nt_0$, and $\hat\theta=nt_0/D$ for $D\ge1$. The interval $$\left[\frac{2T}{\chi^2_{2D+2,\alpha/2}},\ \frac{2T}{\chi^2_{2D,1-\alpha/2}}\right]$$ has coverage at least $1-\alpha$ for every $\theta$ (for $D=0$ the upper end is $+\infty$).
  3. In both schemes $\hat\theta$ is not unbiased — it is undefined with positive probability — but as $n\to\infty$ it is consistent and asymptotically normal, with approximate variance $\hat\theta^2/D$; approximate intervals come from the likelihood ratio (§5.1).

Proof. (1) By Theorem 9.10(2) with exponential $f$ and $R$, $$L(\theta)\propto\theta^{-D}\exp\left(-\frac{\sum_{i\le D}x_{(i)}+(n-D)t_0}{\theta}\right),$$ that is, $L(\theta)\propto\theta^{-D}e^{-T/\theta}$, maximised as in Theorem 9.11 at $T/D$ when $D\ge1$. Each item fails before $t_0$ independently with probability $q=1-e^{-t_0/\theta}$, so $D\sim\operatorname{Bin}(n,q)$; the MLE of $q$ from $D$ is $D/n$, and by invariance of the MLE (§4.2) the MLE of $\theta=-t_0/\log(1-q)$ based on $D$ alone is $\tilde\theta$.

(2) In each of the $n$ positions the successive lifetimes are iid exponential with mean $\theta$, so the failures in one position form a Poisson process of rate $1/\theta$ (interarrival definition, §3.3), and the $n$ independent positions superpose into a Poisson process of rate $n/\theta$ (§3.3). The likelihood is the Poisson probability $e^{-nt_0/\theta}(nt_0/\theta)^D/D!\propto\theta^{-D}e^{-T/\theta}$, maximised at $nt_0/D$. For the interval write $\nu=nt_0/\theta$ for the Poisson mean. The arrival time of the $d$-th event of a unit-rate Poisson process is $\operatorname{Gamma}(d,1)$, so $P(D\ge d)=P(\chi^2_{2d}\le2\nu)$ (the gamma–Poisson link, §2.1). Hence $\{\chi^2_{2D,1-\alpha/2}\le2\nu\}$ fails only if $D$ is unusually large and $\{2\nu\le\chi^2_{2D+2,\alpha/2}\}$ fails only if $D$ is unusually small, each with probability at most $\alpha/2$; this is the classical (Garwood) interval for a Poisson mean. Substituting $\nu=T/\theta$ and solving for $\theta$ gives the stated interval.

(3) $P(D=0)\gt0$ in both schemes, so $E[\hat\theta]$ is not even defined. For large $n$, $D/n\to q$ almost surely and $T/n\to E[\min(X,t_0)]=\theta q$ (strong law), so $\hat\theta=T/D\to\theta$; asymptotic normality follows from the bivariate CLT and the delta method (§2.3), and the observed information $-\ell''(\hat\theta)=D/\hat\theta^2$ estimates the inverse variance. ∎

🧭 Problem-solving strategy — estimating a mean life from a censored exponential test
  1. Identify the scheme: stopped at a fixed number of failures (Type II) or at a fixed time (Type I); with or without replacement.
  2. Compute the total time on test: every failed item contributes its failure time; every survivor contributes the full duration of the test. With replacement, $T=n\times$(duration).
  3. Divide by the number of failures, never by $n$: $\hat\theta=T/r$ or $T/D$. Then $\hat R(t)=e^{-t/\hat\theta}$, and a hazard estimate is $1/\hat\theta$ (invariance).
  4. Interval: Type II, use $2T_r/\theta\sim\chi^2_{2r}$; Type I with replacement, use the Poisson interval with $2D$ and $2D+2$ degrees of freedom; Type I without replacement, use a likelihood-ratio or large-sample interval.
✏️ Example 6 — A Type II test of relays
Twelve relays with exponential lifetimes are tested without replacement until the fifth failure. The failures occur at $52$, $118$, $203$, $290$ and $371$ hours. Find the MLE of the mean life $\theta$, an exact $95\%$ confidence interval for $\theta$, the estimated reliability at $100$ hours, and a $95\%$ lower confidence bound for $R(100)$.
Time on test $n=12$, $r=5$: the failed relays contribute $52+118+203+290+371=1034$ hours and the $7$ survivors $7\times371=2597$ hours, so $T_5=3631$ hours.
MLE $\hat\theta=3631/5=726.2$ hours (Theorem 9.11). The naive average of the five failure times, $206.8$ hours, ignores the $2597$ hours of survival and is far too small.
Interval $2T_5=7262$ and $2r=10$ degrees of freedom: $\chi^2_{10,0.025}=20.483$, $\chi^2_{10,0.975}=3.247$. So $$\theta\in\left[\frac{7262}{20.483},\ \frac{7262}{3.247}\right]=[354.5,\ 2236.5]\ \text{hours}.$$ With only five failures the interval is wide and skewed to the right.
Reliability By invariance, $\hat R(100)=e^{-100/726.2}=0.871$. Since $R(100)=e^{-100/\theta}$ is increasing in $\theta$, a lower bound for $\theta$ gives one for $R(100)$: with $\chi^2_{10,0.05}=18.307$, $\theta\ge7262/18.307=396.7$ with $95\%$ confidence, so $R(100)\ge e^{-100/396.7}=0.777$.
Check The UMVUE of $R(100)$ is $\left(1-\frac{100}{T_5}\right)^{r-1}=0.894$ (Exercise IV.3); the MLE and UMVUE of a nonlinear function of $\theta$ need not agree.
✏️ Example 7 — A Type I test, three estimators, and a fallacy
Twenty fans are tested for $t_0=400$ hours without replacement; six fail, at $35$, $110$, $160$, $240$, $300$ and $385$ hours. (a) Find the MLE of the mean life $\theta$ and the estimate $\tilde\theta$ based on the count alone. (b) Had the test been run with replacement and produced six failures in $400$ hours, what would the MLE and the $95\%$ Poisson interval be? (c) Prove or disprove: "the average of the six observed failure times estimates $\theta$ without bias".
(a) MLE The failures contribute $1230$ hours and the $14$ survivors $14\times400=5600$ hours: $T=6830$ and $\hat\theta=6830/6=1138.3$ hours (Theorem 9.12(1)).
(a) Count only $\hat q=6/20=0.3$ estimates $P(X\le400)=1-e^{-400/\theta}$, so $\tilde\theta=-400/\log0.7=1121.5$ hours — close to the MLE, but it throws away the failure times.
(b) With replacement $T=20\times400=8000$ hours and $\hat\theta=8000/6=1333.3$ hours. With $\chi^2_{14,0.025}=26.119$ and $\chi^2_{12,0.975}=4.404$, $$\theta\in\left[\frac{16000}{26.119},\ \frac{16000}{4.404}\right]=[612.6,\ 3633]\ \text{hours}.$$
(c) False The observed failure times are lifetimes conditioned on being at most $400$, so each has mean $E[X\mid X\le t_0]\lt t_0=400$, however large $\theta$ is. Their average, $205$ hours here, estimates this conditional mean, not $\theta$; it is biased downwards and does not even converge to $\theta$ as $n\to\infty$. The same holds in Type II tests (Exercise IV.9). Censored items carry information, and the total time on test is how the likelihood uses it.
Interactive 2 — Life Testing with Type I and Type II Censoring

$n$ items with exponential lifetimes of true mean $\theta=100$ hours start together. Each lane grows while its item works; a red × marks a failure and a green ○ an item still working when the test stops — at the $r$-th failure (Type II) or at time $t_0$ (Type I). The lower panel shows the relative likelihood $L(\theta)/L(\hat\theta)$ with the MLE $\hat\theta=T/r$, the true value (dashed) and, when the test ends, the $95\%$ interval: exact $\chi^2_{2r}$ for Type II, likelihood-ratio for Type I. Press Play to run test after test and watch the running averages: $T/r$ is centred on $100$, the naive average of the failure times is not.

208
failures so far—
total time on test T—
MLE T / r—
naive mean of failures—
95% interval—
tests run · covered θ—
average MLE · average naive—
⚠️ Counterexamples and pitfalls
  1. A hazard rate is not a probability. $h(t)=2t$ is the hazard of a Rayleigh law and equals $6$ at $t=3$; only $h(t)\Delta$ for small $\Delta$ is (approximately) a probability.
  2. Not every nonnegative function is a hazard rate. $\int_0^\infty h=\infty$ is necessary (Theorem 9.1(3)); $h(t)=e^{-t}$ or $h(t)=1/(1+t)^2$ would leave a positive probability of never failing.
  3. Mixing reverses ageing. A mixture of exponential laws is strictly DFR (Theorem 9.4) although every unit has a constant hazard; mixtures of IFR laws need not be IFR.
  4. Parallel systems of IFR parts need not be IFR. $\max(T_1,T_2)$ with $T_1\sim\operatorname{Exp}(1)$, $T_2\sim\operatorname{Exp}(3)$ has a hazard that rises above its limit $1$ and falls back (Example 3(c)); with equal rates it is IFR. (Such systems are IFRA: coherent systems of independent IFRA components are IFRA — a classical theorem we only state.)
  5. Shifting a DFR life destroys DFR. If $T$ is DFR, $T+1$ has hazard $0$ on $[0,1)$ and $h(t-1)\gt0$ afterwards, so its hazard increases at $t=1$. Shifts and scalings preserve IFR, scalings preserve DFR (Theorem 9.3(5)).
  6. Redundancy can shorten the mean life. A 2-out-of-3 system of $\operatorname{Exp}(\lambda)$ units has MTTF $\frac{5}{6\lambda}\lt\frac1\lambda$; it is better than one unit only for missions shorter than $\frac{\log2}{\lambda}$ (Example 5).
  7. System MTTFs are not sums or minima of component MTTFs. Two iid $\operatorname{Exp}(\lambda)$ units in parallel give $\frac{3}{2\lambda}$, not $\frac2\lambda$ (that is cold standby); in series they give $\frac{1}{2\lambda}$, but two iid $\operatorname{Weib}(2,1)$ units in series give $\Gamma(1.5)/\sqrt2=0.627$, not half the component mean $0.443$.
  8. Series is not "the weakest component". $R_S=\prod R_i\lt\min R_i$ when the other reliabilities are below $1$: fifty components of reliability $0.99$ in series give only $0.605$.
  9. Redundancy belongs at component level. For two components of reliability $0.9$ in series, duplicating each gives $0.9801$, duplicating the whole system only $0.9639$ (Theorem 9.9(2)).
  10. Censored data: divide by failures, not items. The MLE is (total time on test)/(number of failures). Averaging the observed failure times, or dropping the survivors, underestimates the mean life (Example 7(c)).
  11. Degrees of freedom. In a Type II exponential test $2T_r/\theta\sim\chi^2_{2r}$ with $2r$ (not $2n$, not $r$) degrees of freedom; in a Type I test with replacement the exact interval uses $2D$ and $2D+2$.
  12. Type I MLEs can fail to exist. With no failures by $t_0$ the likelihood $e^{-nt_0/\theta}$ has no maximum; $T/D$ is therefore not an unbiased estimator in any Type I scheme.

Exercises

I
Reliability, hazard and mean life.
  • 1 $T$ has density $f(t)=\dfrac{2t}{(1+t^2)^2}$, $t\gt0$. Find $R$, $h$, $H$ and the MTTF. Where is $h$ largest? Is $T$ IFR or DFR?
  • 2 Show that $h(t)=\dfrac{c}{1+t}$ ($c\gt0$) is a hazard rate. Find $R$, decide when the MTTF is finite, and find $m(t)$.
  • 3 Explain why $h(t)=e^{-t}$ is not the hazard rate of a lifetime with $P(T\lt\infty)=1$, and describe the "defective" law it defines.
  • 4 Find the hazard rate and the mean residual life of the uniform lifetime $U(0,b)$. Is it IFR?
  • 5 Prove that if $m(t)=c$ for all $t\ge0$ then $T\sim\operatorname{Exp}(1/c)$, and more generally that $R(t)=\dfrac{m(0)}{m(t)}\exp\left(-\displaystyle\int_0^t\frac{du}{m(u)}\right)$. Check the formula on Exercise 2.
  • 6 For a lifetime $N$ on $\{1,2,\dots\}$ define the discrete hazard $h(n)=P(N=n\mid N\ge n)$. Show that $P(N\gt n)=\prod_{j=1}^n(1-h(j))$ and that $h$ is constant iff $N$ is geometric.
  • 7 Show that a lifetime with hazard $h(t)=a+bt$ ($a,b\gt0$) has the law of $\min(X,Y)$ for independent $X\sim\operatorname{Exp}(a)$ and $Y\sim\operatorname{Weib}\left(2,\sqrt{2/b}\right)$.
  • 8 For $R(t)=e^{-t^2}$ use Theorem 9.2 to find $E[T]$, $E[T^2]$ and $\operatorname{Var}(T)$.
1 $R(t)=\int_t^\infty f=\frac{1}{1+t^2}$, $H(t)=\log(1+t^2)$, $h(t)=\frac{2t}{1+t^2}$, and $\mu=\int_0^\infty\frac{dt}{1+t^2}=\frac\pi2$. $h'(t)=\frac{2(1-t^2)}{(1+t^2)^2}$, so $h$ increases on $[0,1]$, has maximum $h(1)=1$ and decreases afterwards: neither IFR nor DFR (an "upside-down bathtub", typical of log-logistic laws).
2 $h\ge0$ is continuous with $\int_0^\infty\frac{c}{1+t}dt=\infty$, so Theorem 9.1 applies: $H(t)=c\log(1+t)$ and $R(t)=(1+t)^{-c}$. $\mu=\int_0^\infty(1+t)^{-c}dt$ is finite iff $c\gt1$, and then $\mu=\frac1{c-1}$. For $c\gt1$, $m(t)=\frac{(1+t)^{1-c}/(c-1)}{(1+t)^{-c}}=\frac{1+t}{c-1}$, increasing: the law is DFR and IMRL.
3 $\int_0^\infty e^{-t}dt=1\lt\infty$, against Theorem 9.1(3). The formula $R(t)=\exp(-(1-e^{-t}))$ decreases only to $e^{-1}$: it describes a "lifetime" with $P(T=\infty)=e^{-1}\approx0.37$ — a unit that, with probability $0.37$, never fails (a cure fraction, in medical language).
4 $R(t)=1-t/b$ and $f=1/b$ on $[0,b)$, so $h(t)=\frac{1}{b-t}$, increasing to $\infty$ as $t\to b$: IFR. $m(t)=\frac{1}{1-t/b}\int_t^b(1-u/b)\,du=\frac{b-t}{2}$.
5 From Theorem 9.2(3), $h=\frac{1+m'}{m}$. If $m\equiv c$ then $h\equiv1/c$, so $T\sim\operatorname{Exp}(1/c)$. In general $$H(t)=\int_0^t\frac{du}{m(u)}+\int_0^t\frac{m'(u)}{m(u)}du,$$ and the second integral is $\log\frac{m(t)}{m(0)}$; then $R=e^{-H}$ is the stated formula. For Exercise 2, $m(u)=\frac{1+u}{c-1}$: $\int_0^t\frac{c-1}{1+u}du=(c-1)\log(1+t)$ and $\frac{m(0)}{m(t)}=\frac{1}{1+t}$, so $R(t)=(1+t)^{-1}(1+t)^{-(c-1)}=(1+t)^{-c}$.
6 $P(N\gt n\mid N\gt n-1)$ equals $P(N\ne n\mid N\ge n)=1-h(n)$; multiply these for $1,\dots,n$ (with $P(N\gt0)=1$). If $h\equiv p$ then $P(N\gt n)=(1-p)^n$ — the geometric law; conversely the geometric law has $h(n)=\frac{p(1-p)^{n-1}}{(1-p)^{n-1}}=p$. This is the discrete lack of memory (§2.1).
7 $\operatorname{Weib}(2,\lambda)$ has $H(t)=t^2/\lambda^2$, which equals $bt^2/2$ for $\lambda=\sqrt{2/b}$. By Theorem 9.3(4) the minimum has cumulative hazard $at+bt^2/2$, i.e. hazard $a+bt$. So a linear hazard is a series system of a "random shock" component and a "wear-out" component.
8 $E[T]=\int_0^\infty e^{-t^2}dt=\frac{\sqrt\pi}2$; $E[T^2]=2\int_0^\infty te^{-t^2}dt=1$; $\operatorname{Var}(T)=1-\frac\pi4=0.2146$. (This is $\operatorname{Weib}(2,1)$: $\Gamma(1.5)=\frac{\sqrt\pi}2$, $\Gamma(2)=1$.)
II
Ageing classes and standard lifetime laws.
  • 1 Components have $\operatorname{Weib}(0.7,2000)$ lifetimes (hours). Find the MTTF, $R(500)$, $h(500)$ and $h(2000)$. Would a burn-in period make sense for them?
  • 2 Find the hazard rate of $\operatorname{Gamma}(2,\lambda)$ directly and show that it increases to $\lambda$.
  • 3 Show that if $T$ is IFR then so are $cT$ and $T+c$ for every $c\gt0$. Which of the two statements remains true when IFR is replaced by DFR?
  • 4 A unit has cumulative hazard $H(t)=\sqrt t+t^2/16$. Show that its hazard rate has a bathtub shape, find where it is smallest, compute $R(4)$, and interpret $T$ as a series system.
  • 5 Prove that if $T$ is DFR (with a continuous density) and has mean $\mu$, then $R(t)\le e^{-t/\mu}$ for $0\le t\le\mu$.
  • 6 (Weibull plot) A Weibull lifetime has $R(100)=0.9$ and $R(400)=0.5$. Using $\log(-\log R(t))=k\log t-k\log\lambda$, find $k$, $\lambda$ and the MTTF.
  • 7 $n$ identical $\operatorname{Weib}(2,\lambda)$ components are connected in series. Show that the system MTTF is $n^{-1/2}$ times the component MTTF. How many components halve the MTTF?
  • 8 (Competing risks) $X\sim\operatorname{Exp}(\lambda)$ and $Y\sim\operatorname{Exp}(\nu)$ are independent failure causes. Show that $P(X\lt Y)=\frac{\lambda}{\lambda+\nu}$ and that $\min(X,Y)$ is independent of the event $\{X\lt Y\}$.
1 $\mu=2000\,\Gamma(1+1/0.7)$, which is $2000\,\Gamma(2.4286)=2531.6$ h. $R(500)=e^{-0.25^{0.7}}=0.6846$. $h(t)=\frac{0.7}{2000}\left(\frac{t}{2000}\right)^{-0.3}$: $h(500)=0.000531$ and $h(2000)=0.00035$ per hour. The hazard decreases ($k\lt1$, DFR), so running units for a while before delivery removes the weak ones and the survivors are more reliable — burn-in makes sense.
2 $f(t)=\lambda^2te^{-\lambda t}$ and $R(t)=\int_t^\infty\lambda^2se^{-\lambda s}ds=(1+\lambda t)e^{-\lambda t}$, so $h(t)=\frac{\lambda^2t}{1+\lambda t}=\lambda-\frac{\lambda}{1+\lambda t}$, which increases to $\lambda$.
3 $cT$: $P(cT\gt t+x)/P(cT\gt t)=R\left(\frac tc+\frac xc\right)/R\left(\frac tc\right)$, nonincreasing in $t$. $T+c$: its conditional survival is $1$ for $t+x\le c$, equals $R(t+x-c)$ for $t\lt c\lt t+x$ and $R(t+x-c)/R(t-c)$ for $t\ge c$; each piece is nonincreasing in $t$ and they join continuously, so $T+c$ is IFR. For DFR the scaling statement survives, the shift does not: $T+c$ has zero hazard on $[0,c)$ and positive hazard after (pitfall 5).
4 $h(t)=\frac{1}{2\sqrt t}+\frac t8$, which tends to $\infty$ at $0$ and at $\infty$; $h'(t)=-\frac{1}{4t^{3/2}}+\frac18$ vanishes only at $t=2^{2/3}=1.587$, where $h=0.595$ is the minimum. $R(4)=e^{-(2+1)}=e^{-3}=0.0498$. $H$ is the sum of the cumulative hazards of $\operatorname{Weib}(\frac12,1)$ (early failures) and $\operatorname{Weib}(2,4)$ (wear-out), so $T$ is a series system of independent components with these lifetimes (Theorem 9.3(4)).
5 Now $H$ is concave with $H(0)=0$, so Jensen gives $H(\mu)\ge E[H(T)]=1$ (Theorem 9.1(4)). For $0\le t\le\mu$ concavity gives $H(t)\ge\left(1-\frac t\mu\right)H(0)+\frac t\mu H(\mu)\ge\frac t\mu$, so $R(t)=e^{-H(t)}\le e^{-t/\mu}$. (If $\mu=\infty$ the statement is empty.)
6 Dividing the two equations $-\log0.9=(100/\lambda)^k$ and $\log2=(400/\lambda)^k$ gives $4^k=\frac{\log2}{-\log0.9}=6.579$, so $k=\frac{\log6.579}{\log4}=1.359$. Then $\lambda=100\,(0.10536)^{-1/k}=523.8$ and $\mu=\lambda\Gamma(1+1/k)=479.8$.
7 By Theorem 9.6(2) the series system is $\operatorname{Weib}(2,\lambda n^{-1/2})$, with MTTF $\lambda n^{-1/2}\Gamma(1.5)$. Halving needs $n^{-1/2}=\frac12$, i.e. $n=4$ (for exponential components $n=2$ would suffice).
8 Conditioning on $X$, $P(\min(X,Y)\gt t,\ X\lt Y)$ equals $$\int_t^\infty\lambda e^{-\lambda x}P(Y\gt x)\,dx,$$ which is $\frac{\lambda}{\lambda+\nu}e^{-(\lambda+\nu)t}$. At $t=0$ this is $P(X\lt Y)=\frac\lambda{\lambda+\nu}$, and for every $t$ it equals $P(X\lt Y)\cdot P(\min\gt t)$ — independence.
III
Systems.
  • 1 Four iid components have reliability $p$. Compare the reliability of $(A\parallel B)$ in series with $(C\parallel D)$ with that of $(A\text{–}B)\parallel(C\text{–}D)$, in general and for $p=0.9$.
  • 2 Derive the bridge reliability $2p^2+2p^3-5p^4+2p^5$ by pivoting on the bridge component, and evaluate it at $p=0.9$.
  • 3 Three iid $\operatorname{Exp}(0.01)$ units (per hour) work in parallel. Find the system MTTF and the standard deviation of its life.
  • 4 Two iid $\operatorname{Exp}(\lambda)$ units are used either in hot redundancy (parallel) or in cold standby with perfect switching. Compare the reliabilities at $t=1/\lambda$ and the MTTFs.
  • 5 Show that a $k$-out-of-$n$ system of iid components of reliability $p$ has $$r(p)=\frac{n!}{(k-1)!\,(n-k)!}\int_0^pu^{k-1}(1-u)^{n-k}du,$$ and deduce that $r$ is increasing in $p$.
  • 6 How many identical components of reliability $0.8$ must be placed in parallel for a system reliability of at least $0.999$?
  • 7 A device has $50$ components in series, of equal reliability $p$. How large must $p$ be for the device reliability to be at least $0.95$?
  • 8 Prove that every coherent system with independent components satisfies $\prod_ip_i\le r(p)\le1-\prod_i(1-p_i)$: series is the worst and parallel the best arrangement.
1 Component-level redundancy: $(2p-p^2)^2$. System-level: $1-(1-p^2)^2=2p^2-p^4$. Their difference is $(2p-p^2)^2-(2p^2-p^4)=2p^2(1-p)^2\ge0$, as Theorem 9.9(2) predicts. At $p=0.9$: $0.9801$ against $0.9639$.
2 If the bridge works, the system is $(1\parallel3)$ in series with $(2\parallel4)$: $(2p-p^2)^2$. If it fails, it is $(1\text{–}2)\parallel(3\text{–}4)$: $2p^2-p^4$. So $r=p(2p-p^2)^2+(1-p)(2p^2-p^4)$. Expanding, the first term is $4p^3-4p^4+p^5$ and the second is $2p^2-2p^3-p^4+p^5$, which add to $2p^2+2p^3-5p^4+2p^5$. At $p=0.9$: $r=0.97848$.
3 Theorem 9.8(3): $\mu=100\left(1+\frac12+\frac13\right)=183.3$ h. By Rényi's representation the system life is $\frac{D_1}{3}+\frac{D_2}{2}+D_3$ with $D_i$ iid of variance $100^2$, so $\operatorname{Var}=100^2\left(\frac19+\frac14+1\right)$ and $\operatorname{SD}=100\cdot\frac76=116.7$ h.
4 Hot: $R=1-(1-e^{-\lambda t})^2$, which is $1-(1-e^{-1})^2=0.6004$ at $t=1/\lambda$, and $\mu=\frac3{2\lambda}$. Cold: $R=(1+\lambda t)e^{-\lambda t}$, which is $2e^{-1}=0.7358$ at $t=1/\lambda$, and $\mu=\frac2\lambda$. Cold standby wins at every $t\gt0$: with $x=\lambda t$, the difference $(1+x)e^{-x}-(2e^{-x}-e^{-2x})$ equals $e^{-x}\left(x-1+e^{-x}\right)\gt0$, because $e^{-x}\gt1-x$. In practice it needs a reliable switch.
5 $r(p)=P(\operatorname{Bin}(n,p)\ge k)$. If $U_1,\dots,U_n$ are iid $U(0,1)$, then $\mathbf 1\{U_i\le p\}$ are iid Bernoulli$(p)$, so $r(p)=P(U_{(k)}\le p)$, and $U_{(k)}\sim\operatorname{Beta}(k,n-k+1)$ (§4.1), whose density is the integrand. The derivative $\frac{n!}{(k-1)!(n-k)!}p^{k-1}(1-p)^{n-k}$ is positive on $(0,1)$.
6 $1-0.2^n\ge0.999$ means $0.2^n\le0.001$, i.e. $n\ge\frac{\log0.001}{\log0.2}=4.29$: five components.
7 $p^{50}\ge0.95\iff p\ge0.95^{1/50}=0.99897$: each component must fail with probability about $1$ in $1000$.
8 Coherence gives $\varphi(\mathbf 1)=1$ and $\varphi(\mathbf 0)=0$: if component $i$ is relevant at $x$, monotonicity forces $\varphi(1_i,x)=1$ and $\varphi(0_i,x)=0$, so $\varphi(\mathbf 1)\ge1$ and $\varphi(\mathbf 0)\le0$. Hence $\min_ix_i\le\varphi(x)\le\max_ix_i$: if some $x_i=0$ the left side is $0$, and otherwise $x=\mathbf 1$; if all $x_i=0$ the right side is $\varphi(\mathbf 0)=0$. Take expectations: $E[\min X_i]=\prod p_i$ and $E[\max X_i]=1-\prod(1-p_i)$.
IV
Censoring and life testing (lifetimes exponential with mean $\theta$ unless stated otherwise).
  • 1 Ten items are tested until the fourth failure, which occurs at $210$ h; the earlier failures were at $30$, $75$ and $140$ h. Find $\hat\theta$ and an exact $90\%$ confidence interval for $\theta$.
  • 2 A Type II test is to stop at the fifth failure, with $\theta=1000$ h. Find the expected duration of the test when $n=5$, $10$ and $20$ items are used, and the variance of $\hat\theta$ in each case.
  • 3 In a Type II test show that $\left(1-t_0/T_r\right)^{r-1}\mathbf 1\{T_r\gt t_0\}$ is unbiased for $R(t_0)=e^{-t_0/\theta}$, and explain why it is the UMVUE. Evaluate it for Example 6.
  • 4 Fifteen items are tested without replacement for $200$ h; failures occur at $40$, $95$, $150$ and $180$ h. Find the MLE of $\theta$ and the estimate based on the number of failures only.
  • 5 In a Type I test without replacement, find the probability that the MLE does not exist. Evaluate it when $n=10$ and $t_0=0.1\theta$.
  • 6 Ten positions are kept filled (with replacement) for $500$ h and $4$ failures occur. Find $\hat\theta$ and the $95\%$ Poisson confidence interval for $\theta$.
  • 7 Lifetimes are $\operatorname{Weib}(k,\lambda)$ with known shape $k$. Show that $X_i^k$ is exponential with mean $\lambda^k$, and find the MLE of $\lambda$ and an exact confidence interval for $\lambda^k$ from a Type II censored sample.
  • 8 (Random censoring) With independent censoring times whose law is free of $\theta$, show that the MLE is $\hat\theta=\sum_iy_i/\sum_i\delta_i$ and that the Fisher information is $E\left[\sum_i\delta_i\right]/\theta^2$.
  • 9 In a Type II test, show that the average of the $r$ observed failure times has expectation $\dfrac\theta r\displaystyle\sum_{j=1}^r\frac{r-j+1}{n-j+1}$, which is less than $\theta$ when $r\lt n$. Evaluate the factor for Example 6.
1 $T_4=30+75+140+210+6\times210=1715$, $\hat\theta=428.75$ h. With $8$ degrees of freedom, $\chi^2_{8,0.05}=15.507$ and $\chi^2_{8,0.95}=2.733$: $$\theta\in\left[\frac{3430}{15.507},\ \frac{3430}{2.733}\right]=[221.2,\ 1255.2]\ \text{h}.$$
2 $E[X_{(5)}]=1000\sum_{i=1}^5\frac1{n-i+1}$: $1000\left(1+\frac12+\frac13+\frac14+\frac15\right)=2283.3$ h for $n=5$; $1000\left(\frac1{10}+\frac19+\frac18+\frac17+\frac16\right)=645.6$ h for $n=10$; $279.5$ h for $n=20$. In every case $\operatorname{Var}(\hat\theta)=\theta^2/5=200\,000$ h² (Theorem 9.11): more items shorten the test without improving the estimate.
3 $T_r$ has density $\frac{s^{r-1}e^{-s/\theta}}{\Gamma(r)\theta^r}$, so $E\left[\left(1-\tfrac{t_0}{T_r}\right)^{r-1};T_r\gt t_0\right]$ equals $$\int_{t_0}^\infty\frac{(s-t_0)^{r-1}e^{-s/\theta}}{\Gamma(r)\theta^r}ds,$$ and the substitution $s=t_0+v$ turns the integral into $e^{-t_0/\theta}$ times a gamma density integrated over $(0,\infty)$, i.e. $e^{-t_0/\theta}$. It is a function of the complete sufficient statistic $T_r$, hence the UMVUE (Lehmann–Scheffé). For Example 6: $(1-100/3631)^4=0.894$.
4 $T=40+95+150+180+11\times200=2665$, $D=4$, $\hat\theta=666.25$ h. From the count: $\tilde\theta=-200/\log(1-4/15)=644.8$ h.
5 The MLE fails to exist iff $D=0$, i.e. no item fails by $t_0$: probability $\left(e^{-t_0/\theta}\right)^n=e^{-nt_0/\theta}$. For $n=10$, $t_0=0.1\theta$: $e^{-1}=0.368$ — a test that is too short or too small is likely to see nothing.
6 $T=10\times500=5000$, $\hat\theta=5000/4=1250$ h. With $\chi^2_{10,0.025}=20.483$ and $\chi^2_{8,0.975}=2.180$: $$\theta\in\left[\frac{10000}{20.483},\ \frac{10000}{2.180}\right]=[488.2,\ 4588]\ \text{h}.$$
7 $P(X^k\gt s)=P(X\gt s^{1/k})=e^{-s/\lambda^k}$. Since $x\mapsto x^k$ is increasing, the censoring pattern is unchanged, and Theorem 9.11 applies to $Y_i=X_i^k$: the MLE of $\lambda^k$ is $\frac1r\left(\sum_{i=1}^rx_{(i)}^k+(n-r)x_{(r)}^k\right)$, and by invariance $\hat\lambda$ is its $k$-th root. With $W$ this total, $2W/\lambda^k\sim\chi^2_{2r}$, giving $\lambda^k\in\left[\frac{2W}{\chi^2_{2r,\alpha/2}},\frac{2W}{\chi^2_{2r,1-\alpha/2}}\right]$.
8 By Theorem 9.10(3), $$L\propto\prod_i\left(\theta^{-1}e^{-y_i/\theta}\right)^{\delta_i}\left(e^{-y_i/\theta}\right)^{1-\delta_i},$$ that is, $L\propto\theta^{-\sum\delta_i}e^{-\sum y_i/\theta}$, maximised at $\sum y_i/\sum\delta_i$. Next, $-\ell''=-\frac{\sum\delta_i}{\theta^2}+\frac{2\sum y_i}{\theta^3}$. For one item with censoring reliability $G$, $E[\delta]=\int_0^\infty\theta^{-1}e^{-t/\theta}G(t)\,dt$ and $E[Y]=\int_0^\infty e^{-t/\theta}G(t)\,dt=\theta E[\delta]$ (Theorem 9.2). So $I(\theta)=\frac{E\sum\delta_i}{\theta^2}$: again, information comes from failures.
9 By Rényi's representation $X_{(i)}=\sum_{j\le i}\frac{D_j}{n-j+1}$, so $\sum_{i=1}^rX_{(i)}=\sum_{j=1}^rD_j\frac{r-j+1}{n-j+1}$ (each $D_j$ appears in $X_{(j)},\dots,X_{(r)}$). Taking expectations and dividing by $r$ gives the formula; each ratio $\frac{r-j+1}{n-j+1}$ is less than $1$ when $r\lt n$. For $n=12$, $r=5$ the factor is $0.2855$: the naive average estimates about $29\%$ of $\theta$ — in Example 6, $206.8/726.2=0.285$.
🎯 Exam corner CSIR Part BCSIR Part CGATE ST

E1. CSIR Part B A lifetime $T$ has hazard rate $h(t)=2t$, $t\gt0$. Then $P(T\gt2\mid T\gt1)$ equals (A) $e^{-1}$; (B) $e^{-2}$; (C) $e^{-3}$; (D) $e^{-4}$.

Idea $R(t)=e^{-H(t)}$ with $H(t)=t^2$, so $P(T\gt2\mid T\gt1)=e^{-(H(2)-H(1))}$.
Compute $H(2)-H(1)=4-1=3$. Option (D) is $R(2)$, and (A) is $R(1)$ — both forget the conditioning or the ageing.
Answer (C) $e^{-3}$.

E2. CSIR Part B GATE ST A 2-out-of-3 system has three independent components with exponential lifetimes of mean $600$ hours. Its mean time to failure is (A) $300$ h; (B) $500$ h; (C) $600$ h; (D) $1100$ h.

Idea Theorem 9.8(3): $\mu_S=\frac1\lambda\sum_{j=k}^n\frac1j$ with $\frac1\lambda=600$, $k=2$, $n=3$.
Compute $600\left(\frac12+\frac13\right)=500$. (D) is the parallel system ($600\cdot\frac{11}6$), (C) a single unit, and (A) keeps only the term $\frac12$ of the sum.
Answer (B) $500$ h.

E3. CSIR Part B GATE ST Ten items with exponential lifetimes are put on test without replacement, and the test is stopped at the fourth failure. The failure times are $20$, $50$, $90$ and $140$ hours. The maximum likelihood estimate of the mean life is (A) $75$ h; (B) $114$ h; (C) $285$ h; (D) $350$ h.

Idea $\hat\theta=T_r/r$ with $T_r=\sum_{i\le r}x_{(i)}+(n-r)x_{(r)}$ (Theorem 9.11).
Compute $T_4=300+6\times140=1140$ and $\hat\theta=1140/4=285$. (A) is the naive mean of the failures, (B) divides by $n=10$ instead of $r$, (D) uses $x_{(r)}$ for all ten items.
Answer (C) $285$ h.

E4. CSIR Part C Let $T$ be a lifetime with a continuous density, finite mean and hazard rate $h$. Which of the following statements are true? (One or more options may be correct.) (A) If $h$ is nondecreasing, then the mean residual life $m(t)$ is nonincreasing. (B) If $h$ is constant, then $P(T\gt s+t\mid T\gt s)=P(T\gt t)$ for all $s,t\ge0$. (C) $\int_0^\infty h(t)\,dt$ may be finite. (D) If $T$ is DFR, then $T+1$ is DFR.

(A) True IFR implies DMRL (Theorem 9.3(2)): $m(t)=\int_0^\infty R(t+x)/R(t)\,dx$ with a nonincreasing integrand.
(B) True Constant hazard $\lambda$ means $T\sim\operatorname{Exp}(\lambda)$, which is memoryless: $e^{-\lambda(s+t)}/e^{-\lambda s}=e^{-\lambda t}$.
(C) False $R(t)=e^{-H(t)}\to0$ forces $H(t)\to\infty$ (Theorem 9.1(3)); a finite integral would leave $P(T=\infty)\gt0$.
(D) False $T+1$ cannot fail before time $1$, so its hazard is $0$ on $[0,1)$ and $h_T(t-1)\gt0$ just after $1$ (e.g. $T\sim\operatorname{Exp}(1)$): it increases, so $T+1$ is not DFR.
Answer (A) and (B).

E5. CSIR Part C Components work independently. Which of the following are true? (One or more options may be correct.) (A) A series system is never more reliable than its least reliable component. (B) A 2-out-of-3 system of iid components is more reliable than a single component for every component reliability $p\in(0,1)$. (C) For two components in series, putting a spare in parallel with each component is at least as reliable as putting a second copy of the whole system in parallel. (D) The MTTF of a parallel system of $n$ iid exponential components tends to $\infty$ as $n\to\infty$.

(A) True $\prod_ip_i\le p_j$ for every $j$, because the other factors are at most $1$.
(B) False $3p^2-2p^3\gt p$ iff $p\gt\frac12$ (Example 5(d)); at $p=0.3$ the system has reliability $0.216\lt0.3$.
(C) True The redundancy principle (Theorem 9.9(2)); for $p=0.9$, $0.9801\ge0.9639$, and in general the difference is $2p^2(1-p)^2$ (Exercise III.1).
(D) True $\mu_n=\frac1\lambda\sum_{j=1}^n\frac1j$, and the harmonic series diverges (like $\log n$).
Answer (A), (C) and (D).

E6. CSIR Part C GATE ST In a Type II censored test of $n$ items with exponential lifetimes of mean $\theta$, stopped at the $r$-th failure ($r\lt n$), let $T_r$ be the total time on test. Which of the following are true? (One or more options may be correct.) (A) $T_r/r$ is an unbiased estimator of $\theta$. (B) $2T_r/\theta$ has the $\chi^2_{2r}$ distribution. (C) $\operatorname{Var}(T_r/r)=\theta^2/n$. (D) For fixed $t_0\gt0$, the MLE $e^{-rt_0/T_r}$ of $R(t_0)=e^{-t_0/\theta}$ is unbiased.

(A) True $T_r$ is a sum of $r$ iid exponential spacings of mean $\theta$ (Theorem 9.11(2)).
(B) True $T_r\sim\operatorname{Gamma}(r,1/\theta)$, so $2T_r/\theta\sim\operatorname{Gamma}(r,\frac12)=\chi^2_{2r}$.
(C) False $\operatorname{Var}(T_r/r)=\theta^2/r$, which does not depend on $n$.
(D) False $T_r$ is complete, so there is only one unbiased function of $T_r$ for $e^{-t_0/\theta}$, namely $(1-t_0/T_r)^{r-1}\mathbf 1\{T_r\gt t_0\}$ (Exercise IV.3). The MLE is a different function of $T_r$ (it is positive when $T_r\le t_0$), so it is biased. Numerically, for $r=5$, $\theta=500$, $t_0=100$: $E[e^{-5t_0/T_5}]=0.786$ while $e^{-0.2}=0.819$.
Answer (A) and (B).
🧑‍🏫 For the teacher

Suggested plan — 3 lectures × 60 min.

  1. Hazard and ageing. Motivate with a population of light bulbs: "of those still burning at age $t$, what fraction fails this week?" Definition 9.1, Theorem 9.1 with its proof (ten minutes), Theorem 9.2 and Example 1 on the board. Then Definition 9.3 and Theorem 9.3; finish with Theorem 9.4 and the selection explanation — ask the class to predict, before the computation, whether a mixture of two exponential batches ages.
  2. Lifetime laws and systems. Theorem 9.5 (a two-line Jensen proof students enjoy), Theorem 9.6 and Example 2; the Weibull plot. Definition 9.4 and Theorems 9.7–9.8; project Interactive 1, moving $k$ from $n$ (series) to $1$ (parallel), then set $n=3$, $k=2$ and let students find the crossing time $\log2$ before you reveal Example 5. Do Example 4 and Theorem 9.9 with the two block diagrams of Exercise III.1.
  3. Censoring. Start from the fallacy of Example 7(c): ask for an estimate of the mean life from Example 6's data and collect the class's answers. Definition 9.5, Theorem 9.10, Theorem 9.11 with Rényi's representation, Example 6; Interactive 2 in Type II mode, then Type I with a short $t_0$ to show tests with no failures. Theorem 9.12 and Example 7; close with Exam-corner E3 and E6 done live.

Discussion questions. (1) Human mortality after childhood is close to Gompertz, $h(t)=ae^{bt}$. Is it IFR? What does a decreasing infant hazard say about the mixture of a population? (2) Why can adding a third computer to a two-out-of-three voting system lower the mean life of the system, and why do spacecraft designers still use it? (3) A Type II test with $n=100$ items and $r=5$ failures and one with $n=5$ items run to the end give estimates with the same variance. Why would anyone run the second?

Misconceptions to watch for. Treating $h(t)$ as a probability; "IFR" read as strictly increasing (so that the exponential law is excluded); assuming a parallel system of IFR components is IFR, or that a mixture of IFR laws is IFR; adding MTTFs for a parallel system; believing redundancy always increases MTTF; dividing the total time on test by $n$ instead of the number of failures; using $2n$ degrees of freedom; treating Type I and Type II tests as interchangeable.

Suggested homework. I(1, 2, 5, 7), II(1, 4, 5, 6), III(1, 2, 4, 8), IV(1, 2, 3, 6, 9); Exam corner E2, E4, E5.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. When only a handful of units have failed, how well can we tell wear-out from early failure? Compare two ways of estimating the Weibull shape $k$ from censored data — the slope of a Weibull probability plot and the maximum likelihood estimate — for small samples.

Why it matters. Whether $k$ is above or below $1$ decides maintenance policy: replace bearings, wind-turbine gearboxes or aircraft pumps on a schedule (wear-out), or leave them alone and only burn in new units (early failures). Reliability engineers make this call from a few failures, and several cloud-storage providers publish quarterly failure statistics for tens of thousands of hard drives, which show all three regimes of the bathtub curve.

What to do.

  1. Simulate $5000$ complete samples of size $n=10$ from $\operatorname{Weib}(1.5,1)$ (use $T=\lambda(-\log U)^{1/k}$, Theorem 9.1(4)).
  2. For each sample compute (a) the plot estimate: the least-squares slope of $\log(-\log(1-\hat F_i))$ on $\log x_{(i)}$ with median ranks $\hat F_i=(i-0.3)/(n+0.4)$; (b) the MLE of $k$, which solves $\dfrac{\sum w_i\log y_i}{\sum w_i}-\dfrac1k=\dfrac1r\sum_{i\le r}\log x_{(i)}$, where $y_i$ are the times on test and $w_i=y_i^k$ (derive this from Theorem 9.10).
  3. Report the mean and the root-mean-square error of both estimators. Repeat with $n=20$ items censored at the $10$th failure (Type II), and with $n=20$ complete.
  4. Find the smallest number of failures for which the MLE classifies a $k=1.5$ population as wearing out ($\hat k\gt1$) in at least $90\%$ of samples.

Tools: Python (NumPy, SciPy's root finders) or R; a spreadsheet is enough for the plot estimate.

Hand in: A table of bias and RMSE for the three designs, one Weibull plot of a simulated censored sample, and a short recommendation for an engineer who has seen ten failures.

Going further. Read about the bias-corrected MLE of the Weibull shape and about the IFRA closure theorem (coherent systems of independent IFRA components are IFRA); then download a public hard-drive failure data set and estimate the hazard of one drive model by age.

Instructor note: in our simulation (20 000 samples each) the MLE of $k=1.5$ has mean about $1.76$ for $n=10$ complete (bias $\approx+17\%$), about $1.82$ for $n=20$ censored at $r=10$, and about $1.62$ for $n=20$ complete; the median-rank plot estimate is nearly unbiased in these designs (means $1.44$–$1.50$) with a similar or slightly smaller RMSE ($\approx0.48$, $0.61$, $0.34$ against $0.59$, $0.72$, $0.33$ for the MLE). The lesson: with few failures both estimates are very noisy, and the MLE is biased upward — it tends to see wear-out that is not there.

🧪 Self-check quiz

0/5 correct

Q1. A lifetime has hazard rate $h(t)=3t^2$. Then $R(1)=P(T\gt1)$ equals

Q2. Two independent components with $\operatorname{Exp}(\lambda_1)$ and $\operatorname{Exp}(\lambda_2)$ lifetimes are connected in series. The system lifetime is

Q3. The Weibull law $\operatorname{Weib}(k,\lambda)$ has a decreasing failure rate exactly when

Q4. Two iid $\operatorname{Exp}(\lambda)$ units in parallel (both working from the start) have mean time to failure

Q5. Eight items with exponential lifetimes are tested without replacement until the third failure; the failures occur at $10$, $20$ and $30$ hours. The MLE of the mean life is