5.5 Numerical Solution of ODEs
Picard's successive approximations and the Taylor series method; Euler, modified Euler (Heun) and Runge–Kutta methods of orders 2 and 4; local and global truncation error and order; absolute stability for $y'=\lambda y$; systems and higher-order equations.
Weather forecasts, satellite orbits, the spread of an epidemic, the current in a circuit, the concentration of a drug in the blood: each is the solution of an initial value problem $y'=f(x,y)$, $y(x_0)=y_0$, and in almost every case the solution has no formula. The Picard–Lindelöf theorem (§1.1, Existence and Uniqueness of Solutions) tells us that a unique solution exists when $f$ is continuous and Lipschitz in $y$; this section is about computing it. The answer is a table of approximations $y_n\approx y(x_n)$ at grid points $x_n=x_0+nh$, produced step by step from a rule that uses only values of $f$.
We start with two "analytic" methods that produce formulas — Picard's successive approximations and the Taylor series method — and then turn to the workhorses: Euler's method, its modification by Heun, and the Runge–Kutta methods, culminating in the classical fourth-order method RK4 that is still the default in many codes. Two ideas organise everything: order (how fast the error falls as $h\to0$; local error $O(h^{p+1})$ per step, global error $O(h^p)$) and stability (whether, for a fixed $h$, errors are damped or amplified). CSIR NET asks for one or two steps of Euler, modified Euler or RK4, the Picard iterates of a simple equation, the order of a method, and the step size beyond which a method blows up on $y'=\lambda y$. Throughout, $f$ is continuous on $[x_0,X]\times\mathbb{R}$ and Lipschitz in $y$ with constant $L$, so the solution $y$ exists on $[x_0,X]$ (§1.1); $y_n$ is the computed value, $f_n=f(x_n,y_n)$, and $e_n=y(x_n)-y_n$ is the global error.
Picard's Method and the Taylor Series Method
Picard's method is the proof of the existence theorem used as an algorithm: starting from $\varphi_0(x)=y_0$, compute
$$\varphi_{n+1}(x)=y_0+\int_{x_0}^xf\big(t,\varphi_n(t)\big)\,dt\qquad(n\ge0).$$In §1.1 we proved that on the Picard interval $|x-x_0|\le h_0=\min(a,b/M)$ the iterates converge uniformly to the solution, with a factorially small error. As a computational method its key property is that each iteration fixes one more Taylor coefficient.
Let $f$ be continuous on $R=\{|x-x_0|\le a,\ |y-y_0|\le b\}$, $|f|\le M$ there, and Lipschitz in $y$ with constant $L$; let $y$ be the solution on $I=[x_0-h_0,x_0+h_0]$, $h_0=\min(a,b/M)$, and $\varphi_n$ its Picard iterates. Then for every $n\ge0$ and $x\in I$
$$|y(x)-\varphi_n(x)|\le\frac{ML^n}{(n+1)!}\,|x-x_0|^{n+1}.$$Consequently, if $f\in C^n$ near $(x_0,y_0)$ and $\varphi_n$ is a polynomial, then the terms of degree $\le n$ of $\varphi_n$ (expanded in powers of $x-x_0$) are exactly the Taylor polynomial of degree $n$ of $y$ at $x_0$.
Proof. By Steps 1 and 5 of the proof of the Picard–Lindelöf theorem (§1.1), the graphs of $y$ and of every $\varphi_n$ lie in $R$. For $n=0$: $|y(x)-y_0|=\big|\int_{x_0}^xf(t,y(t))\,dt\big|\le M|x-x_0|$. If the bound holds for $n$, then for $x\ge x_0$ (the other side is symmetric)
$$|y(x)-\varphi_{n+1}(x)|\le\int_{x_0}^x\big|f(t,y)-f(t,\varphi_n)\big|\,dt\le L\int_{x_0}^x\frac{ML^n(t-x_0)^{n+1}}{(n+1)!}\,dt,$$which equals $\dfrac{ML^{n+1}(x-x_0)^{n+2}}{(n+2)!}$. For the second statement: if $f\in C^n$, then $y'=f(x,y)$ shows inductively that $y\in C^{n+1}$, so $y-T_n=O(|x-x_0|^{n+1})$, where $T_n$ is the Taylor polynomial of $y$. With the bound just proved, $\varphi_n-T_n=O(|x-x_0|^{n+1})$. If $P$ denotes the part of degree $\le n$ of the polynomial $\varphi_n$, then $P-T_n$ is a polynomial of degree $\le n$ that is $O(|x-x_0|^{n+1})$ as $x\to x_0$, and such a polynomial is identically zero (divide by the lowest power present and let $x\to x_0$). ∎
Example 1 shows both the strength and the weakness of Picard's method. It produces approximations valid on a whole interval, with an error bound, but each iteration requires a symbolic integration whose complexity grows rapidly, and for most $f$ (say $f=\sin(xy)$) the integrals cannot be done in closed form at all. Since Theorem 5.45 says that the useful output of $n$ iterations is the Taylor polynomial of degree $n$, we may as well compute that polynomial directly.
Let $f$ be $p$ times continuously differentiable. Define the total derivatives $f^{[0]}=f$ and $f^{[k+1]}=\partial_xf^{[k]}+f\,\partial_yf^{[k]}$, so that $y^{(k+1)}(x)=f^{[k]}(x,y(x))$ for every solution; for instance
$$y''=f_x+ff_y,$$ $$y'''=f_{xx}+2ff_{xy}+f^2f_{yy}+f_y\big(f_x+ff_y\big).$$The Taylor series method of order $p$ is $$y_{n+1}=y_n+hf^{[0]}_n+\frac{h^2}{2!}f^{[1]}_n+\cdots+\frac{h^p}{p!}f^{[p-1]}_n,$$ with all $f^{[k]}$ evaluated at $(x_n,y_n)$. For $p=1$ this is Euler's method.
The Taylor method is accurate and conceptually simple, but each new $f$ needs its own derivative formulas, which become unwieldy for systems. Runge–Kutta methods achieve the same order using only values of $f$.
Euler, Modified Euler and Runge–Kutta Methods
A one-step method computes $y_{n+1}=y_n+h\,\Phi(x_n,y_n;h)$ with an increment function $\Phi$ built from values of $f$. Its local truncation error at step $n+1$ is what the exact solution fails to satisfy:
$$T_{n+1}=y(x_{n+1})-y(x_n)-h\,\Phi\big(x_n,y(x_n);h\big).$$The method has order $p$ if $T_{n+1}=O(h^{p+1})$ for every sufficiently smooth $f$ (uniformly in $n$), and is consistent if $\Phi(x,y;0)=f(x,y)$. (Some books divide $T_{n+1}$ by $h$ and call the result the local truncation error; it is then $O(h^p)$. Either way, a method of order $p$ has global error $O(h^p)$ — Theorem 5.48.)
On names. In CSIR NET and most Indian texts, "modified Euler" means Heun's predictor–corrector above; sometimes the corrector is iterated, $y^{(k+1)}_{n+1}=y_n+\tfrac h2\big[f_n+f(x_{n+1},y^{(k)}_{n+1})\big]$, until two successive values agree (this converges, when $hL\lt2$, to the implicit trapezoidal rule). A few books call the midpoint method "modified Euler" or the "improved polygon method". In an exam, use the formula that is printed; if only the name is given, use Heun.
For an equation $y'=f(x)$ without $y$ these methods are quadrature rules on $[x_n,x_{n+1}]$: Euler is the left-endpoint rectangle rule, Heun the trapezoidal rule and the midpoint method the midpoint rule of §5.4 (Numerical Differentiation and Integration). This already suggests orders $1$, $2$, $2$, and also that more slopes, cleverly combined, will give higher order.
- If $y\in C^2$, Euler's method has $T_{n+1}=\tfrac{h^2}{2}y''(\xi_n)$ for some $\xi_n\in(x_n,x_{n+1})$; it has order $1$.
- Consider the two-stage family $$k_1=f(x_n,y_n),\quad k_2=f(x_n+ch,\ y_n+chk_1),$$ $$y_{n+1}=y_n+h(b_1k_1+b_2k_2),$$ with $f$ smooth. It has order $\ge1$ iff $b_1+b_2=1$, and order $\ge2$ iff in addition $b_2c=\tfrac12$. No choice of $b_1,b_2,c$ gives order $3$.
- Heun ($b_1=b_2=\tfrac12$, $c=1$), the midpoint method ($b_1=0$, $b_2=1$, $c=\tfrac12$) and Ralston's method ($b_1=\tfrac14$, $b_2=\tfrac34$, $c=\tfrac23$) are therefore of order exactly $2$.
Proof. (1) Taylor: $y(x_{n+1})=y(x_n)+hy'(x_n)+\tfrac{h^2}{2}y''(\xi_n)$ and $y'(x_n)=f(x_n,y(x_n))$.
(2) Write $f,f_x,\dots$ for values at $(x_n,y(x_n))$ and $y=y(x_n)$. The exact solution satisfies (Definition 5.27)
$$y(x_{n+1})-y=hf+\tfrac{h^2}{2}(f_x+ff_y)+\tfrac{h^3}{6}\big(f_{xx}+2ff_{xy}+f^2f_{yy}+f_y(f_x+ff_y)\big)+O(h^4).$$For the method (with $y_n=y$), expand $k_2$ by Taylor's theorem in two variables:
$$k_2=f+ch(f_x+ff_y)+\tfrac{c^2h^2}{2}\big(f_{xx}+2ff_{xy}+f^2f_{yy}\big)+O(h^3),$$so
$$h(b_1k_1+b_2k_2)=(b_1+b_2)hf+b_2c\,h^2(f_x+ff_y)+\tfrac{b_2c^2}{2}h^3\big(f_{xx}+2ff_{xy}+f^2f_{yy}\big)+O(h^4).$$Comparing coefficients: the $h$ terms agree for all $f$ iff $b_1+b_2=1$, and the $h^2$ terms iff $b_2c=\tfrac12$. For order $3$ the $h^3$ terms would have to agree for every $f$; but the term $f_y(f_x+ff_y)$ never appears in the method. Concretely, for $f(x,y)=y$ the exact $h^3$ coefficient is $\tfrac{y}{6}$ while the method's is $0$ (all second partial derivatives of $f$ vanish). So $T_{n+1}=\tfrac{h^3}{6}y+O(h^4)\ne O(h^4)$. (3) is a substitution. ∎
An explicit $s$-stage Runge–Kutta method is given by coefficients $a_{ij}$ ($j\lt i$), $b_i$ and $c_i=\sum_ja_{ij}$:
$$k_i=f\Big(x_n+c_ih,\ y_n+h\sum_{j\lt i}a_{ij}k_j\Big)\ (i=1,\dots,s),$$ $$y_{n+1}=y_n+h\sum_{i=1}^sb_ik_i.$$The classical Runge–Kutta method (RK4) has $s=4$:
$$k_1=f(x_n,y_n),\qquad k_2=f\big(x_n+\tfrac h2,\ y_n+\tfrac h2k_1\big),$$ $$k_3=f\big(x_n+\tfrac h2,\ y_n+\tfrac h2k_2\big),\qquad k_4=f(x_n+h,\ y_n+hk_3),$$ $$y_{n+1}=y_n+\frac h6\big(k_1+2k_2+2k_3+k_4\big).$$Euler ($s=1$), Heun, midpoint and Ralston ($s=2$) are Runge–Kutta methods too. The coefficients are often displayed as a Butcher tableau with the $c_i$ in a column, the $a_{ij}$ in a strictly lower triangle and the $b_i$ in the bottom row.
If $f$ is four times continuously differentiable, RK4 has local truncation error $O(h^5)$ and order $4$. Moreover:
- for $y'=f(x)$, RK4 is Simpson's rule on $[x_n,x_{n+1}]$ (with half-step $h/2$), and $T_{n+1}=-\tfrac{h^5}{2880}f^{(4)}(\xi)$;
- for $y'=\lambda y$, RK4 gives $y_{n+1}=R(h\lambda)y_n$ with $R(z)=1+z+\tfrac{z^2}{2}+\tfrac{z^3}{6}+\tfrac{z^4}{24}$, the Taylor polynomial of $e^z$ of degree $4$.
Proof (sketch). (1) If $f$ does not depend on $y$, then $k_2=k_3=f(x_n+\tfrac h2)$, and $$y_{n+1}-y_n=\tfrac h6\big[f(x_n)+4f(x_n+\tfrac h2)+f(x_{n+1})\big],$$ Simpson's rule with step $\tfrac h2$, whose error is $-\tfrac{(h/2)^5}{90}f^{(4)}(\xi)$ (the Simpson error term of §5.4, Numerical Differentiation and Integration). (2) With $z=h\lambda$: $hk_1=zy_n$, $hk_2=z(1+\tfrac z2)y_n$, $hk_3=z(1+\tfrac z2+\tfrac{z^2}4)y_n$, $hk_4=z(1+z+\tfrac{z^2}2+\tfrac{z^3}4)y_n$; inserting these into $\tfrac16(hk_1+2hk_2+2hk_3+hk_4)$ gives $\big(z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}\big)y_n$. (3) In general, expanding as in Theorem 5.46 to order $h^4$ produces eight independent "elementary differentials", and order $4$ is equivalent to the eight order conditions (with $c=(c_i)$, $A=(a_{ij})$, $b=(b_i)$, powers of $c$ taken entrywise)
| order | condition | RK4 value |
|---|---|---|
| 1 | $\sum b_i=1$ | $1$ |
| 2 | $b^{\mathsf T}c=\tfrac12$ | $\tfrac12$ |
| 3 | $b^{\mathsf T}c^2=\tfrac13,\ \ b^{\mathsf T}Ac=\tfrac16$ | $\tfrac13,\ \tfrac16$ |
| 4 | $b^{\mathsf T}c^3=\tfrac14,\ \ b^{\mathsf T}(c\cdot Ac)=\tfrac18,\ \ b^{\mathsf T}Ac^2=\tfrac1{12},\ \ b^{\mathsf T}A^2c=\tfrac1{24}$ | all satisfied |
With $b=\tfrac16(1,2,2,1)$, $c=(0,\tfrac12,\tfrac12,1)$, $a_{21}=a_{32}=\tfrac12$, $a_{43}=1$, each is a one-line check (e.g. $b^{\mathsf T}A^2c=b_4a_{43}a_{32}c_2=\tfrac16\cdot1\cdot\tfrac12\cdot\tfrac12=\tfrac1{24}$). The derivation of the eight conditions — Butcher's theory of rooted trees — is beyond our scope. ∎
| method | $y_1$ | error at $0.1$ | $y_2$ | error at $0.2$ |
|---|---|---|---|---|
| Euler | 1.1 | $4.8\times10^{-3}$ | 1.209 | $9.6\times10^{-3}$ |
| Heun | 1.1045 | $3.3\times10^{-4}$ | 1.2179225 | $6.7\times10^{-4}$ |
| Midpoint | 1.10475 | $7.9\times10^{-5}$ | 1.21844875 | $1.5\times10^{-4}$ |
| RK4 | 1.10482896 | $1.2\times10^{-7}$ | 1.21859699 | $2.5\times10^{-7}$ |
| exact | 1.10482908 | 1.21859724 |
| $h$ | Euler | Heun | RK4 |
|---|---|---|---|
| 0.1 | $3.48\times10^{-2}$ | $3.96\times10^{-3}$ | $1.32\times10^{-6}$ |
| 0.05 | $1.77\times10^{-2}$ | $1.00\times10^{-3}$ | $8.24\times10^{-8}$ |
| 0.025 | $8.91\times10^{-3}$ | $2.52\times10^{-4}$ | $5.14\times10^{-9}$ |
| ratio | $1.97,\ 1.98$ | $3.95,\ 3.98$ | $16.0,\ 16.0$ |
Local and Global Error: Convergence
The local truncation error measures one step taken from the exact solution. The global error $e_n=y(x_n)-y_n$ accumulates these local errors and the propagation of earlier errors through the equation. The next theorem — a discrete Gronwall argument, parallel to the continuous dependence theorem of §1.1 — shows that the propagation costs at most a constant factor and one power of $h$.
Suppose that for $0\lt h\le h_1$ the increment function is Lipschitz in $y$, $|\Phi(x,u;h)-\Phi(x,v;h)|\le\Lambda|u-v|$, and that $|T_{n+1}|\le Ch^{p+1}$ for all $n$ with $x_{n+1}\le X$. Then for $x_n\le X$
$$|e_n|\le e^{\Lambda(x_n-x_0)}|e_0|+\frac{Ch^p}{\Lambda}\Big(e^{\Lambda(x_n-x_0)}-1\Big).$$In particular, with exact starting value $e_0=0$, a method of order $p$ has global error $O(h^p)$ on $[x_0,X]$. For Euler's method, $\Phi=f$, $\Lambda=L$, $C=\tfrac12M_2$ with $M_2=\max|y''|$: $$|e_n|\le\frac{hM_2}{2L}\Big(e^{L(x_n-x_0)}-1\Big).$$
Proof. Subtract the method, $y_{n+1}=y_n+h\Phi(x_n,y_n;h)$, from the definition of $T_{n+1}$:
$$e_{n+1}=e_n+h\big[\Phi(x_n,y(x_n);h)-\Phi(x_n,y_n;h)\big]+T_{n+1}.$$Hence $|e_{n+1}|\le(1+h\Lambda)|e_n|+Ch^{p+1}$. By induction,
$$|e_n|\le(1+h\Lambda)^n|e_0|+Ch^{p+1}\sum_{k=0}^{n-1}(1+h\Lambda)^k=(1+h\Lambda)^n|e_0|+Ch^{p+1}\,\frac{(1+h\Lambda)^n-1}{h\Lambda}.$$Finally $1+h\Lambda\le e^{h\Lambda}$ gives $(1+h\Lambda)^n\le e^{nh\Lambda}=e^{\Lambda(x_n-x_0)}$. For Euler, $T_{n+1}=\tfrac{h^2}{2}y''(\xi_n)$ by Theorem 5.46. ∎
For RK methods $\Phi$ inherits a Lipschitz constant from $f$: for Heun (Exercise III.5) $$|\Phi(u)-\Phi(v)|\le L\big(1+\tfrac{hL}{2}\big)|u-v|,$$ and similarly for RK4. So Heun and RK4 converge with orders $2$ and $4$. The bound is usually pessimistic in its constant (it allows every local error to be amplified by the worst possible factor), but it has the right power of $h$, and it shows that consistency plus a Lipschitz increment function implies convergence.
- Compute $y$ at the final point $X$ with steps $h$ and $h/2$; call the results $Y_h$, $Y_{h/2}$.
- For a method of order $p$, $y(X)-Y_{h/2}\approx\dfrac{Y_{h/2}-Y_h}{2^p-1}$ — Richardson's error estimate (Richardson extrapolation, §5.4), justified because the global error has an expansion $y(X)-Y_h=c_ph^p+O(h^{p+1})$.
- If the estimate is too large, reduce $h$ by the factor $(\text{tolerance}/\text{estimate})^{1/p}$. Production codes do this step by step ("adaptive step size"), comparing two embedded RK formulas of orders $4$ and $5$.
- Also check the observed ratio $\dfrac{Y_h-Y_{2h}}{Y_{h/2}-Y_h}\approx2^p$: if it is far from $2^p$, the step is not yet in the asymptotic range (or the solution is not smooth), and the error estimate is not trustworthy.
Absolute Stability and Stiffness
Theorem 5.48 is about $h\to0$. In practice $h$ is fixed, and a second question arises: does the computed solution behave qualitatively like the true one? The standard test is the linear equation $y'=\lambda y$, $\lambda\in\mathbb{C}$, $\operatorname{Re}\lambda\lt0$, whose solutions $e^{\lambda x}$ decay. Near any solution of a smooth system, the linearisation $u'=Ju$ ($J$ the Jacobian) behaves like a combination of such equations with $\lambda$ the eigenvalues of $J$, so the test is far more relevant than it looks.
If a method applied to $y'=\lambda y$ with step $h$ gives $y_{n+1}=R(z)\,y_n$ with $z=h\lambda$, then $R$ is its stability function. The region of absolute stability is
$$\mathcal{S}=\{z\in\mathbb{C}:|R(z)|\lt1\},$$and the method is absolutely stable for the step $h$ if $h\lambda\in\mathcal{S}$; then $y_n\to0$, as the exact solution does. The set $\mathcal{S}\cap\mathbb{R}$ is the interval of absolute stability. The method is A-stable if $\mathcal{S}$ contains the whole left half-plane $\{\operatorname{Re}z\lt0\}$.
- Euler: $R(z)=1+z$; every two-stage method of order $2$ (Heun, midpoint, Ralston): $R(z)=1+z+\tfrac{z^2}2$; RK4: $R(z)=1+z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}$.
- For an explicit $s$-stage RK method, $R$ is a polynomial of degree $\le s$; if the method has order $p$, then $R(z)=\sum_{k=0}^p\tfrac{z^k}{k!}+O(z^{p+1})$.
- No explicit Runge–Kutta method is A-stable; indeed $\mathcal{S}$ is bounded.
- The intervals of absolute stability are $(-2,0)$ for Euler and for the order-2 two-stage methods, and $(-\alpha,0)$ with $\alpha=2.7853\ldots$ for RK4.
- The implicit backward Euler method $y_{n+1}=y_n+hf(x_{n+1},y_{n+1})$ has $R(z)=\tfrac1{1-z}$, and the implicit trapezoidal rule $y_{n+1}=y_n+\tfrac h2\big[f(x_n,y_n)+f(x_{n+1},y_{n+1})\big]$ has $R(z)=\tfrac{1+z/2}{1-z/2}$; both are A-stable.
Proof. (2) Apply the method to $f=\lambda y$. By induction on $i$, $hk_i=z\big(y_n+\sum_{j\lt i}a_{ij}hk_j\big)$ equals $z\,q_i(z)\,y_n$ with $q_i$ a polynomial of degree $\le i-1$; so $y_{n+1}=\big(1+z\sum b_iq_i(z)\big)y_n=R(z)y_n$ with $\deg R\le s$. The exact solution satisfies $y(x_{n+1})=e^zy(x_n)$, so $T_{n+1}=(e^z-R(z))y(x_n)$. Order $p$ means this is $O(h^{p+1})$ for each fixed $\lambda$, i.e. $e^z-R(z)=O(z^{p+1})$: the coefficients of $R$ up to $z^p$ are those of $e^z$. (1) follows from (2) when $s=p$ (Euler: $s=p=1$; the two-stage methods: $s=p=2$, also computed in Example 4; RK4: $s=p=4$, Theorem 5.47).
(3) $R$ is a non-constant polynomial, so $|R(z)|\to\infty$ as $|z|\to\infty$; $\mathcal{S}$ is bounded and cannot contain the left half-plane.
(4) Euler: $|1+z|\lt1$ for real $z$ iff $-2\lt z\lt0$. Two-stage: $1+z+\tfrac{z^2}2=\tfrac12\big((1+z)^2+1\big)\ge\tfrac12$ for real $z$, and it is $\lt1$ iff $(1+z)^2\lt1$ iff $-2\lt z\lt0$. RK4: first, $R(z)\gt0$ for all real $z$: at a critical point $R'(z)=1+z+\tfrac{z^2}2+\tfrac{z^3}6=0$ we have $R(z)=R'(z)+\tfrac{z^4}{24}=\tfrac{z^4}{24}\gt0$ (and $z\ne0$ there because $R'(0)=1$), so the minimum value of $R$ is positive. So $|R(z)|\lt1$ iff $R(z)-1=z\,q(z)\lt0$, where $q(z)=1+\tfrac z2+\tfrac{z^2}6+\tfrac{z^3}{24}$. Since $q'(z)=\tfrac12+\tfrac z3+\tfrac{z^2}8$ has negative discriminant, $q$ is increasing and has one real zero $-\alpha$, $\alpha=2.7853$. For $z\lt0$ we need $q(z)\gt0$, i.e. $z\gt-\alpha$.
(5) Backward Euler: $y_{n+1}=y_n+zy_{n+1}$, so $R=\tfrac1{1-z}$, and $|1-z|\gt1$ whenever $\operatorname{Re}z\lt0$. Trapezoidal: $(1-\tfrac z2)y_{n+1}=(1+\tfrac z2)y_n$; for $\operatorname{Re}z\lt0$ the point $\tfrac z2$ is closer to $-1$ than to $+1$, so $|1+\tfrac z2|\lt|1-\tfrac z2|$. ∎
The practical meaning of (4): for $y'=-50y$ Euler needs $h\lt0.04$ and RK4 $h\lt0.0557$, however smooth the solution is. An equation (or system) is called stiff when it contains rapidly decaying components (eigenvalues with large negative real part) alongside the slow behaviour we actually want to follow: explicit methods are then forced to use tiny steps for stability long after the fast components have died out, while A-stable implicit methods can take steps suited to the slow solution.
Systems and Higher-Order Equations
Every method above applies word for word to a system $\mathbf{u}'=\mathbf{F}(x,\mathbf{u})$, $\mathbf{u}(x_0)=\mathbf{u}_0\in\mathbb{R}^m$: the unknowns $y_n$, the slopes $k_i$ and the increments become vectors, and the theory (Theorems 5.46–5.48, with $|\cdot|$ replaced by a norm and $L$ by a Lipschitz constant of $\mathbf{F}$) is unchanged. A higher-order equation is first reduced to a first-order system (§1.3, Systems of First-Order Equations): for $y''=g(x,y,y')$ put $u_1=y$, $u_2=y'$,
$$u_1'=u_2,\qquad u_2'=g(x,u_1,u_2).$$For a linear system $\mathbf{u}'=A\mathbf{u}$ with $A$ diagonalisable, a Runge–Kutta method gives $\mathbf{u}_{n+1}=R(hA)\mathbf{u}_n$, and in the eigenbasis of $A$ this decouples into scalar test equations: the method is absolutely stable iff $h\lambda\in\mathcal{S}$ for every eigenvalue $\lambda$ of $A$. For oscillations ($\lambda=\pm i\omega$) the relevant part of $\mathcal{S}$ is the imaginary axis, which Euler's region does not meet at all.
- Local versus global order. Euler's local truncation error is $O(h^2)$ but its global error is $O(h)$; RK4: $O(h^5)$ locally, $O(h^4)$ globally. "Euler's method is of order 2" is false (under the convention $T_{n+1}=O(h^{p+1})$).
- "Modified Euler" is not one formula. Usually Heun's predictor–corrector, sometimes with the corrector iterated, occasionally the midpoint method. All are of order $2$, but their numerical values differ (Example 3: $1.1045$ versus $1.10475$ after one step).
- Convergence is not stability. Euler converges as $h\to0$ for $y'=-20y$, yet with $h=0.15$ it produces $(-2)^n$. Convergence is a statement about $h\to0$; absolute stability about a fixed $h$.
- Higher order does not cure instability. RK4's stability interval $(-2.785,0)$ is barely longer than Euler's $(-2,0)$. No explicit RK method is A-stable (Theorem 5.49(3)); stiff problems need implicit methods.
- Euler cannot represent oscillations. On $y''+y=0$ the energy grows by the factor $1+h^2$ every step for every $h\gt0$; Heun also grows slightly ($|R(ih)|^2=1+\tfrac{h^4}4$); RK4 decays slightly. Only the (implicit) trapezoidal rule preserves it exactly.
- Two stages cannot give order 3, and $s$ stages never give order above $s$ (order $5$ needs $6$ stages). A question claiming a "third-order two-stage Runge–Kutta method" is wrong.
- Picard iterates exist everywhere but converge only where the solution does. For $y'=y^2$, $y(0)=1$, every $\varphi_n$ is a polynomial defined on all of $\mathbb{R}$, but $y=\tfrac1{1-x}$ blows up at $x=1$; for $x\ge1$ the iterates approximate nothing.
- Orders assume smoothness. For $y'=\sqrt{|y|}$ or $y'=|x-\tfrac12|$ the expansions behind Theorems 5.46–5.47 fail at the non-smooth point, and RK4 drops to low order on steps that straddle it.
- Same cost, different accuracy. Compare methods per function evaluation, not per step: one RK4 step costs as much as four Euler steps (Example 3 shows RK4 still wins by orders of magnitude).
- The global error bound is not an estimate. Theorem 5.48 often overestimates by large factors (it grows like $e^{\Lambda(X-x_0)}$ even when the solution decays). Use step halving to estimate errors.
Exercises
- 1 For $y'=x^2+y$, $y(0)=1$, compute $\varphi_1,\varphi_2,\varphi_3$. Evaluate them at $x=0.2$ and compare with the exact solution $y=3e^x-x^2-2x-2$.
- 2 For $y'=y^2$, $y(0)=1$, compute $\varphi_1,\varphi_2,\varphi_3$ and show that $\varphi_n$ agrees with $\tfrac1{1-x}=\sum x^k$ up to the term $x^n$. What happens for $x\ge1$?
- 3 Apply Picard's method to the system $y'=z$, $z'=-y$, $y(0)=0$, $z(0)=1$ and identify the limits.
- 4 Derive the formulas for $y''$ and $y'''$ in Definition 5.27.
- 5 Use the Taylor method of order $2$ with $h=0.1$ to compute $y(0.1)$ and $y(0.2)$ for $y'=x+y$, $y(0)=1$; compare with $y=2e^x-x-1$.
- 6 Use the Taylor method of order $4$ to take one step of size $0.2$ for $y'=-2xy$, $y(0)=1$.
- 7 Show that the Taylor method of order $p$ has order $p$ in the sense of Definition 5.28 and deduce from Theorem 5.48 that its global error is $O(h^p)$.
- 1 Use Euler's method with $h=0.1$ for $y'=x+y$, $y(0)=1$ up to $x=0.3$ and tabulate the errors.
- 2 Use modified Euler (Heun) with $h=0.1$ for the same IVP up to $x=0.2$.
- 3 For $y'=xy$, $y(0)=1$, take one step of $h=0.2$ with Heun and with the midpoint method; compare with $e^{0.02}$.
- 4 Take one RK4 step of size $0.2$ for $y'=x+y$, $y(0)=1$, listing $k_1,\dots,k_4$.
- 5 Take one RK4 step of size $0.5$ for $y'=-2xy$, $y(0)=1$, and compare with $e^{-0.25}$.
- 6 Show that for $y'=f(x)$ Euler, Heun, the midpoint method and RK4 reduce to the left rectangle, trapezoidal, midpoint and Simpson rules.
- 7 Verify that RK4 applied to $y'=\lambda y$ gives $y_{n+1}=\big(1+z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}\big)y_n$, $z=h\lambda$.
- 8 Take one step of Ralston's method ($b=(\tfrac14,\tfrac34)$, $c=\tfrac23$) with $h=0.1$ for $y'=y-x^2$, $y(0)=1$, and compare with Example 3.
- 9 A cup of tea cools according to Newton's law $T'=-0.1(T-20)$ ($T$ in °C, time in minutes), $T(0)=90$. Use modified Euler with $h=1$ min to find $T(1)$ and $T(2)$, and compare with the exact solution.
- 1 Compute the local truncation error of Euler's method for $y'=y$ exactly, and of Heun's method, as power series in $h$.
- 2 Show that Euler's method for $y'=\lambda y$, $y(0)=1$ ($\lambda\in\mathbb{R}$) has global error $e^{\lambda x}-y_n=\tfrac{\lambda^2x}{2}e^{\lambda x}h+O(h^2)$ at a fixed $x=nh$.
- 3 Use Theorem 5.48 to bound the global error of Euler's method at $x=1$ for $y'=x-y$, $y(0)=1$, $h=0.1$, and compare with the actual error (exact $y=x-1+2e^{-x}$).
- 4 Heun's method for $y'=y-x^2$, $y(0)=1$ gives $2.277757$ at $x=1$ with $h=0.1$ and $2.280715$ with $h=0.05$. Estimate the error of the second value and give an improved value. Compare with $y(1)=5-e$.
- 5 Prove that the increment function of Heun's method is Lipschitz with constant $L(1+\tfrac{hL}2)$.
- 6 Prove: if $\Phi$ is continuous, Lipschitz in $y$, and consistent ($\Phi(x,y;0)=f(x,y)$), then $\max_n|e_n|\to0$ as $h\to0$ (with $e_0=0$), even without assuming any order.
- 7 A method gives errors $2.1\times10^{-3}$, $2.7\times10^{-4}$, $3.4\times10^{-5}$ at a fixed point with $h$, $h/2$, $h/4$. What is its order?
- 8 For $y'=y-x^2$, $y(0)=1$ the errors at $x=1$ behave like $0.36h$ (Euler) and $1.3\times10^{-6}(10h)^4$ (RK4). How many evaluations of $f$ does each method need for an error of $10^{-6}$?
- 1 For $y'=-50y$, find the largest $h$ for which Euler, Heun and RK4 are absolutely stable.
- 2 Apply backward Euler to $y'=-50y$, $y(0)=1$ with $h=0.1$. Find $y_n$ and show that backward Euler is A-stable.
- 3 Show that the implicit trapezoidal rule has $R(z)=\tfrac{1+z/2}{1-z/2}$, that $|R(iy)|=1$ for real $y$, and that it is A-stable.
- 4 Write $y''+3y'+2y=0$, $y(0)=1$, $y'(0)=0$ as a system; take one Euler step and one Heun step with $h=0.1$ and compare with the exact solution $y=2e^{-x}-e^{-2x}$. For which $h$ is Euler absolutely stable on this system?
- 5 For Euler's method on $y''+y=0$ with $h=0.01$, after how many steps has the energy $y^2+y'^2$ doubled?
- 6 Show that for RK4, $|R(iy)|^2=1-\tfrac{y^6}{72}+\tfrac{y^8}{576}$, and deduce that RK4 is stable on $y'=i\omega y$ iff $h|\omega|\le2\sqrt2$.
- 7 The IVP $y'=-1000(y-\cos x)-\sin x$, $y(0)=1$ has solution $y=\cos x$. What step does Euler's method need to be stable? Why is this problem called stiff?
- 8 Prove that an explicit $s$-stage Runge–Kutta method has order at most $s$.
E1. CSIR Part B GATE MA For $y'=x+y$, $y(0)=1$, one step of the modified Euler (Heun) method with $h=0.1$ gives $y(0.1)\approx$ (A) $1.1000$; (B) $1.1050$; (C) $1.1100$; (D) $1.1103$.
E2. CSIR Part B Euler's method with constant step $h\gt0$ is applied to $y'=-50y$, $y(0)=1$. The computed values satisfy $y_n\to0$ as $n\to\infty$ if and only if (A) $h\lt0.02$; (B) $h\lt0.04$; (C) $h\le0.05$; (D) $h\lt0.1$.
E3. CSIR Part B For $y'=x-y$, $y(0)=1$, the second Picard iterate (with $\varphi_0=1$) is (A) $1-x+\tfrac{x^2}2$; (B) $1-x+x^2-\tfrac{x^3}6$; (C) $1-x+x^2-\tfrac{x^3}3$; (D) $1-x+\tfrac{x^2}{2}-\tfrac{x^3}6$.
E4. CSIR Part C Which of the following statements are true? (One or more options may be correct.) (A) Applied to $y'=f(x)$, one step of the classical RK4 method is Simpson's 1/3 rule on $[x_n,x_{n+1}]$ with nodes $x_n$, $x_n+\tfrac h2$, $x_{n+1}$. (B) Applied to $y'=f(x)$, Heun's method is the trapezoidal rule. (C) For $\lambda\lt0$, RK4 applied to $y'=\lambda y$ gives $y_n\to0$ for every $h\gt0$. (D) Some explicit two-stage Runge–Kutta method has order $3$.
E5. CSIR Part C Euler's method with $h=\tfrac1n$ is applied to $y'=y$, $y(0)=1$, and $Y_n$ denotes the computed approximation to $y(1)=e$. Which of the following are true? (A) $Y_n=(1+\tfrac1n)^n$. (B) $Y_n\lt e$ for every $n$. (C) $n(e-Y_n)\to\tfrac e2$. (D) $e-Y_{2n}=\tfrac12(e-Y_n)$ for every $n$.
E6. CSIR Part C The system $u'=v$, $v'=-u$ is solved with a fixed step $h\gt0$, and $E_n=u_n^2+v_n^2$. Which of the following are true? (A) For Euler's method $E_{n+1}=(1+h^2)E_n$. (B) For Euler's method the computed solution remains bounded as $n\to\infty$. (C) For the backward Euler method $E_{n+1}=E_n/(1+h^2)$. (D) For the implicit trapezoidal rule $E_{n+1}=E_n$.
Suggested plan — 3 lectures × 60 min.
- From existence to computation. Recall Picard–Lindelöf from §1.1; Theorem 5.45 with its short proof; Example 1 (do $\varphi_1,\varphi_2$ on the board) and Example 2. Definitions 5.28–5.29; draw Euler's tangent polygon and Heun's averaged slope. Theorem 5.46(1) and the naming warning about "modified Euler".
- Runge–Kutta and order. Theorem 5.46(2) — the Taylor comparison is worth doing in full once; Definition 5.30 and Theorem 5.47 (prove (1) and (2), state the order conditions). Example 3 on the board for the first step only; project Interactive 1 and let students read off the ratios $2,4,16$. Theorem 5.48 with the discrete Gronwall proof; Example 5.
- Stability and systems. The test equation, Definition 5.31 and Theorem 5.49 (prove (4) for Euler and RK2, show the RK4 argument); Example 6; Interactive 2 with $\lambda=-20$ (slide $h$ through $0.1$) and $\lambda=6i$ (Euler versus RK4). Systems and Example 7; finish with Exam-corner E2 and E6.
Discussion questions. (1) Why does a local error of $O(h^{p+1})$ become a global error of $O(h^p)$ and not, say, $O(h^{p+1})$ or $O(h^{p-1})$? (2) If the exact solution of $y'=-1000(y-\cos x)-\sin x$ is just $\cos x$, why does the coefficient $1000$ matter at all? (3) Picard's method and the Taylor method give the same polynomials (Theorem 5.45). Which would you program, and why do neither appear in modern ODE software?
Misconceptions to watch for. Confusing local and global order; using Euler's slope at the new point in Heun's predictor; forgetting to update $x$ in $k_2,k_3,k_4$; thinking a higher-order method is always more accurate or always more stable; believing that convergence as $h\to0$ guarantees sensible results at a given $h$; computing stability intervals from $|R(z)|\le1$ versus $\lt1$ inconsistently (the endpoints give non-decaying, bounded solutions).
Suggested homework. I(1, 2, 5, 6), II(1, 2, 4, 6, 9), III(2, 3, 4, 7), IV(1, 2, 4, 6, 8); Exam corner E1, E3, E4, E6.
Research question. How large a time step can an epidemic model afford? In the SIR model $S'=-\beta SI$, $I'=\beta SI-\gamma I$, $R'=\gamma I$ (fractions of a population, time in days), how do the predicted number of infected people and the timing of the peak depend on the method and on $h$ — and when does a cheap method give a misleading forecast?
Why it matters. Compartmental models of this kind were run daily by public-health agencies around the world during the COVID-19 pandemic, and similar systems model the spread of influenza, measles and dengue. A step-size artefact that shifts the predicted peak by a few days, or overstates it by 40%, can change hospital planning.
What to do.
- Take $\beta=0.3$, $\gamma=0.1$ per day, $S(0)=0.99$, $I(0)=0.01$, $R(0)=0$. Compute a reference solution on $[0,160]$ with a library solver (tight tolerances) and record $I(40)$ and the peak of $I$.
- Solve with Euler and with RK4 for $h=8,4,2,1,0.5$ days. Tabulate the error in $I(40)$ and compute the observed orders from successive ratios.
- Find the largest $h$ for which each method predicts $I(40)$ to three significant figures; count function evaluations.
- Check that $S+I+R$ stays equal to $1$ for both methods (why is this exact?), and that the computed $S$ is decreasing. Can you make Euler produce a negative $I$ with a large step?
Tools: Python with NumPy and scipy.integrate.solve_ivp, or a spreadsheet for Euler.
Hand in: A one-page note with the error table, the observed orders, your recommended step for each method, and one plot of $I(t)$ for Euler with $h=4$ against the reference.
Going further. Linearise the model near the disease-free state $(S,I)=(1,0)$ to find the eigenvalue $\beta-\gamma$; then increase $\gamma$ (fast recovery) and $\beta$ together and explore when the explicit methods become unstable. Read about how adaptive solvers (embedded RK pairs such as Dormand–Prince) choose their steps.
🧪 Self-check quiz
Q1. The global error of the classical fourth-order Runge–Kutta method on a fixed interval is
Q2. The local truncation error $y(x_{n+1})-y(x_n)-hf(x_n,y(x_n))$ of Euler's method is
Q3. Euler's method applied to $y'=-10y$ produces decaying values exactly when
Q4. If $f$ is smooth, the $n$-th Picard iterate $\varphi_n$ of $y'=f(x,y)$, $y(x_0)=y_0$ agrees with the Taylor series of the solution at least up to the term
Q5. The modified Euler (Heun) method is