Chapter 5 · Numerical Analysis

5.5 Numerical Solution of ODEs

Picard's successive approximations and the Taylor series method; Euler, modified Euler (Heun) and Runge–Kutta methods of orders 2 and 4; local and global truncation error and order; absolute stability for $y'=\lambda y$; systems and higher-order equations.

Weather forecasts, satellite orbits, the spread of an epidemic, the current in a circuit, the concentration of a drug in the blood: each is the solution of an initial value problem $y'=f(x,y)$, $y(x_0)=y_0$, and in almost every case the solution has no formula. The Picard–Lindelöf theorem (§1.1, Existence and Uniqueness of Solutions) tells us that a unique solution exists when $f$ is continuous and Lipschitz in $y$; this section is about computing it. The answer is a table of approximations $y_n\approx y(x_n)$ at grid points $x_n=x_0+nh$, produced step by step from a rule that uses only values of $f$.

We start with two "analytic" methods that produce formulas — Picard's successive approximations and the Taylor series method — and then turn to the workhorses: Euler's method, its modification by Heun, and the Runge–Kutta methods, culminating in the classical fourth-order method RK4 that is still the default in many codes. Two ideas organise everything: order (how fast the error falls as $h\to0$; local error $O(h^{p+1})$ per step, global error $O(h^p)$) and stability (whether, for a fixed $h$, errors are damped or amplified). CSIR NET asks for one or two steps of Euler, modified Euler or RK4, the Picard iterates of a simple equation, the order of a method, and the step size beyond which a method blows up on $y'=\lambda y$. Throughout, $f$ is continuous on $[x_0,X]\times\mathbb{R}$ and Lipschitz in $y$ with constant $L$, so the solution $y$ exists on $[x_0,X]$ (§1.1); $y_n$ is the computed value, $f_n=f(x_n,y_n)$, and $e_n=y(x_n)-y_n$ is the global error.

Picard's Method and the Taylor Series Method

Picard's method is the proof of the existence theorem used as an algorithm: starting from $\varphi_0(x)=y_0$, compute

$$\varphi_{n+1}(x)=y_0+\int_{x_0}^xf\big(t,\varphi_n(t)\big)\,dt\qquad(n\ge0).$$

In §1.1 we proved that on the Picard interval $|x-x_0|\le h_0=\min(a,b/M)$ the iterates converge uniformly to the solution, with a factorially small error. As a computational method its key property is that each iteration fixes one more Taylor coefficient.

📐 Theorem 5.45 — Picard iterates and Taylor polynomials

Let $f$ be continuous on $R=\{|x-x_0|\le a,\ |y-y_0|\le b\}$, $|f|\le M$ there, and Lipschitz in $y$ with constant $L$; let $y$ be the solution on $I=[x_0-h_0,x_0+h_0]$, $h_0=\min(a,b/M)$, and $\varphi_n$ its Picard iterates. Then for every $n\ge0$ and $x\in I$

$$|y(x)-\varphi_n(x)|\le\frac{ML^n}{(n+1)!}\,|x-x_0|^{n+1}.$$

Consequently, if $f\in C^n$ near $(x_0,y_0)$ and $\varphi_n$ is a polynomial, then the terms of degree $\le n$ of $\varphi_n$ (expanded in powers of $x-x_0$) are exactly the Taylor polynomial of degree $n$ of $y$ at $x_0$.

Proof. By Steps 1 and 5 of the proof of the Picard–Lindelöf theorem (§1.1), the graphs of $y$ and of every $\varphi_n$ lie in $R$. For $n=0$: $|y(x)-y_0|=\big|\int_{x_0}^xf(t,y(t))\,dt\big|\le M|x-x_0|$. If the bound holds for $n$, then for $x\ge x_0$ (the other side is symmetric)

$$|y(x)-\varphi_{n+1}(x)|\le\int_{x_0}^x\big|f(t,y)-f(t,\varphi_n)\big|\,dt\le L\int_{x_0}^x\frac{ML^n(t-x_0)^{n+1}}{(n+1)!}\,dt,$$

which equals $\dfrac{ML^{n+1}(x-x_0)^{n+2}}{(n+2)!}$. For the second statement: if $f\in C^n$, then $y'=f(x,y)$ shows inductively that $y\in C^{n+1}$, so $y-T_n=O(|x-x_0|^{n+1})$, where $T_n$ is the Taylor polynomial of $y$. With the bound just proved, $\varphi_n-T_n=O(|x-x_0|^{n+1})$. If $P$ denotes the part of degree $\le n$ of the polynomial $\varphi_n$, then $P-T_n$ is a polynomial of degree $\le n$ that is $O(|x-x_0|^{n+1})$ as $x\to x_0$, and such a polynomial is identically zero (divide by the lowest power present and let $x\to x_0$). ∎

✏️ Example 1 — Picard's method for $y'=x+y^2$
For $y'=x+y^2$, $y(0)=1$ compute $\varphi_1,\varphi_2,\varphi_3$, compare their values at $x=0.1$ with the accurate value $y(0.1)=1.1164924$, and check Theorem 5.45 against the Taylor series $y=1+x+\tfrac32x^2+\tfrac43x^3+\tfrac{17}{12}x^4+\cdots$.
$\varphi_1$ $\varphi_1=1+\int_0^x(t+1)\,dt=1+x+\tfrac{x^2}{2}$.
$\varphi_2$ $\varphi_2=1+\int_0^x\big(t+(1+t+\tfrac{t^2}{2})^2\big)dt$. Expanding the square, $1+2t+2t^2+t^3+\tfrac{t^4}4$, so $$\varphi_2=1+x+\tfrac32x^2+\tfrac23x^3+\tfrac14x^4+\tfrac1{20}x^5.$$
$\varphi_3$ Similarly (a longer but mechanical computation) $$\varphi_3=1+x+\tfrac32x^2+\tfrac43x^3+\tfrac{13}{12}x^4+\tfrac{49}{60}x^5+\cdots,$$ a polynomial of degree $11$.
Values At $x=0.1$: $\varphi_1=1.105$, $\varphi_2=1.115692$, $\varphi_3=1.116450$; the errors are $-1.1\times10^{-2}$, $-8.0\times10^{-4}$, $-4.2\times10^{-5}$.
Theorem 5.45 $\varphi_1$ agrees with the Taylor series up to $x^1$ (its $x^2$ coefficient $\tfrac12$ is wrong); $\varphi_2$ up to $x^2$ (its $x^3$ coefficient $\tfrac23$ is wrong); $\varphi_3$ up to $x^3$ (its $x^4$ coefficient $\tfrac{13}{12}\ne\tfrac{17}{12}$). Each iteration fixes exactly one more coefficient here — and costs a lot more algebra, because squaring doubles the degree.

Example 1 shows both the strength and the weakness of Picard's method. It produces approximations valid on a whole interval, with an error bound, but each iteration requires a symbolic integration whose complexity grows rapidly, and for most $f$ (say $f=\sin(xy)$) the integrals cannot be done in closed form at all. Since Theorem 5.45 says that the useful output of $n$ iterations is the Taylor polynomial of degree $n$, we may as well compute that polynomial directly.

📘 Definition 5.27 — Taylor series method of order $p$

Let $f$ be $p$ times continuously differentiable. Define the total derivatives $f^{[0]}=f$ and $f^{[k+1]}=\partial_xf^{[k]}+f\,\partial_yf^{[k]}$, so that $y^{(k+1)}(x)=f^{[k]}(x,y(x))$ for every solution; for instance

$$y''=f_x+ff_y,$$ $$y'''=f_{xx}+2ff_{xy}+f^2f_{yy}+f_y\big(f_x+ff_y\big).$$

The Taylor series method of order $p$ is $$y_{n+1}=y_n+hf^{[0]}_n+\frac{h^2}{2!}f^{[1]}_n+\cdots+\frac{h^p}{p!}f^{[p-1]}_n,$$ with all $f^{[k]}$ evaluated at $(x_n,y_n)$. For $p=1$ this is Euler's method.

✏️ Example 2 — The Taylor method of order 4
For the IVP of Example 1, use the Taylor method of order $4$ with $h=0.1$ to compute $y(0.1)$ and $y(0.2)$, and compare with a single step of size $h=0.2$. (Accurate values: $y(0.1)=1.1164924$, $y(0.2)=1.2735643$.)
Derivatives Differentiating $y'=x+y^2$ along the solution: $y''=1+2yy'$, $y'''=2y'^2+2yy''$, $y^{(4)}=6y'y''+2yy'''$.
Step 1 At $(0,1)$: $y'=1$, $y''=3$, $y'''=8$, $y^{(4)}=34$. So $$y_1=1+0.1+\tfrac{0.01}{2}\cdot3+\tfrac{0.001}{6}\cdot8+\tfrac{0.0001}{24}\cdot34=1.116475,$$ error $-1.7\times10^{-5}$. (These are the coefficients $1,1,\tfrac32,\tfrac43,\tfrac{17}{12}$ of Example 1.)
Step 2 At $(0.1,1.116475)$: $y'=1.346516$, $y''=4.006704$, $y'''=12.572982$, $y^{(4)}=60.445396$. Then $y_2=1.273508$, error $-5.7\times10^{-5}$.
One big step With $h=0.2$ from $(0,1)$: $1+0.2+0.06+0.010667+0.002267=1.272933$, error $-6.3\times10^{-4}$, about $11$ times larger. For an order-4 method halving $h$ should divide the global error by $16$; the ratio is a bit smaller here because $y$ grows quickly (the solution blows up near $x\approx0.93$).

The Taylor method is accurate and conceptually simple, but each new $f$ needs its own derivative formulas, which become unwieldy for systems. Runge–Kutta methods achieve the same order using only values of $f$.

Euler, Modified Euler and Runge–Kutta Methods

📘 Definition 5.28 — One-step method; local truncation error; order

A one-step method computes $y_{n+1}=y_n+h\,\Phi(x_n,y_n;h)$ with an increment function $\Phi$ built from values of $f$. Its local truncation error at step $n+1$ is what the exact solution fails to satisfy:

$$T_{n+1}=y(x_{n+1})-y(x_n)-h\,\Phi\big(x_n,y(x_n);h\big).$$

The method has order $p$ if $T_{n+1}=O(h^{p+1})$ for every sufficiently smooth $f$ (uniformly in $n$), and is consistent if $\Phi(x,y;0)=f(x,y)$. (Some books divide $T_{n+1}$ by $h$ and call the result the local truncation error; it is then $O(h^p)$. Either way, a method of order $p$ has global error $O(h^p)$ — Theorem 5.48.)

📘 Definition 5.29 — Euler, modified Euler (Heun) and midpoint methods
Euler's method$y_{n+1}=y_n+hf(x_n,y_n)$: follow the tangent line for one step.
Modified Euler / Heunpredictor $y^*_{n+1}=y_n+hf(x_n,y_n)$; corrector $y_{n+1}=y_n+\tfrac h2\big[f(x_n,y_n)+f(x_{n+1},y^*_{n+1})\big]$: average the slopes at both ends.
Midpoint method$y_{n+1}=y_n+hf\big(x_n+\tfrac h2,\,y_n+\tfrac h2f(x_n,y_n)\big)$: use the slope at the midpoint of an Euler half-step.

On names. In CSIR NET and most Indian texts, "modified Euler" means Heun's predictor–corrector above; sometimes the corrector is iterated, $y^{(k+1)}_{n+1}=y_n+\tfrac h2\big[f_n+f(x_{n+1},y^{(k)}_{n+1})\big]$, until two successive values agree (this converges, when $hL\lt2$, to the implicit trapezoidal rule). A few books call the midpoint method "modified Euler" or the "improved polygon method". In an exam, use the formula that is printed; if only the name is given, use Heun.

For an equation $y'=f(x)$ without $y$ these methods are quadrature rules on $[x_n,x_{n+1}]$: Euler is the left-endpoint rectangle rule, Heun the trapezoidal rule and the midpoint method the midpoint rule of §5.4 (Numerical Differentiation and Integration). This already suggests orders $1$, $2$, $2$, and also that more slopes, cleverly combined, will give higher order.

📐 Theorem 5.46 — Local errors of Euler and of two-stage methods
  1. If $y\in C^2$, Euler's method has $T_{n+1}=\tfrac{h^2}{2}y''(\xi_n)$ for some $\xi_n\in(x_n,x_{n+1})$; it has order $1$.
  2. Consider the two-stage family $$k_1=f(x_n,y_n),\quad k_2=f(x_n+ch,\ y_n+chk_1),$$ $$y_{n+1}=y_n+h(b_1k_1+b_2k_2),$$ with $f$ smooth. It has order $\ge1$ iff $b_1+b_2=1$, and order $\ge2$ iff in addition $b_2c=\tfrac12$. No choice of $b_1,b_2,c$ gives order $3$.
  3. Heun ($b_1=b_2=\tfrac12$, $c=1$), the midpoint method ($b_1=0$, $b_2=1$, $c=\tfrac12$) and Ralston's method ($b_1=\tfrac14$, $b_2=\tfrac34$, $c=\tfrac23$) are therefore of order exactly $2$.

Proof. (1) Taylor: $y(x_{n+1})=y(x_n)+hy'(x_n)+\tfrac{h^2}{2}y''(\xi_n)$ and $y'(x_n)=f(x_n,y(x_n))$.

(2) Write $f,f_x,\dots$ for values at $(x_n,y(x_n))$ and $y=y(x_n)$. The exact solution satisfies (Definition 5.27)

$$y(x_{n+1})-y=hf+\tfrac{h^2}{2}(f_x+ff_y)+\tfrac{h^3}{6}\big(f_{xx}+2ff_{xy}+f^2f_{yy}+f_y(f_x+ff_y)\big)+O(h^4).$$

For the method (with $y_n=y$), expand $k_2$ by Taylor's theorem in two variables:

$$k_2=f+ch(f_x+ff_y)+\tfrac{c^2h^2}{2}\big(f_{xx}+2ff_{xy}+f^2f_{yy}\big)+O(h^3),$$

so

$$h(b_1k_1+b_2k_2)=(b_1+b_2)hf+b_2c\,h^2(f_x+ff_y)+\tfrac{b_2c^2}{2}h^3\big(f_{xx}+2ff_{xy}+f^2f_{yy}\big)+O(h^4).$$

Comparing coefficients: the $h$ terms agree for all $f$ iff $b_1+b_2=1$, and the $h^2$ terms iff $b_2c=\tfrac12$. For order $3$ the $h^3$ terms would have to agree for every $f$; but the term $f_y(f_x+ff_y)$ never appears in the method. Concretely, for $f(x,y)=y$ the exact $h^3$ coefficient is $\tfrac{y}{6}$ while the method's is $0$ (all second partial derivatives of $f$ vanish). So $T_{n+1}=\tfrac{h^3}{6}y+O(h^4)\ne O(h^4)$. (3) is a substitution. ∎

📘 Definition 5.30 — Runge–Kutta methods; the classical RK4

An explicit $s$-stage Runge–Kutta method is given by coefficients $a_{ij}$ ($j\lt i$), $b_i$ and $c_i=\sum_ja_{ij}$:

$$k_i=f\Big(x_n+c_ih,\ y_n+h\sum_{j\lt i}a_{ij}k_j\Big)\ (i=1,\dots,s),$$ $$y_{n+1}=y_n+h\sum_{i=1}^sb_ik_i.$$

The classical Runge–Kutta method (RK4) has $s=4$:

$$k_1=f(x_n,y_n),\qquad k_2=f\big(x_n+\tfrac h2,\ y_n+\tfrac h2k_1\big),$$ $$k_3=f\big(x_n+\tfrac h2,\ y_n+\tfrac h2k_2\big),\qquad k_4=f(x_n+h,\ y_n+hk_3),$$ $$y_{n+1}=y_n+\frac h6\big(k_1+2k_2+2k_3+k_4\big).$$

Euler ($s=1$), Heun, midpoint and Ralston ($s=2$) are Runge–Kutta methods too. The coefficients are often displayed as a Butcher tableau with the $c_i$ in a column, the $a_{ij}$ in a strictly lower triangle and the $b_i$ in the bottom row.

📐 Theorem 5.47 — The classical RK4 has order 4

If $f$ is four times continuously differentiable, RK4 has local truncation error $O(h^5)$ and order $4$. Moreover:

  1. for $y'=f(x)$, RK4 is Simpson's rule on $[x_n,x_{n+1}]$ (with half-step $h/2$), and $T_{n+1}=-\tfrac{h^5}{2880}f^{(4)}(\xi)$;
  2. for $y'=\lambda y$, RK4 gives $y_{n+1}=R(h\lambda)y_n$ with $R(z)=1+z+\tfrac{z^2}{2}+\tfrac{z^3}{6}+\tfrac{z^4}{24}$, the Taylor polynomial of $e^z$ of degree $4$.

Proof (sketch). (1) If $f$ does not depend on $y$, then $k_2=k_3=f(x_n+\tfrac h2)$, and $$y_{n+1}-y_n=\tfrac h6\big[f(x_n)+4f(x_n+\tfrac h2)+f(x_{n+1})\big],$$ Simpson's rule with step $\tfrac h2$, whose error is $-\tfrac{(h/2)^5}{90}f^{(4)}(\xi)$ (the Simpson error term of §5.4, Numerical Differentiation and Integration). (2) With $z=h\lambda$: $hk_1=zy_n$, $hk_2=z(1+\tfrac z2)y_n$, $hk_3=z(1+\tfrac z2+\tfrac{z^2}4)y_n$, $hk_4=z(1+z+\tfrac{z^2}2+\tfrac{z^3}4)y_n$; inserting these into $\tfrac16(hk_1+2hk_2+2hk_3+hk_4)$ gives $\big(z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}\big)y_n$. (3) In general, expanding as in Theorem 5.46 to order $h^4$ produces eight independent "elementary differentials", and order $4$ is equivalent to the eight order conditions (with $c=(c_i)$, $A=(a_{ij})$, $b=(b_i)$, powers of $c$ taken entrywise)

orderconditionRK4 value
1$\sum b_i=1$$1$
2$b^{\mathsf T}c=\tfrac12$$\tfrac12$
3$b^{\mathsf T}c^2=\tfrac13,\ \ b^{\mathsf T}Ac=\tfrac16$$\tfrac13,\ \tfrac16$
4$b^{\mathsf T}c^3=\tfrac14,\ \ b^{\mathsf T}(c\cdot Ac)=\tfrac18,\ \ b^{\mathsf T}Ac^2=\tfrac1{12},\ \ b^{\mathsf T}A^2c=\tfrac1{24}$all satisfied

With $b=\tfrac16(1,2,2,1)$, $c=(0,\tfrac12,\tfrac12,1)$, $a_{21}=a_{32}=\tfrac12$, $a_{43}=1$, each is a one-line check (e.g. $b^{\mathsf T}A^2c=b_4a_{43}a_{32}c_2=\tfrac16\cdot1\cdot\tfrac12\cdot\tfrac12=\tfrac1{24}$). The derivation of the eight conditions — Butcher's theory of rooted trees — is beyond our scope. ∎

✏️ Example 3 — Four methods side by side
For $y'=y-x^2$, $y(0)=1$ (exact solution $y=x^2+2x+2-e^x$) take two steps of size $h=0.1$ with Euler, Heun, the midpoint method and RK4. Then compare the errors at $x=1$ for $h=0.1,0.05,0.025$.
Euler $y_1=1+0.1(1-0)=1.1$; $y_2=1.1+0.1(1.1-0.01)=1.209$.
Heun Step 1: $k_1=1$, predictor $1.1$, $k_2=f(0.1,1.1)=1.09$, $y_1=1+0.05(2.09)=1.1045$. Step 2 similarly gives $y_2=1.2179225$.
Midpoint $k_2=f(0.05,1.05)=1.0475$, $y_1=1.10475$; then $y_2=1.21844875$.
RK4 Step 1: $k_1=1$, $k_2=f(0.05,1.05)=1.0475$, $k_3=f(0.05,1.052375)=1.049875$, $k_4=f(0.1,1.1049875)=1.0949875$; $$y_1=1+\tfrac{0.1}{6}(1+2.095+2.09975+1.0949875)=1.10482896.$$ Step 2: $y_2=1.21859699$.
method$y_1$error at $0.1$$y_2$error at $0.2$
Euler1.1$4.8\times10^{-3}$1.209$9.6\times10^{-3}$
Heun1.1045$3.3\times10^{-4}$1.2179225$6.7\times10^{-4}$
Midpoint1.10475$7.9\times10^{-5}$1.21844875$1.5\times10^{-4}$
RK41.10482896$1.2\times10^{-7}$1.21859699$2.5\times10^{-7}$
exact1.104829081.21859724
Orders Errors $y(1)-y_N$ at $x=1$ ($y(1)=5-e=2.281718$):
$h$EulerHeunRK4
0.1$3.48\times10^{-2}$$3.96\times10^{-3}$$1.32\times10^{-6}$
0.05$1.77\times10^{-2}$$1.00\times10^{-3}$$8.24\times10^{-8}$
0.025$8.91\times10^{-3}$$2.52\times10^{-4}$$5.14\times10^{-9}$
ratio$1.97,\ 1.98$$3.95,\ 3.98$$16.0,\ 16.0$
Lesson Halving $h$ divides the global error by $2$, $4$, $16$: orders $1$, $2$, $4$. RK4 costs four evaluations of $f$ per step, Euler one; yet RK4 with $h=0.1$ (40 evaluations) is almost $7000$ times more accurate than Euler with $h=0.025$ (also 40 evaluations).
Interactive 1 — Euler, Heun and RK4: Watching the Order

The blue curve is the exact solution; the polygons are Euler, Heun (modified Euler) and RK4 with $N$ steps. The readouts give each method's error at the right end and the factor by which it shrank when $N$ was doubled — it approaches $2$, $4$ and $16$. Press Play to double $N$ repeatedly. The last problem is mildly stiff: for large $h$ every explicit method explodes (see the stability subsection).

—
step h—
Euler error · ratio—
Heun error · ratio—
RK4 error · ratio—
✏️ Example 4 — Second order, but never third: prove or disprove
(a) Show that for every $c\in(0,1]$ the method $k_1=f(x_n,y_n)$, $k_2=f(x_n+ch,y_n+chk_1)$, $y_{n+1}=y_n+h\big[(1-\tfrac1{2c})k_1+\tfrac1{2c}k_2\big]$ has order $2$, and that applied to $y'=\lambda y$ every member gives the same $y_{n+1}$. (b) Prove or disprove: some member has order $3$ for the equation $y'=y$.
(a) Order Here $b_1+b_2=1$ and $b_2c=\tfrac1{2c}\cdot c=\tfrac12$, so Theorem 5.46 gives order $\ge2$, and no two-stage method has order $3$. ($c=1$: Heun; $c=\tfrac12$: midpoint; $c=\tfrac23$: Ralston.)
(a) Linear test For $f=\lambda y$: $k_1=\lambda y_n$, $k_2=\lambda(1+c h\lambda)y_n$, so $$y_{n+1}=y_n+h\lambda y_n\Big[(1-\tfrac1{2c})+\tfrac1{2c}(1+ch\lambda)\Big]=\Big(1+z+\tfrac{z^2}{2}\Big)y_n,$$ $z=h\lambda$, independent of $c$.
(b) Disprove For $y'=y$ the exact factor is $e^h=1+h+\tfrac{h^2}2+\tfrac{h^3}6+\cdots$ while every member gives $1+h+\tfrac{h^2}2$. So $T_{n+1}=\big(\tfrac{h^3}{6}+O(h^4)\big)y(x_n)$, which is not $O(h^4)$: order exactly $2$ even for this simplest equation.
Remark More generally, an explicit $s$-stage method applied to $y'=\lambda y$ yields a polynomial $R(z)$ of degree $\le s$, which can agree with $e^z$ only up to $z^s$: order $\le s$ (Exercise IV.8). For $s\ge5$ the order is even strictly less than $s$ (Butcher's barriers): order $5$ needs $6$ stages.

Local and Global Error: Convergence

The local truncation error measures one step taken from the exact solution. The global error $e_n=y(x_n)-y_n$ accumulates these local errors and the propagation of earlier errors through the equation. The next theorem — a discrete Gronwall argument, parallel to the continuous dependence theorem of §1.1 — shows that the propagation costs at most a constant factor and one power of $h$.

📐 Theorem 5.48 — Convergence of one-step methods; global error bound

Suppose that for $0\lt h\le h_1$ the increment function is Lipschitz in $y$, $|\Phi(x,u;h)-\Phi(x,v;h)|\le\Lambda|u-v|$, and that $|T_{n+1}|\le Ch^{p+1}$ for all $n$ with $x_{n+1}\le X$. Then for $x_n\le X$

$$|e_n|\le e^{\Lambda(x_n-x_0)}|e_0|+\frac{Ch^p}{\Lambda}\Big(e^{\Lambda(x_n-x_0)}-1\Big).$$

In particular, with exact starting value $e_0=0$, a method of order $p$ has global error $O(h^p)$ on $[x_0,X]$. For Euler's method, $\Phi=f$, $\Lambda=L$, $C=\tfrac12M_2$ with $M_2=\max|y''|$: $$|e_n|\le\frac{hM_2}{2L}\Big(e^{L(x_n-x_0)}-1\Big).$$

Proof. Subtract the method, $y_{n+1}=y_n+h\Phi(x_n,y_n;h)$, from the definition of $T_{n+1}$:

$$e_{n+1}=e_n+h\big[\Phi(x_n,y(x_n);h)-\Phi(x_n,y_n;h)\big]+T_{n+1}.$$

Hence $|e_{n+1}|\le(1+h\Lambda)|e_n|+Ch^{p+1}$. By induction,

$$|e_n|\le(1+h\Lambda)^n|e_0|+Ch^{p+1}\sum_{k=0}^{n-1}(1+h\Lambda)^k=(1+h\Lambda)^n|e_0|+Ch^{p+1}\,\frac{(1+h\Lambda)^n-1}{h\Lambda}.$$

Finally $1+h\Lambda\le e^{h\Lambda}$ gives $(1+h\Lambda)^n\le e^{nh\Lambda}=e^{\Lambda(x_n-x_0)}$. For Euler, $T_{n+1}=\tfrac{h^2}{2}y''(\xi_n)$ by Theorem 5.46. ∎

For RK methods $\Phi$ inherits a Lipschitz constant from $f$: for Heun (Exercise III.5) $$|\Phi(u)-\Phi(v)|\le L\big(1+\tfrac{hL}{2}\big)|u-v|,$$ and similarly for RK4. So Heun and RK4 converge with orders $2$ and $4$. The bound is usually pessimistic in its constant (it allows every local error to be amplified by the worst possible factor), but it has the right power of $h$, and it shows that consistency plus a Lipschitz increment function implies convergence.

✏️ Example 5 — Euler's method meets $e$
Apply Euler's method to $y'=y$, $y(0)=1$ on $[0,1]$ with $h=1/N$. Find $y_N$ exactly, show that the global error at $x=1$ is $\tfrac{e}{2N}+O(N^{-2})$, and compare with the bound of Theorem 5.48.
Iterate $y_{n+1}=y_n+hy_n=(1+h)y_n$, so $y_N=(1+\tfrac1N)^N$ — the sequence that defines $e$.
Expand $\log y_N=N\log(1+\tfrac1N)=1-\tfrac{1}{2N}+\tfrac{1}{3N^2}-\cdots$, so $y_N=e\cdot e^{-1/(2N)+O(N^{-2})}=e\big(1-\tfrac1{2N}\big)+O(N^{-2})$. Hence $e-y_N=\tfrac{e}{2N}+O(N^{-2})$: first order, as predicted. Also $y_N\lt e$ for every $N$, since $\log(1+t)\lt t$ for $t\gt0$.
Numbers $N=10$: $y_N=2.593742$, error $0.1245$ (and $\tfrac{e}{2N}=0.1359$); $N=100$: error $0.01347$ ($0.01359$); $N=1000$: error $0.001358$ ($0.001359$).
Bound $L=1$, $M_2=\max_{[0,1]}|y''|=e$: $|e_N|\le\tfrac{he}{2}(e-1)=2.34h$, while the true error is about $1.36h$. Same order, pessimistic constant.
🧭 Estimating the error in practice
  1. Compute $y$ at the final point $X$ with steps $h$ and $h/2$; call the results $Y_h$, $Y_{h/2}$.
  2. For a method of order $p$, $y(X)-Y_{h/2}\approx\dfrac{Y_{h/2}-Y_h}{2^p-1}$ — Richardson's error estimate (Richardson extrapolation, §5.4), justified because the global error has an expansion $y(X)-Y_h=c_ph^p+O(h^{p+1})$.
  3. If the estimate is too large, reduce $h$ by the factor $(\text{tolerance}/\text{estimate})^{1/p}$. Production codes do this step by step ("adaptive step size"), comparing two embedded RK formulas of orders $4$ and $5$.
  4. Also check the observed ratio $\dfrac{Y_h-Y_{2h}}{Y_{h/2}-Y_h}\approx2^p$: if it is far from $2^p$, the step is not yet in the asymptotic range (or the solution is not smooth), and the error estimate is not trustworthy.

Absolute Stability and Stiffness

Theorem 5.48 is about $h\to0$. In practice $h$ is fixed, and a second question arises: does the computed solution behave qualitatively like the true one? The standard test is the linear equation $y'=\lambda y$, $\lambda\in\mathbb{C}$, $\operatorname{Re}\lambda\lt0$, whose solutions $e^{\lambda x}$ decay. Near any solution of a smooth system, the linearisation $u'=Ju$ ($J$ the Jacobian) behaves like a combination of such equations with $\lambda$ the eigenvalues of $J$, so the test is far more relevant than it looks.

📘 Definition 5.31 — Stability function; region of absolute stability; A-stability

If a method applied to $y'=\lambda y$ with step $h$ gives $y_{n+1}=R(z)\,y_n$ with $z=h\lambda$, then $R$ is its stability function. The region of absolute stability is

$$\mathcal{S}=\{z\in\mathbb{C}:|R(z)|\lt1\},$$

and the method is absolutely stable for the step $h$ if $h\lambda\in\mathcal{S}$; then $y_n\to0$, as the exact solution does. The set $\mathcal{S}\cap\mathbb{R}$ is the interval of absolute stability. The method is A-stable if $\mathcal{S}$ contains the whole left half-plane $\{\operatorname{Re}z\lt0\}$.

📐 Theorem 5.49 — Stability of Runge–Kutta methods
  1. Euler: $R(z)=1+z$; every two-stage method of order $2$ (Heun, midpoint, Ralston): $R(z)=1+z+\tfrac{z^2}2$; RK4: $R(z)=1+z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}$.
  2. For an explicit $s$-stage RK method, $R$ is a polynomial of degree $\le s$; if the method has order $p$, then $R(z)=\sum_{k=0}^p\tfrac{z^k}{k!}+O(z^{p+1})$.
  3. No explicit Runge–Kutta method is A-stable; indeed $\mathcal{S}$ is bounded.
  4. The intervals of absolute stability are $(-2,0)$ for Euler and for the order-2 two-stage methods, and $(-\alpha,0)$ with $\alpha=2.7853\ldots$ for RK4.
  5. The implicit backward Euler method $y_{n+1}=y_n+hf(x_{n+1},y_{n+1})$ has $R(z)=\tfrac1{1-z}$, and the implicit trapezoidal rule $y_{n+1}=y_n+\tfrac h2\big[f(x_n,y_n)+f(x_{n+1},y_{n+1})\big]$ has $R(z)=\tfrac{1+z/2}{1-z/2}$; both are A-stable.

Proof. (2) Apply the method to $f=\lambda y$. By induction on $i$, $hk_i=z\big(y_n+\sum_{j\lt i}a_{ij}hk_j\big)$ equals $z\,q_i(z)\,y_n$ with $q_i$ a polynomial of degree $\le i-1$; so $y_{n+1}=\big(1+z\sum b_iq_i(z)\big)y_n=R(z)y_n$ with $\deg R\le s$. The exact solution satisfies $y(x_{n+1})=e^zy(x_n)$, so $T_{n+1}=(e^z-R(z))y(x_n)$. Order $p$ means this is $O(h^{p+1})$ for each fixed $\lambda$, i.e. $e^z-R(z)=O(z^{p+1})$: the coefficients of $R$ up to $z^p$ are those of $e^z$. (1) follows from (2) when $s=p$ (Euler: $s=p=1$; the two-stage methods: $s=p=2$, also computed in Example 4; RK4: $s=p=4$, Theorem 5.47).

(3) $R$ is a non-constant polynomial, so $|R(z)|\to\infty$ as $|z|\to\infty$; $\mathcal{S}$ is bounded and cannot contain the left half-plane.

(4) Euler: $|1+z|\lt1$ for real $z$ iff $-2\lt z\lt0$. Two-stage: $1+z+\tfrac{z^2}2=\tfrac12\big((1+z)^2+1\big)\ge\tfrac12$ for real $z$, and it is $\lt1$ iff $(1+z)^2\lt1$ iff $-2\lt z\lt0$. RK4: first, $R(z)\gt0$ for all real $z$: at a critical point $R'(z)=1+z+\tfrac{z^2}2+\tfrac{z^3}6=0$ we have $R(z)=R'(z)+\tfrac{z^4}{24}=\tfrac{z^4}{24}\gt0$ (and $z\ne0$ there because $R'(0)=1$), so the minimum value of $R$ is positive. So $|R(z)|\lt1$ iff $R(z)-1=z\,q(z)\lt0$, where $q(z)=1+\tfrac z2+\tfrac{z^2}6+\tfrac{z^3}{24}$. Since $q'(z)=\tfrac12+\tfrac z3+\tfrac{z^2}8$ has negative discriminant, $q$ is increasing and has one real zero $-\alpha$, $\alpha=2.7853$. For $z\lt0$ we need $q(z)\gt0$, i.e. $z\gt-\alpha$.

(5) Backward Euler: $y_{n+1}=y_n+zy_{n+1}$, so $R=\tfrac1{1-z}$, and $|1-z|\gt1$ whenever $\operatorname{Re}z\lt0$. Trapezoidal: $(1-\tfrac z2)y_{n+1}=(1+\tfrac z2)y_n$; for $\operatorname{Re}z\lt0$ the point $\tfrac z2$ is closer to $-1$ than to $+1$, so $|1+\tfrac z2|\lt|1-\tfrac z2|$. ∎

The practical meaning of (4): for $y'=-50y$ Euler needs $h\lt0.04$ and RK4 $h\lt0.0557$, however smooth the solution is. An equation (or system) is called stiff when it contains rapidly decaying components (eigenvalues with large negative real part) alongside the slow behaviour we actually want to follow: explicit methods are then forced to use tiny steps for stability long after the fast components have died out, while A-stable implicit methods can take steps suited to the slow solution.

✏️ Example 6 — When a small error explodes
(a) Apply Euler's method to $y'=-20y$, $y(0)=1$ with $h=0.05$, $0.1$, $0.15$, and RK4 with $h=0.1$ and $0.15$. (b) For $y'=-20(y-\sin x)+\cos x$, $y(0)=1$ (exact solution $y=\sin x+e^{-20x}$), what happens with Euler for $h=0.09$ and $h=0.11$?
(a) Euler $y_n=(1-20h)^n$. $h=0.05$: factor $0$, so $y_n=0$ for $n\ge1$ (crude but stable). $h=0.1$: factor $-1$, $y_n=(-1)^n$ — neither decaying nor growing, on the boundary of $\mathcal{S}$. $h=0.15$: factor $-2$, $y_n=(-2)^n$: $1,-2,4,-8,16,\dots$, although the true solution is $e^{-20x}\approx0$.
(a) RK4 $z=-2$: $R(-2)=1-2+2-\tfrac43+\tfrac23=\tfrac13$, stable. $z=-3$: $R(-3)=1-3+4.5-4.5+3.375=1.375\gt1$: unstable, as $-3\lt-2.785$.
(b) The solution is essentially $\sin x$ after $x\approx0.3$; nothing in it changes on a scale of $0.1$. But the error equation is $e'=-20e$ (plus local errors), so Euler multiplies errors by $1-20h$ each step. With $h=0.09$ the factor is $-0.8$: Euler gives $y(1.98)\approx0.9269$ against $0.9174$, an honest first-order error. With $h=0.11$ the factor is $-1.2$: the computed value at $x=1.98$ is $27.5$, and it keeps growing like $1.2^n$.
Lesson Stability, not accuracy, limits the step here — the signature of stiffness. Backward Euler, with $R(z)=\tfrac1{1-z}$, is stable for every $h\gt0$ on this problem.
Interactive 2 — Regions of Absolute Stability

Left: the complex $z$-plane with the region $|R(z)|\lt1$ of the chosen method shaded. The point $z=h\lambda$ moves along the dashed ray as $h$ changes. Right: the numerical solution $\operatorname{Re}y_n$ of $y'=\lambda y$, $y(0)=1$ (dots) against the exact $\operatorname{Re}e^{\lambda x_n}$ (grey). When $z$ leaves the shaded region the dots stop decaying and explode. Compare Euler and RK4 on the oscillatory $\lambda=6i$, and try the implicit backward Euler method.

—
z = hλ—
|R(z)|—
largest stable h for this λ—
|y₃₀| (exact: e^(30·Re z))—

Systems and Higher-Order Equations

Every method above applies word for word to a system $\mathbf{u}'=\mathbf{F}(x,\mathbf{u})$, $\mathbf{u}(x_0)=\mathbf{u}_0\in\mathbb{R}^m$: the unknowns $y_n$, the slopes $k_i$ and the increments become vectors, and the theory (Theorems 5.46–5.48, with $|\cdot|$ replaced by a norm and $L$ by a Lipschitz constant of $\mathbf{F}$) is unchanged. A higher-order equation is first reduced to a first-order system (§1.3, Systems of First-Order Equations): for $y''=g(x,y,y')$ put $u_1=y$, $u_2=y'$,

$$u_1'=u_2,\qquad u_2'=g(x,u_1,u_2).$$

For a linear system $\mathbf{u}'=A\mathbf{u}$ with $A$ diagonalisable, a Runge–Kutta method gives $\mathbf{u}_{n+1}=R(hA)\mathbf{u}_n$, and in the eigenbasis of $A$ this decouples into scalar test equations: the method is absolutely stable iff $h\lambda\in\mathcal{S}$ for every eigenvalue $\lambda$ of $A$. For oscillations ($\lambda=\pm i\omega$) the relevant part of $\mathcal{S}$ is the imaginary axis, which Euler's region does not meet at all.

✏️ Example 7 — The harmonic oscillator
Write $y''+y=0$ as a system. (a) Show that Euler's method multiplies the "energy" $E=y^2+y'^2$ by exactly $1+h^2$ at every step, and compute the result at $x=1$ for $y(0)=1$, $y'(0)=0$, $h=0.1$. (b) Take one RK4 step with $h=0.5$ for $y(0)=0$, $y'(0)=1$, and compare with $(\sin0.5,\cos0.5)$.
System $\mathbf{u}=(u,v)=(y,y')$, $\mathbf{u}'=J\mathbf{u}$ with $J=\begin{pmatrix}0&1\\-1&0\end{pmatrix}$, eigenvalues $\pm i$. The exact flow is a rotation, so $E=u^2+v^2$ is constant.
(a) Euler $u_{n+1}=u_n+hv_n$, $v_{n+1}=v_n-hu_n$, so $$u_{n+1}^2+v_{n+1}^2=(1+h^2)(u_n^2+v_n^2),$$ the cross terms cancelling. The numerical orbit spirals outwards for every $h\gt0$ ($|R(\pm ih)|=\sqrt{1+h^2}\gt1$). With $h=0.1$, ten steps give $(u_{10},v_{10})=(0.5708,-0.8825)$ against $(\cos1,-\sin1)=(0.5403,-0.8415)$, and $E_{10}=1.01^{10}=1.1046$.
(b) RK4 With $\mathbf{u}_0=(0,1)$, $h=0.5$: $\mathbf{k}_1=(1,0)$, $\mathbf{k}_2=J(\mathbf{u}_0+0.25\mathbf{k}_1)=(1,-0.25)$, $\mathbf{k}_3=J(\mathbf{u}_0+0.25\mathbf{k}_2)=(0.9375,-0.25)$, $\mathbf{k}_4=J(\mathbf{u}_0+0.5\mathbf{k}_3)=(0.875,-0.46875)$. Then $$\mathbf{u}_1=\mathbf{u}_0+\tfrac{0.5}{6}(\mathbf{k}_1+2\mathbf{k}_2+2\mathbf{k}_3+\mathbf{k}_4)=(0.479167,\ 0.877604),$$ against $(0.479426,\ 0.877583)$.
Energy For RK4, $|R(ih)|^2=1-\tfrac{h^6}{72}+\tfrac{h^8}{576}$ (Exercise IV.6), which is $0.99979$ for $h=0.5$: a very slow inward spiral, stable as long as $h\le2\sqrt2$. A qualitatively correct method for oscillations must keep $|R(i\omega h)|$ very close to $1$.
⚠️ Counterexamples and pitfalls
  1. Local versus global order. Euler's local truncation error is $O(h^2)$ but its global error is $O(h)$; RK4: $O(h^5)$ locally, $O(h^4)$ globally. "Euler's method is of order 2" is false (under the convention $T_{n+1}=O(h^{p+1})$).
  2. "Modified Euler" is not one formula. Usually Heun's predictor–corrector, sometimes with the corrector iterated, occasionally the midpoint method. All are of order $2$, but their numerical values differ (Example 3: $1.1045$ versus $1.10475$ after one step).
  3. Convergence is not stability. Euler converges as $h\to0$ for $y'=-20y$, yet with $h=0.15$ it produces $(-2)^n$. Convergence is a statement about $h\to0$; absolute stability about a fixed $h$.
  4. Higher order does not cure instability. RK4's stability interval $(-2.785,0)$ is barely longer than Euler's $(-2,0)$. No explicit RK method is A-stable (Theorem 5.49(3)); stiff problems need implicit methods.
  5. Euler cannot represent oscillations. On $y''+y=0$ the energy grows by the factor $1+h^2$ every step for every $h\gt0$; Heun also grows slightly ($|R(ih)|^2=1+\tfrac{h^4}4$); RK4 decays slightly. Only the (implicit) trapezoidal rule preserves it exactly.
  6. Two stages cannot give order 3, and $s$ stages never give order above $s$ (order $5$ needs $6$ stages). A question claiming a "third-order two-stage Runge–Kutta method" is wrong.
  7. Picard iterates exist everywhere but converge only where the solution does. For $y'=y^2$, $y(0)=1$, every $\varphi_n$ is a polynomial defined on all of $\mathbb{R}$, but $y=\tfrac1{1-x}$ blows up at $x=1$; for $x\ge1$ the iterates approximate nothing.
  8. Orders assume smoothness. For $y'=\sqrt{|y|}$ or $y'=|x-\tfrac12|$ the expansions behind Theorems 5.46–5.47 fail at the non-smooth point, and RK4 drops to low order on steps that straddle it.
  9. Same cost, different accuracy. Compare methods per function evaluation, not per step: one RK4 step costs as much as four Euler steps (Example 3 shows RK4 still wins by orders of magnitude).
  10. The global error bound is not an estimate. Theorem 5.48 often overestimates by large factors (it grows like $e^{\Lambda(X-x_0)}$ even when the solution decays). Use step halving to estimate errors.

Exercises

I
Picard's method and the Taylor series method.
  • 1 For $y'=x^2+y$, $y(0)=1$, compute $\varphi_1,\varphi_2,\varphi_3$. Evaluate them at $x=0.2$ and compare with the exact solution $y=3e^x-x^2-2x-2$.
  • 2 For $y'=y^2$, $y(0)=1$, compute $\varphi_1,\varphi_2,\varphi_3$ and show that $\varphi_n$ agrees with $\tfrac1{1-x}=\sum x^k$ up to the term $x^n$. What happens for $x\ge1$?
  • 3 Apply Picard's method to the system $y'=z$, $z'=-y$, $y(0)=0$, $z(0)=1$ and identify the limits.
  • 4 Derive the formulas for $y''$ and $y'''$ in Definition 5.27.
  • 5 Use the Taylor method of order $2$ with $h=0.1$ to compute $y(0.1)$ and $y(0.2)$ for $y'=x+y$, $y(0)=1$; compare with $y=2e^x-x-1$.
  • 6 Use the Taylor method of order $4$ to take one step of size $0.2$ for $y'=-2xy$, $y(0)=1$.
  • 7 Show that the Taylor method of order $p$ has order $p$ in the sense of Definition 5.28 and deduce from Theorem 5.48 that its global error is $O(h^p)$.
1 $\varphi_1=1+x+\tfrac{x^3}3$; $\varphi_2=1+x+\tfrac{x^2}2+\tfrac{x^3}3+\tfrac{x^4}{12}$; $\varphi_3=1+x+\tfrac{x^2}2+\tfrac{x^3}2+\tfrac{x^4}{12}+\tfrac{x^5}{60}$. The exact Taylor series is $1+x+\tfrac{x^2}2+\tfrac{x^3}2+\tfrac{x^4}8+\cdots$; $\varphi_n$ is correct up to $x^n$ (Theorem 5.45). At $0.2$: $1.202667$, $1.222800$, $1.224139$; exact $1.224208$.
2 $\varphi_1=1+x$, $\varphi_2=1+x+x^2+\tfrac{x^3}3$, $\varphi_3=1+x+x^2+x^3+\tfrac23x^4+\tfrac13x^5+\tfrac19x^6+\tfrac1{63}x^7$. By Theorem 5.45 the part of degree $\le n$ is $1+x+\cdots+x^n$. The solution exists only for $x\lt1$; the iterates are polynomials defined for all $x$, but for $x\ge1$ they diverge as $n\to\infty$ (e.g. $\varphi_n(1)\ge n+1$) and approximate nothing.
3 $(y_0,z_0)=(0,1)$; $y_1=x$, $z_1=1$; $y_2=x$, $z_2=1-\tfrac{x^2}2$; $y_3=x-\tfrac{x^3}{6}$, $z_3=1-\tfrac{x^2}2$; $z_4=1-\tfrac{x^2}2+\tfrac{x^4}{24}$; … The iterates are the partial sums of the Maclaurin series of $\sin x$ and $\cos x$, which they converge to on all of $\mathbb{R}$ (the system is linear, globally Lipschitz).
4 $y''=\tfrac{d}{dx}f(x,y(x))=f_x+f_yy'=f_x+ff_y$. Differentiating again by the chain rule, $\tfrac{d}{dx}f_x=f_{xx}+ff_{xy}$, $\tfrac{d}{dx}(ff_y)=(f_x+ff_y)f_y+f(f_{xy}+ff_{yy})$; adding gives $y'''=f_{xx}+2ff_{xy}+f^2f_{yy}+f_y(f_x+ff_y)$.
5 $y''=1+y'=1+x+y$. Step 1: $y_1=1+0.1(1)+0.005(2)=1.11$. Step 2: at $(0.1,1.11)$: $y'=1.21$, $y''=2.21$, $y_2=1.11+0.121+0.01105=1.24205$. Exact: $1.110342$, $1.242806$ (errors $3.4\times10^{-4}$, $7.6\times10^{-4}$). (For this linear equation the result coincides with Heun's method — Exercise II.2.)
6 At $0$: $y=1$, $y'=-2xy=0$, $y''=-2y-2xy'=-2$, $y'''=-4y'-2xy''=0$, $y^{(4)}=-6y''-2xy'''=12$. So $y(0.2)\approx1-\tfrac{0.04}{2}\cdot2+\tfrac{0.0016}{24}\cdot12=0.9608$; exact $e^{-0.04}=0.960789$, error $1.1\times10^{-5}$.
7 $\Phi=\sum_{k=1}^p\tfrac{h^{k-1}}{k!}f^{[k-1]}$, and Taylor's theorem gives $y(x_{n+1})-y(x_n)-h\Phi(x_n,y(x_n);h)=\tfrac{h^{p+1}}{(p+1)!}y^{(p+1)}(\xi)$, so $|T_{n+1}|\le Ch^{p+1}$ with $C=\max|y^{(p+1)}|/(p+1)!$. If the $f^{[k]}$ are Lipschitz in $y$ with constants $L_k$ (e.g. bounded partial derivatives on the relevant region), then $\Phi$ is Lipschitz with $\Lambda=\sum_k\tfrac{h_1^{k-1}}{k!}L_{k-1}$. Theorem 5.48 gives $|e_n|=O(h^p)$.
II
Euler, modified Euler and Runge–Kutta steps.
  • 1 Use Euler's method with $h=0.1$ for $y'=x+y$, $y(0)=1$ up to $x=0.3$ and tabulate the errors.
  • 2 Use modified Euler (Heun) with $h=0.1$ for the same IVP up to $x=0.2$.
  • 3 For $y'=xy$, $y(0)=1$, take one step of $h=0.2$ with Heun and with the midpoint method; compare with $e^{0.02}$.
  • 4 Take one RK4 step of size $0.2$ for $y'=x+y$, $y(0)=1$, listing $k_1,\dots,k_4$.
  • 5 Take one RK4 step of size $0.5$ for $y'=-2xy$, $y(0)=1$, and compare with $e^{-0.25}$.
  • 6 Show that for $y'=f(x)$ Euler, Heun, the midpoint method and RK4 reduce to the left rectangle, trapezoidal, midpoint and Simpson rules.
  • 7 Verify that RK4 applied to $y'=\lambda y$ gives $y_{n+1}=\big(1+z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}\big)y_n$, $z=h\lambda$.
  • 8 Take one step of Ralston's method ($b=(\tfrac14,\tfrac34)$, $c=\tfrac23$) with $h=0.1$ for $y'=y-x^2$, $y(0)=1$, and compare with Example 3.
  • 9 A cup of tea cools according to Newton's law $T'=-0.1(T-20)$ ($T$ in °C, time in minutes), $T(0)=90$. Use modified Euler with $h=1$ min to find $T(1)$ and $T(2)$, and compare with the exact solution.
1 $y_1=1.1$, $y_2=1.1+0.1(0.1+1.1)=1.22$, $y_3=1.22+0.1(0.2+1.22)=1.362$. Exact $1.110342$, $1.242806$, $1.399718$; errors $0.0103$, $0.0228$, $0.0377$ — growing roughly linearly with $x$.
2 Step 1: $k_1=1$, $y^*=1.1$, $k_2=0.1+1.1=1.2$, $y_1=1+0.05(2.2)=1.11$. Step 2: $k_1=1.21$, $y^*=1.231$, $k_2=0.2+1.231=1.431$, $y_2=1.11+0.05(2.641)=1.24205$. Errors $3.4\times10^{-4}$, $7.6\times10^{-4}$.
3 Heun: $k_1=0$, $k_2=f(0.2,1)=0.2$, $y_1=1+0.1(0.2)=1.02$. Midpoint: $k_2=f(0.1,1)=0.1$, $y_1=1+0.2(0.1)=1.02$. Exact $e^{0.02}=1.020201$: error $2.0\times10^{-4}$ for both.
4 $k_1=1$, $k_2=f(0.1,1.1)=1.2$, $k_3=f(0.1,1.12)=1.22$, $k_4=f(0.2,1.244)=1.444$. $y_1=1+\tfrac{0.2}{6}(1+2.4+2.44+1.444)=1.2428$; exact $2e^{0.2}-1.2=1.2428055$, error $5.5\times10^{-6}$.
5 $k_1=0$, $k_2=f(0.25,1)=-0.5$, $k_3=f(0.25,0.875)=-0.4375$, $k_4=f(0.5,0.78125)=-0.78125$. $y_1=1+\tfrac{0.5}{6}(0-1-0.875-0.78125)=0.778646$; exact $0.778801$, error $1.5\times10^{-4}$.
6 With $f=f(x)$ the stages no longer depend on $y$: Euler $hf(x_n)$; Heun $\tfrac h2[f(x_n)+f(x_{n+1})]$; midpoint $hf(x_n+\tfrac h2)$; RK4 $\tfrac h6[f(x_n)+4f(x_n+\tfrac h2)+f(x_{n+1})]$ since $k_2=k_3$. These are the rules of §5.4 on $[x_n,x_{n+1}]$.
7 See the proof of Theorem 5.47(2): $hk_1=zy_n$, $hk_2=(z+\tfrac{z^2}2)y_n$, $hk_3=(z+\tfrac{z^2}2+\tfrac{z^3}4)y_n$, $hk_4=(z+z^2+\tfrac{z^3}2+\tfrac{z^4}4)y_n$; then $\tfrac16(1,2,2,1)$-weighting gives $z+\tfrac{z^2}2+\tfrac{z^3}6+\tfrac{z^4}{24}$.
8 $k_1=1$, $k_2=f\big(\tfrac{0.2}{3},1+\tfrac{0.2}{3}\big)=1.066667-0.004444=1.062222$, $y_1=1+0.1\big(0.25+0.75\cdot1.062222\big)=1.104667$. Error $1.6\times10^{-4}$: between Heun ($3.3\times10^{-4}$) and midpoint ($7.9\times10^{-5}$) — all order $2$, different constants.
9 For $T'=-0.1(T-20)$ one Heun step multiplies $T-20$ by $1-0.1+\tfrac{0.01}{2}=0.905$. So $T(1)\approx20+70(0.905)=83.35$ °C and $T(2)\approx20+70(0.905)^2=77.33$ °C. Exact: $T=20+70e^{-0.1t}$, giving $83.34$ °C and $77.31$ °C.
III
Errors, order and convergence.
  • 1 Compute the local truncation error of Euler's method for $y'=y$ exactly, and of Heun's method, as power series in $h$.
  • 2 Show that Euler's method for $y'=\lambda y$, $y(0)=1$ ($\lambda\in\mathbb{R}$) has global error $e^{\lambda x}-y_n=\tfrac{\lambda^2x}{2}e^{\lambda x}h+O(h^2)$ at a fixed $x=nh$.
  • 3 Use Theorem 5.48 to bound the global error of Euler's method at $x=1$ for $y'=x-y$, $y(0)=1$, $h=0.1$, and compare with the actual error (exact $y=x-1+2e^{-x}$).
  • 4 Heun's method for $y'=y-x^2$, $y(0)=1$ gives $2.277757$ at $x=1$ with $h=0.1$ and $2.280715$ with $h=0.05$. Estimate the error of the second value and give an improved value. Compare with $y(1)=5-e$.
  • 5 Prove that the increment function of Heun's method is Lipschitz with constant $L(1+\tfrac{hL}2)$.
  • 6 Prove: if $\Phi$ is continuous, Lipschitz in $y$, and consistent ($\Phi(x,y;0)=f(x,y)$), then $\max_n|e_n|\to0$ as $h\to0$ (with $e_0=0$), even without assuming any order.
  • 7 A method gives errors $2.1\times10^{-3}$, $2.7\times10^{-4}$, $3.4\times10^{-5}$ at a fixed point with $h$, $h/2$, $h/4$. What is its order?
  • 8 For $y'=y-x^2$, $y(0)=1$ the errors at $x=1$ behave like $0.36h$ (Euler) and $1.3\times10^{-6}(10h)^4$ (RK4). How many evaluations of $f$ does each method need for an error of $10^{-6}$?
1 Euler: $T_{n+1}=y(x_n)\big(e^h-1-h\big)=y(x_n)\big(\tfrac{h^2}2+\tfrac{h^3}6+\cdots\big)$. Heun: the factor is $1+h+\tfrac{h^2}2$, so $T_{n+1}=y(x_n)\big(\tfrac{h^3}6+\tfrac{h^4}{24}+\cdots\big)$. Orders $1$ and $2$.
2 $y_n=(1+\lambda h)^n$ and $n\log(1+\lambda h)=\tfrac xh\big(\lambda h-\tfrac{\lambda^2h^2}2+O(h^3)\big)=\lambda x-\tfrac{\lambda^2x}2h+O(h^2)$. Hence $y_n=e^{\lambda x}\big(1-\tfrac{\lambda^2x}{2}h\big)+O(h^2)$ and the error is $\tfrac{\lambda^2x}{2}e^{\lambda x}h+O(h^2)$. (For $\lambda=x=1$ this is Example 5.)
3 $L=1$, $y''=2e^{-x}$, $M_2=2$: $|e_{10}|\le\tfrac{0.1\cdot2}{2}(e-1)=0.172$. Actually $y_{10}=0.697357$ and $y(1)=0.735759$, so $e_{10}=0.0384$ — about $4.5$ times smaller than the bound (the solution decays, which the bound ignores).
4 $p=2$: error estimate $\tfrac{2.280715-2.277757}{3}=0.000986$; improved value $2.280715+0.000986=2.281701$. Exact $2.281718$: the true error of $2.280715$ was $0.001003$, and the improved value is off by only $1.7\times10^{-5}$.
5 $\Phi(u)=\tfrac12\big[f(x,u)+f(x+h,u+hf(x,u))\big]$. Then $|\Phi(u)-\Phi(v)|\le\tfrac12L|u-v|+\tfrac12L\big(|u-v|+h|f(x,u)-f(x,v)|\big)\le\tfrac12L|u-v|+\tfrac12L(1+hL)|u-v|=L(1+\tfrac{hL}2)|u-v|$.
6 By the mean value theorem $y(x_{n+1})-y(x_n)=hf(\eta_n,y(\eta_n))$, so $T_{n+1}=h\big[f(\eta_n,y(\eta_n))-\Phi(x_n,y(x_n);h)\big]$. The bracket tends to $0$ uniformly in $n$ as $h\to0$: $\Phi$ is uniformly continuous on the compact set $\{(x,y(x),h)\}$ (with $h\in[0,h_1]$), $\Phi(x,y;0)=f(x,y)$, and $|\eta_n-x_n|\le h$. So $|T_{n+1}|\le h\,\omega(h)$ with $\omega(h)\to0$, and the proof of Theorem 5.48 (with $Ch^{p+1}$ replaced by $h\omega(h)$) gives $|e_n|\le\tfrac{\omega(h)}{\Lambda}(e^{\Lambda(X-x_0)}-1)\to0$.
7 Ratios $2.1/0.27=7.8$ and $2.7/0.34=7.9\approx8=2^3$: order $3$.
8 Euler: $0.36h\le10^{-6}$ needs $h\le2.8\times10^{-6}$, about $360\,000$ steps and evaluations. RK4: $1.3\times10^{-6}(10h)^4\le10^{-6}$ needs $10h\le0.94$, $h\le0.094$, i.e. $11$ steps ($44$ evaluations; actual error with $h=\tfrac1{11}$: $9.0\times10^{-7}$). High order pays off enormously at high accuracy.
IV
Stability, systems and higher-order equations.
  • 1 For $y'=-50y$, find the largest $h$ for which Euler, Heun and RK4 are absolutely stable.
  • 2 Apply backward Euler to $y'=-50y$, $y(0)=1$ with $h=0.1$. Find $y_n$ and show that backward Euler is A-stable.
  • 3 Show that the implicit trapezoidal rule has $R(z)=\tfrac{1+z/2}{1-z/2}$, that $|R(iy)|=1$ for real $y$, and that it is A-stable.
  • 4 Write $y''+3y'+2y=0$, $y(0)=1$, $y'(0)=0$ as a system; take one Euler step and one Heun step with $h=0.1$ and compare with the exact solution $y=2e^{-x}-e^{-2x}$. For which $h$ is Euler absolutely stable on this system?
  • 5 For Euler's method on $y''+y=0$ with $h=0.01$, after how many steps has the energy $y^2+y'^2$ doubled?
  • 6 Show that for RK4, $|R(iy)|^2=1-\tfrac{y^6}{72}+\tfrac{y^8}{576}$, and deduce that RK4 is stable on $y'=i\omega y$ iff $h|\omega|\le2\sqrt2$.
  • 7 The IVP $y'=-1000(y-\cos x)-\sin x$, $y(0)=1$ has solution $y=\cos x$. What step does Euler's method need to be stable? Why is this problem called stiff?
  • 8 Prove that an explicit $s$-stage Runge–Kutta method has order at most $s$.
1 $z=-50h$ must lie in the stability interval: Euler and Heun $50h\lt2$, $h\lt0.04$; RK4 $50h\lt2.7853$, $h\lt0.0557$.
2 $y_{n+1}=y_n-5y_{n+1}$, so $y_{n+1}=\tfrac16y_n$ and $y_n=6^{-n}$ — decaying, as it should, although $h$ is $2.5$ times Euler's limit. $|R(z)|=\tfrac1{|1-z|}\lt1$ iff $|1-z|\gt1$, which holds whenever $\operatorname{Re}z\lt0$ (then $\operatorname{Re}(1-z)\gt1$).
3 $y_{n+1}=y_n+\tfrac z2(y_n+y_{n+1})$ gives $R=\tfrac{1+z/2}{1-z/2}$. For $z=iy$: numerator and denominator are complex conjugates, so $|R|=1$. For $\operatorname{Re}z\lt0$, $|1+\tfrac z2|^2-|1-\tfrac z2|^2=2\operatorname{Re}z\lt0$, so $|R|\lt1$.
4 $\mathbf u=(y,y')$, $\mathbf u'=(u_2,\,-2u_1-3u_2)$. Euler: $\mathbf u_1=(1,0)+0.1(0,-2)=(1,-0.2)$. Heun: $\mathbf k_1=(0,-2)$, predictor $(1,-0.2)$, $\mathbf k_2=(-0.2,-1.4)$, $\mathbf u_1=(1,0)+0.05(-0.2,-3.4)=(0.99,-0.17)$. Exact $(0.990944,-0.172213)$. Eigenvalues of the matrix are $-1,-2$; Euler needs $|1-h|\lt1$ and $|1-2h|\lt1$, i.e. $0\lt h\lt1$.
5 $(1+h^2)^n=1.0001^n=2$ gives $n=\tfrac{\ln2}{\ln1.0001}=6931.8$: after $6932$ steps ($x\approx69.3$, about $11$ periods).
6 $R(iy)=\big(1-\tfrac{y^2}2+\tfrac{y^4}{24}\big)+i\big(y-\tfrac{y^3}6\big)$. Squaring and adding, the terms up to $y^4$ cancel and $|R(iy)|^2=1-\tfrac{y^6}{72}+\tfrac{y^8}{576}$. This is $\le1$ iff $\tfrac{y^8}{576}\le\tfrac{y^6}{72}$ iff $y^2\le8$, i.e. $|h\omega|\le2\sqrt2\approx2.83$ (with $|R|\lt1$ strictly for $0\lt y^2\lt8$).
7 Errors propagate with the factor $1-1000h$, so stability needs $h\lt0.002$. The solution $\cos x$ is slow and smooth (steps of $0.1$ would resolve it well), but the fast decaying component $e^{-1000x}$ — invisible in the solution, since its coefficient is $0$ — dictates the step of every explicit method. That mismatch of time scales is stiffness.
8 By Theorem 5.49(2), $R$ is a polynomial of degree $\le s$ and order $p$ forces $R(z)=e^z+O(z^{p+1})$. If $p\ge s+1$, the coefficient of $z^{s+1}$ in $R$ would have to be $\tfrac1{(s+1)!}\ne0$, impossible for a polynomial of degree $\le s$. So $p\le s$ (already for the single equation $y'=y$).
🎯 Exam corner CSIR Part BCSIR Part CGATE MA

E1. CSIR Part B GATE MA For $y'=x+y$, $y(0)=1$, one step of the modified Euler (Heun) method with $h=0.1$ gives $y(0.1)\approx$ (A) $1.1000$; (B) $1.1050$; (C) $1.1100$; (D) $1.1103$.

Predictor $y^*=1+0.1(0+1)=1.1$.
Corrector $y_1=1+0.05\big[(0+1)+(0.1+1.1)\big]=1+0.05(2.2)=1.11$. ((A) is Euler; (D) is the exact value $2e^{0.1}-1.1=1.11034$ rounded.)
Answer (C).

E2. CSIR Part B Euler's method with constant step $h\gt0$ is applied to $y'=-50y$, $y(0)=1$. The computed values satisfy $y_n\to0$ as $n\to\infty$ if and only if (A) $h\lt0.02$; (B) $h\lt0.04$; (C) $h\le0.05$; (D) $h\lt0.1$.

Idea $y_n=(1-50h)^n\to0$ iff $|1-50h|\lt1$ iff $0\lt h\lt0.04$.
Check $h=0.04$ gives $(-1)^n$, which does not tend to $0$; $h=0.02$ is the step for which $y_n=0$ for all $n\ge1$, not the limit.
Answer (B).

E3. CSIR Part B For $y'=x-y$, $y(0)=1$, the second Picard iterate (with $\varphi_0=1$) is (A) $1-x+\tfrac{x^2}2$; (B) $1-x+x^2-\tfrac{x^3}6$; (C) $1-x+x^2-\tfrac{x^3}3$; (D) $1-x+\tfrac{x^2}{2}-\tfrac{x^3}6$.

Iterate $\varphi_1=1+\int_0^x(t-1)\,dt=1-x+\tfrac{x^2}2$. $\varphi_2=1+\int_0^x\big(t-1+t-\tfrac{t^2}2\big)dt=1-x+x^2-\tfrac{x^3}6$.
Trap (C) is the Taylor polynomial of degree $3$ of the exact solution $x-1+2e^{-x}$. By Theorem 5.45, $\varphi_2$ is only guaranteed to match up to $x^2$, and indeed its $x^3$ coefficient differs. (A) is $\varphi_1$.
Answer (B).

E4. CSIR Part C Which of the following statements are true? (One or more options may be correct.) (A) Applied to $y'=f(x)$, one step of the classical RK4 method is Simpson's 1/3 rule on $[x_n,x_{n+1}]$ with nodes $x_n$, $x_n+\tfrac h2$, $x_{n+1}$. (B) Applied to $y'=f(x)$, Heun's method is the trapezoidal rule. (C) For $\lambda\lt0$, RK4 applied to $y'=\lambda y$ gives $y_n\to0$ for every $h\gt0$. (D) Some explicit two-stage Runge–Kutta method has order $3$.

(A) True $k_2=k_3=f(x_n+\tfrac h2)$, so $y_{n+1}-y_n=\tfrac h6[f_n+4f_{n+1/2}+f_{n+1}]$ (Theorem 5.47(1)).
(B) True $\tfrac h2[f(x_n)+f(x_{n+1})]$.
(C) False The stability interval is $(-2.785,0)$; e.g. $h\lambda=-3$ gives $R=1.375$ and $|y_n|\to\infty$.
(D) False Theorem 5.46(2): the elementary differential $f_y(f_x+ff_y)$ cannot be matched; for $y'=y$ every two-stage method gives $1+h+\tfrac{h^2}2$.
Answer (A) and (B).

E5. CSIR Part C Euler's method with $h=\tfrac1n$ is applied to $y'=y$, $y(0)=1$, and $Y_n$ denotes the computed approximation to $y(1)=e$. Which of the following are true? (A) $Y_n=(1+\tfrac1n)^n$. (B) $Y_n\lt e$ for every $n$. (C) $n(e-Y_n)\to\tfrac e2$. (D) $e-Y_{2n}=\tfrac12(e-Y_n)$ for every $n$.

(A) True $Y_{k+1}=(1+h)Y_k$.
(B) True $n\log(1+\tfrac1n)\lt n\cdot\tfrac1n=1$ because $\log(1+t)\lt t$ for $t\gt0$.
(C) True Example 5: $e-Y_n=\tfrac{e}{2n}+O(n^{-2})$.
(D) False Halving is only asymptotic: for $n=10$, $e-Y_{10}=0.12454$ and $e-Y_{20}=0.06498$, ratio $1.92$.
Answer (A), (B) and (C).

E6. CSIR Part C The system $u'=v$, $v'=-u$ is solved with a fixed step $h\gt0$, and $E_n=u_n^2+v_n^2$. Which of the following are true? (A) For Euler's method $E_{n+1}=(1+h^2)E_n$. (B) For Euler's method the computed solution remains bounded as $n\to\infty$. (C) For the backward Euler method $E_{n+1}=E_n/(1+h^2)$. (D) For the implicit trapezoidal rule $E_{n+1}=E_n$.

(A) True Example 7(a).
(B) False $E_n=(1+h^2)^nE_0\to\infty$ (unless $E_0=0$).
(C) True $(I-hJ)\mathbf u_{n+1}=\mathbf u_n$ with $J$ skew-symmetric, so $|(I-hJ)\mathbf w|^2=|\mathbf w|^2+h^2|J\mathbf w|^2=(1+h^2)|\mathbf w|^2$ (the cross term $\mathbf w\cdot J\mathbf w$ vanishes). Hence $E_n=(1+h^2)E_{n+1}$.
(D) True $(I-\tfrac h2J)\mathbf u_{n+1}=(I+\tfrac h2J)\mathbf u_n$, and both sides have squared length $(1+\tfrac{h^2}4)$ times that of $\mathbf u_{n+1}$, resp. $\mathbf u_n$. Equivalently $|R(\pm ih)|=1$ (Exercise IV.3).
Answer (A), (C) and (D).
🧑‍🏫 For the teacher

Suggested plan — 3 lectures × 60 min.

  1. From existence to computation. Recall Picard–Lindelöf from §1.1; Theorem 5.45 with its short proof; Example 1 (do $\varphi_1,\varphi_2$ on the board) and Example 2. Definitions 5.28–5.29; draw Euler's tangent polygon and Heun's averaged slope. Theorem 5.46(1) and the naming warning about "modified Euler".
  2. Runge–Kutta and order. Theorem 5.46(2) — the Taylor comparison is worth doing in full once; Definition 5.30 and Theorem 5.47 (prove (1) and (2), state the order conditions). Example 3 on the board for the first step only; project Interactive 1 and let students read off the ratios $2,4,16$. Theorem 5.48 with the discrete Gronwall proof; Example 5.
  3. Stability and systems. The test equation, Definition 5.31 and Theorem 5.49 (prove (4) for Euler and RK2, show the RK4 argument); Example 6; Interactive 2 with $\lambda=-20$ (slide $h$ through $0.1$) and $\lambda=6i$ (Euler versus RK4). Systems and Example 7; finish with Exam-corner E2 and E6.

Discussion questions. (1) Why does a local error of $O(h^{p+1})$ become a global error of $O(h^p)$ and not, say, $O(h^{p+1})$ or $O(h^{p-1})$? (2) If the exact solution of $y'=-1000(y-\cos x)-\sin x$ is just $\cos x$, why does the coefficient $1000$ matter at all? (3) Picard's method and the Taylor method give the same polynomials (Theorem 5.45). Which would you program, and why do neither appear in modern ODE software?

Misconceptions to watch for. Confusing local and global order; using Euler's slope at the new point in Heun's predictor; forgetting to update $x$ in $k_2,k_3,k_4$; thinking a higher-order method is always more accurate or always more stable; believing that convergence as $h\to0$ guarantees sensible results at a given $h$; computing stability intervals from $|R(z)|\le1$ versus $\lt1$ inconsistently (the endpoints give non-decaying, bounded solutions).

Suggested homework. I(1, 2, 5, 6), II(1, 2, 4, 6, 9), III(2, 3, 4, 7), IV(1, 2, 4, 6, 8); Exam corner E1, E3, E4, E6.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. How large a time step can an epidemic model afford? In the SIR model $S'=-\beta SI$, $I'=\beta SI-\gamma I$, $R'=\gamma I$ (fractions of a population, time in days), how do the predicted number of infected people and the timing of the peak depend on the method and on $h$ — and when does a cheap method give a misleading forecast?

Why it matters. Compartmental models of this kind were run daily by public-health agencies around the world during the COVID-19 pandemic, and similar systems model the spread of influenza, measles and dengue. A step-size artefact that shifts the predicted peak by a few days, or overstates it by 40%, can change hospital planning.

What to do.

  1. Take $\beta=0.3$, $\gamma=0.1$ per day, $S(0)=0.99$, $I(0)=0.01$, $R(0)=0$. Compute a reference solution on $[0,160]$ with a library solver (tight tolerances) and record $I(40)$ and the peak of $I$.
  2. Solve with Euler and with RK4 for $h=8,4,2,1,0.5$ days. Tabulate the error in $I(40)$ and compute the observed orders from successive ratios.
  3. Find the largest $h$ for which each method predicts $I(40)$ to three significant figures; count function evaluations.
  4. Check that $S+I+R$ stays equal to $1$ for both methods (why is this exact?), and that the computed $S$ is decreasing. Can you make Euler produce a negative $I$ with a large step?

Tools: Python with NumPy and scipy.integrate.solve_ivp, or a spreadsheet for Euler.

Hand in: A one-page note with the error table, the observed orders, your recommended step for each method, and one plot of $I(t)$ for Euler with $h=4$ against the reference.

Going further. Linearise the model near the disease-free state $(S,I)=(1,0)$ to find the eigenvalue $\beta-\gamma$; then increase $\gamma$ (fast recovery) and $\beta$ together and explore when the explicit methods become unstable. Read about how adaptive solvers (embedded RK pairs such as Dormand–Prince) choose their steps.

Instructor note: the reference peak is $I_{\max}=0.3038$ at $t\approx26.6$ days and $I(40)=0.17845$. Errors in $I(40)$: Euler $0.248,\ 0.085,\ 0.034,\ 0.015,\ 0.0073$ for $h=8,4,2,1,0.5$ (ratios approaching $2$); RK4 $1.5\times10^{-3},\ 1.8\times10^{-4},\ 1.5\times10^{-5},\ 1.1\times10^{-6},\ 7.5\times10^{-8}$ (ratios $8.4,\ 11.9,\ 13.9,\ 14.9\to16$). Three figures in $I(40)$ need $h\approx0.03$ day for Euler (about $5000$ evaluations) but $h=4$ days for RK4 ($160$ evaluations). Euler with $h=8$ overstates the peak by $40\%$ ($0.426$). $S+I+R$ is conserved exactly by every Runge–Kutta method because the right-hand sides sum to zero (linear invariants are preserved).

🧪 Self-check quiz

0/5 correct

Q1. The global error of the classical fourth-order Runge–Kutta method on a fixed interval is

Q2. The local truncation error $y(x_{n+1})-y(x_n)-hf(x_n,y(x_n))$ of Euler's method is

Q3. Euler's method applied to $y'=-10y$ produces decaying values exactly when

Q4. If $f$ is smooth, the $n$-th Picard iterate $\varphi_n$ of $y'=f(x,y)$, $y(x_0)=y_0$ agrees with the Taylor series of the solution at least up to the term

Q5. The modified Euler (Heun) method is