Flow Lab

Syllabus / Module 4: Numerical Methods for ODEs — The Theory

Lesson 4.2

The Euler Family: Forward, Backward, Heun & Midpoint

Explicit and implicit Euler, improved Euler (Heun) and midpoint — derivations, orders and first look at stability on y' = λy.

Written byClaude Sonnet

🌊
Why this matters for simulation: Stam's 'stable fluids' uses implicit (backward Euler) diffusion so it never blows up.

By the end of this lesson you can…

  • Derive the four Euler-type methods from left-, right-, trapezoid- and midpoint-rule quadrature of the integral of f.
  • Write the Newton iteration for a nonlinear implicit step and say when a fixed-point iteration fails.
  • Read Heun as the trapezoidal rule with an Euler predictor, say what that loses, and say when Heun and midpoint agree.
  • Compute the amplification factor R(z), z = h times lambda, of each method and compare it with the exponential to read off the order.
  • State where each method decays, oscillates, blows up, over-damps or rings on y' = lambda y with lambda negative.
On this page
·
  1. One integral, four rules
  2. Implicit steps: solving for yn+1y_{n+1}yn+1​
  3. Heun: the trapezoid with a predictor
  4. The test equation and amplification factors
  5. A first look at stability
  6. Exercise: implicit steps by hand
  7. Problems
  8. What comes next
  9. Key Takeaways

You have already coded forward Euler, backward Euler, the trapezoidal rule with its explicit cousin Heun, and the midpoint method. This lesson is the why. Where do the formulas come from? Why do two need a solve at every step, and what does that buy? And the question that decides what a fluid or flight code can use: at what step size does each one stay sane?

It builds on 4.1 Local Error, Global Error & Order (order, local error) and the Euler picture of 3.2 Direction Fields & Euler's Method Revisited. Oversized steps on nonlinear problems appeared in 3.8 Autonomous Equations, Equilibria & Stability: Euler on the logistic equation turns into the logistic map, and RK4 can invent steady states. Here we study the same failure on a linear equation, where everything can be computed exactly.

▶ Video Four quadrature rules give four methods; the implicit solve; Heun as predictor and corrector; amplification factors and the first stability picture on y' = lambda ycaptions only
Transcript
  1. Four methods, one idea. Over one step, y gains the integral of g, the slope along the true solution. We cannot do that integral exactly, because g depends on the unknown solution, so each method swaps in a quadrature rule.
  2. Left rectangle: forward Euler. Right rectangle: backward Euler. Trapezoid: the trapezoidal rule. Middle rectangle: the midpoint method, using half an Euler step to find y there. The trapezoid overshoots by 0.104, the midpoint undershoots by about half that. Rules exact for lines leave h cubed errors, for constants h squared.
  3. Backward Euler and the trapezoid have y n plus one on both sides: each step is an equation. Take y prime equals minus y squared, y one, h one: G of w is w minus one plus w squared, root 0.618034. Newton jumps to where each tangent crosses zero: 1, 0.666667, 0.619048, 0.618034, roughly squaring the error each pass.
  4. Fixed-point iteration sets w to one minus w squared: no derivative, no solve. From one it goes 1, 0, 1, 0, forever. Each pass multiplies the error by the update's slope at the root, minus 1.236, bigger than one in size. It converges only if h times f y is below one in size: a step limit again.
  5. Heun breaks the circle. An Euler predictor lands at 0.2, on slope minus one; the slope there is minus 0.04. The trapezoid's average slope, minus 0.52, lands at 0.584, against the exact 0.5556. That is one fixed-point pass from the Euler value. Order two survives; the implicit equation is lost.
  6. On the test equation, y prime equals lambda y, one step multiplies by R of z, with z equal to h lambda. Forward Euler: one plus z. Backward Euler: one over one minus z. Trapezoid: one plus z over two, over one minus z over two. Heun and midpoint, both: one plus z plus z squared over two. Halving z shrinks the gap to e to the z by 4, 4, 8, 8.
  7. Heun and midpoint agree on the test equation, and only there. In units of h cubed, y prime equals minus y gives minus 0.1661 for both. For 3 t squared, no predictor error: Heun minus 0.5, midpoint plus 0.25. For minus y squared: minus 0.4877 and minus 0.7377, near the formulas' minus 0.5 and minus 0.75.
  8. Plot R against e to the z, with size one dashed. Forward and backward Euler peel away first, their gap starting at z squared; the trapezoid and Heun hold on to z cubed. At z equals minus two, forward Euler reaches minus one and Heun plus one, and both keep going.
  9. Forward Euler on y prime equals minus ten y, to t equals three. At h 0.05 each step multiplies by 0.5: smooth decay to 8.67 times ten to the minus 19. At 0.15, minus 0.5: decay to 9.54 times ten to the minus 7, flipping sign. At 0.2, minus one: plus or minus one forever. At 0.25, minus 1.5: 130.
  10. At h 0.25, backward Euler multiplies by 0.2857 and ends at 2.96 times ten to the minus 7. The trapezoid, minus 0.1111, alternates to 3.54 times ten to the minus 12. Heun multiplies by 1.625 and reaches 339: its degree-two polynomial R exceeds one in size beyond z equals minus two.
  11. This is the sign and size of R. The step sizes sit at z equals minus ten h. Forward Euler: smooth to minus one, sign flips to minus two, then growth. Backward Euler: smooth always. The trapezoid goes negative past minus two, and at minus 100 is still minus 0.9608: it rings. Heun and midpoint grow past minus two.
  12. Recap: four rules, four methods. An implicit step is an equation, solved by Newton. Heun is the trapezoid with an Euler predictor. Order is how long R matches e to the z. Backward Euler stays smooth; forward Euler, Heun and midpoint need z at least minus two; the trapezoid rings. Next: Runge-Kutta.

One integral, four rules

Integrating y′=f(t,y)y' = f(t, y) over one step is exact:

y(tn+h)=y(tn)+∫tntn+hg(s) ds,g(s)=f(s,y(s)).y(t_n + h) = y(t_n) + \int_{t_n}^{t_n + h} g(s)\,ds, \qquad g(s) = f\big(s, y(s)\big).

The trouble is that gg depends on the unknown solution. Every method in the family replaces the integral by a quadrature rule and deals with the unknown in its own way. Write g′=y′′g' = y'' and g′′=y′′′g'' = y''', since g=y′g = y'.

  • Left rectangle h g(tn)h\,g(t_n) uses the slope we know: forward Euler, yn+1=yn+hf(tn,yn)y_{n+1} = y_n + h f(t_n, y_n). Taylor expansion of gg about tnt_n gives ∫g=hg(tn)+h22g′(tn)+⋯\int g = h g(t_n) + \tfrac{h^2}{2} g'(t_n) + \cdots, so the rule is off by h22 y′′\tfrac{h^2}{2}\,y''.
  • Right rectangle h g(tn+1)h\,g(t_{n+1}) uses the slope at the new point: backward Euler, yn+1=yn+hf(tn+1,yn+1)y_{n+1} = y_n + h f(t_{n+1}, y_{n+1}). Expanding about tn+1t_{n+1}, the rule is off by −h22 y′′-\tfrac{h^2}{2}\,y''.
  • Trapezoid h2(g(tn)+g(tn+1))\tfrac h2\big(g(t_n) + g(t_{n+1})\big) averages the two: the trapezoidal rule, yn+1=yn+h2(f(tn,yn)+f(tn+1,yn+1))y_{n+1} = y_n + \tfrac h2\big(f(t_n, y_n) + f(t_{n+1}, y_{n+1})\big).
  • Midpoint h g(tn+h2)h\,g(t_n + \tfrac h2) samples the middle, but that needs yy at the half step. Take half an Euler step to get it: the midpoint method, yn+1=yn+hf(tn+h2, yn+h2f(tn,yn))y_{n+1} = y_n + h f\big(t_n + \tfrac h2,\ y_n + \tfrac h2 f(t_n, y_n)\big).

Why do the last two beat the first two? Expand gg about the midpoint m=tn+h2m = t_n + \tfrac h2: g(s)=g(m)+g′(m)(s−m)+12g′′(m)(s−m)2+⋯g(s) = g(m) + g'(m)(s - m) + \tfrac12 g''(m)(s-m)^2 + \cdots. The linear term integrates to zero over the symmetric interval, so ∫g=h g(m)+h324g′′(m)\int g = h\,g(m) + \tfrac{h^3}{24} g''(m), and the midpoint rule is off by h324 y′′′\tfrac{h^3}{24}\,y'''. The trapezoid rule is h g(m)+h38g′′(m)h\,g(m) + \tfrac{h^3}{8} g''(m), so it is off by h324−h38=−h312 y′′′\tfrac{h^3}{24} - \tfrac{h^3}{8} = -\tfrac{h^3}{12}\,y'''. Rules exact for straight lines leave h3h^3 errors, rules exact only for constants leave h2h^2: the local errors of the last lesson, orders 1,1,2,21, 1, 2, 2.

methodslope usedexplicit?error of the quadratureorder
forward Eulerat tnt_nyes+h22y′′+\tfrac{h^2}{2}y''1
backward Eulerat tn+1t_{n+1}no−h22y′′-\tfrac{h^2}{2}y''1
trapezoidaverage of both endsno−h312y′′′-\tfrac{h^3}{12}y'''2
midpointat tn+h/2t_n + h/2yes (Euler half step)+h324y′′′+\tfrac{h^3}{24}y''', plus the half-step error2

For the implicit rows the error is the residual left when the exact solution is substituted into the formula; it matches the “start on the curve, take one step” error of the last lesson to leading order. The midpoint row hides a subtlety: the half step is itself approximate, so the quadrature error is not the whole story (see below).

Implicit steps: solving for yn+1y_{n+1}

Forward Euler and the midpoint method compute yn+1y_{n+1} from known quantities. Backward Euler and the trapezoidal rule have yn+1y_{n+1} on both sides, so each step is an equation to solve. For a linear equation y′=λy+g(t)y' = \lambda y + g(t) that is easy: w=yn+h(λw+g(tn+1))w = y_n + h(\lambda w + g(t_{n+1})) gives w=(yn+h g(tn+1))/(1−hλ)w = \big(y_n + h\,g(t_{n+1})\big)/(1 - h\lambda). For a nonlinear ff it is a root-finding problem. Call the unknown ww and put everything on one side:

G(w)=w−yn−h f(tn+1,w)=0.G(w) = w - y_n - h\,f(t_{n+1}, w) = 0.

Newton’s method linearizes GG at the current guess and jumps to the root of that line. Since G′(w)=1−h fy(tn+1,w)G'(w) = 1 - h\,f_y(t_{n+1}, w),

w←w−G(w)1−h fy(tn+1,w),started at w=yn.w \leftarrow w - \frac{G(w)}{1 - h\,f_y(t_{n+1}, w)}, \qquad \text{started at } w = y_n.

(For a system, fyf_y becomes the Jacobian matrix JJ and each update solves the linear system (I−hJ) δ=−G(I - hJ)\,\delta = -G: the price of an implicit step.) The alternative is fixed-point iteration, w←yn+hf(tn+1,w)w \leftarrow y_n + h f(t_{n+1}, w): no derivative, no solve. Its error is multiplied by about hfyh f_y per pass, so it converges only if ∣hfy∣<1\lvert h f_y\rvert \lt 1, a limit on hh of the same kind the explicit methods have, which defeats the purpose.

Block F of the cell below takes one backward Euler step for y′=−y2y' = -y^2 from yn=1y_n = 1 with h=1h = 1, so G(w)=w+w2−1G(w) = w + w^2 - 1 and the root is (5−1)/2=0.618034(\sqrt5 - 1)/2 = 0.618034. Newton goes 1, 0.666667, 0.619048, 0.6180341,\ 0.666667,\ 0.619048,\ 0.618034, roughly doubling the correct digits each pass. Fixed-point iteration goes 1, 0, 1, 0,…1,\ 0,\ 1,\ 0, \ldots forever, because at the root ∣hfy∣=2×0.618=1.236>1\lvert h f_y\rvert = 2 \times 0.618 = 1.236 \gt 1.

The rest of the cell is the whole family at work (the implicit steps call a scipy root-finder). Block A checks the R(z)R(z) formulas of a later section against actual steps, B measures orders on y′=−y2y' = -y^2 (exact 1/(1+t)1/(1+t)), C compares Heun and midpoint, D runs y′=−10yy' = -10y at four step sizes, E uses imaginary λ\lambda, G tabulates RR, H is a diffusion grid.

🐍 PythonThe Euler family side by side: R(z), orders, midpoint versus Heun, stability, Newton versus fixed point
⌘/Ctrl + Enter

Heun: the trapezoid with a predictor

The trapezoidal rule has the best accuracy of the lot per unit of work, but f(tn+1,yn+1)f(t_{n+1}, y_{n+1}) needs the unknown. Heun’s method breaks the circle by predicting yn+1y_{n+1} with a forward Euler step and then correcting with the trapezoid formula:

y~=yn+hf(tn,yn)(predict),yn+1=yn+h2(f(tn,yn)+f(tn+1,y~))(correct).\tilde y = y_n + h f(t_n, y_n) \quad\text{(predict)}, \qquad y_{n+1} = y_n + \tfrac h2\big(f(t_n, y_n) + f(t_{n+1}, \tilde y)\big) \quad\text{(correct)}.

Equivalently, it is one pass of the fixed-point iteration for the trapezoid equation, started from the Euler value. What survives? The predictor’s error is h22y′′\tfrac{h^2}{2}y'', and it enters the corrector multiplied by h2fy\tfrac h2 f_y, which makes an h3h^3 contribution, the same size as the trapezoid’s own error. So order 2 survives:

y(tn+1)−yn+1Heun=−h312y′′′+h34fy y′′.y(t_{n+1}) - y^{\text{Heun}}_{n+1} = -\tfrac{h^3}{12}y''' + \tfrac{h^3}{4} f_y\,y''.

(The trapezoid’s error plus the price of the predictor.) More correction passes (the “PECE” pattern, see 4.6 Multistep Methods) drag the result toward the true trapezoid value, at a rate set by h2fy\tfrac h2 f_y. What is lost is not accuracy but the implicit equation itself: the trapezoid’s amplification factor is a ratio of polynomials whose denominator tames large steps; Heun’s is a bare polynomial. The next sections show the consequence.

Midpoint versus Heun. On the test equation y′=λyy' = \lambda y they coincide (derived in the next section). Block A prints identical numbers for the two, and the sim’s max difference readout is zero to round-off. That is a statement about a linear constant-coefficient equation, no more. On a general ff they have different error constants; the same expansion gives

Heun: −112y′′′+14fyy′′,midpoint: +124y′′′+18fyy′′(coefficients of h3).\text{Heun: } -\tfrac{1}{12}y''' + \tfrac14 f_y y'', \qquad \text{midpoint: } +\tfrac{1}{24}y''' + \tfrac18 f_y y'' \qquad (\text{coefficients of } h^3).

Block C tests this, printing the one-step error over h3h^3. For y′=−yy' = -y both give −0.1661-0.1661 at h=0.0125h = 0.0125, heading to −1/6-1/6. For y′=3t2y' = 3t^2 there is no yy in the right side, hence no predictor error, and the quadrature errors show alone: Heun −0.5-0.5, midpoint +0.25+0.25 (that is −112⋅6-\tfrac{1}{12}\cdot 6 and 124⋅6\tfrac{1}{24}\cdot 6). For y′=−y2y' = -y^2 from y=1y = 1: y′′=2y'' = 2, y′′′=−6y''' = -6, fy=−2f_y = -2, so the formulas predict 0.5−1=−0.50.5 - 1 = -0.5 for Heun and −0.25−0.5=−0.75-0.25 - 0.5 = -0.75 for midpoint; the block prints −0.4877-0.4877 and −0.7377-0.7377 at h=0.0125h = 0.0125. So two second-order methods can differ in error constant by a factor 1.51.5 on a nonlinear problem. Block B shows it: the global error at T=2T = 2 and h=0.1h = 0.1 is 3.95×10−43.95\times10^{-4} for Heun, 6.23×10−46.23\times10^{-4} for midpoint and 3.71×10−43.71\times10^{-4} for the trapezoid.

The test equation and amplification factors

The standard pencil-solvable comparison is the test equation y′=λyy' = \lambda y, exact solution eλte^{\lambda t}. It is not arbitrary: near a steady state, small deviations obey the linearization δ′=fy δ\delta' = f_y\,\delta (3.8 Autonomous Equations, Equilibria & Stability), and for a system the same holds along each eigen-direction of the Jacobian. A method that mishandles y′=λyy' = \lambda y mishandles nearby nonlinear problems too. We take λ<0\lambda \lt 0 (decay).

Every method here is linear in yny_n on this equation, so one step multiplies by a number that depends only on z=hλz = h\lambda:

yn+1=R(z) yn,yn=R(z)ny0(exact: y(tn)=(ez)ny0).y_{n+1} = R(z)\,y_n, \qquad y_n = R(z)^n y_0 \quad \text{(exact: } y(t_n) = (e^{z})^n y_0\text{)}.

Deriving RR is one line per method:

  • Forward Euler: yn+1=yn+z yny_{n+1} = y_n + z\,y_n, so R=1+zR = 1 + z.
  • Backward Euler: w=yn+z ww = y_n + z\,w, so R=11−zR = \dfrac{1}{1 - z}.
  • Trapezoid: w=yn+z2(yn+w)w = y_n + \tfrac z2 (y_n + w), so R=1+z/21−z/2R = \dfrac{1 + z/2}{1 - z/2}.
  • Heun: k1=λyk_1 = \lambda y, k2=λ(y+hk1)=λy(1+z)k_2 = \lambda(y + h k_1) = \lambda y (1 + z), so yn+1=y(1+z2(1+1+z))y_{n+1} = y\big(1 + \tfrac z2 (1 + 1 + z)\big) and R=1+z+z22R = 1 + z + \tfrac{z^2}{2}.
  • Midpoint: k1=λyk_1 = \lambda y, k2=λ(y+h2k1)=λy (1+z2)k_2 = \lambda(y + \tfrac h2 k_1) = \lambda y\,(1 + \tfrac z2), so yn+1=y (1+z+z22)y_{n+1} = y\,(1 + z + \tfrac{z^2}{2}): the same RR as Heun.

Compare each with ez=1+z+z22+z36+⋯e^z = 1 + z + \tfrac{z^2}{2} + \tfrac{z^3}{6} + \cdots by expanding. Forward Euler: ez−R=z22+⋯e^z - R = \tfrac{z^2}{2} + \cdots. Backward Euler: 1/(1−z)=1+z+z2+⋯1/(1-z) = 1 + z + z^2 + \cdots, so ez−R=−z22+⋯e^z - R = -\tfrac{z^2}{2} + \cdots. Trapezoid: R=1+z+z22+z34+⋯R = 1 + z + \tfrac{z^2}{2} + \tfrac{z^3}{4} + \cdots, so ez−R=−z312+⋯e^z - R = -\tfrac{z^3}{12} + \cdots. Heun and midpoint: ez−R=z36+⋯e^z - R = \tfrac{z^3}{6} + \cdots. The first two agree with eze^z through the z1z^1 term, the others through z2z^2: a method has order pp when R(z)R(z) matches eze^z through the zpz^p term, which is the Taylor-matching statement from the last lesson in its cleanest form. (Check: on this equation fyy′′=y′′′f_y y'' = y''', so Heun’s coefficient is −112+14=16-\tfrac1{12} + \tfrac14 = \tfrac16 and midpoint’s is 124+18=16\tfrac1{24} + \tfrac18 = \tfrac16.)

Block A confirms it by halving zz from −0.02-0.02 to −0.01-0.01. The gap ez−Re^z - R shrinks by 3.993.99 (forward), 3.933.93 (backward), 7.927.92 (trapezoid), 7.987.98 (Heun and midpoint): 2p+1=4,4,8,82^{p+1} = 4, 4, 8, 8.

A first look at stability

For λ<0\lambda \lt 0 the true solution decays monotonically; the numerical one is Rny0R^n y_0. Two failures show in RR alone: ∣R∣>1\lvert R\rvert \gt 1 makes it grow, and R<0R \lt 0 makes it flip sign every step, an oscillation the exact solution lacks. Requiring ∣R(z)∣≤1\lvert R(z)\rvert \le 1 is absolute stability, a statement about one fixed hh, unlike the h→0h \to 0 stability of the last lesson. The table is block G.

z=hλz = h\lambda−0.5-0.5−1.5-1.5−2.5-2.5−10-10−100-100
forward Euler0.50000.5000−0.5000-0.5000−1.5000-1.5000−9.0000-9.0000−99.0000-99.0000
backward Euler0.66670.66670.40000.40000.28570.28570.09090.09090.00990.0099
trapezoid0.60000.60000.14290.1429−0.1111-0.1111−0.6667-0.6667−0.9608-0.9608
Heun, midpoint0.62500.62500.62500.62501.62501.625041.000041.00004901.00004901.0000
exact eze^z0.6070.6070.2230.2230.08210.08214.5×10−54.5\times10^{-5}3.7×10−443.7\times10^{-44}

Read the formulas against it.

Forward Euler. ∣1+z∣≤1\lvert 1 + z\rvert \le 1 exactly when −2≤z≤0-2 \le z \le 0, that is h≤2/∣λ∣h \le 2/\lvert\lambda\rvert. Inside that, R>0R \gt 0 for z>−1z \gt -1 (monotone decay) and R<0R \lt 0 for −2<z<−1-2 \lt z \lt -1: it decays but oscillates. At z=−2z = -2, R=−1R = -1 and the solution flips between ±1\pm1 forever; past it, it explodes. On y′=−10yy' = -10y block D prints ∣yN∣\lvert y_N\rvert at T=3T = 3 (exact: 9.4×10−149.4\times10^{-14}): 8.67×10−198.67\times10^{-19} at h=0.05h = 0.05, 9.54×10−79.54\times10^{-7} with alternating signs at h=0.15h = 0.15, exactly 1.001.00 at h=0.2h = 0.2, and 130130 at h=0.25h = 0.25.

Backward Euler. For z<0z \lt 0, 1/(1−z)1/(1-z) lies strictly between 00 and 11: monotone decay for every hh, no oscillation, no blow-up (2.96×10−72.96\times10^{-7} at h=0.25h = 0.25 in block D). It is not accurate (2.72×10−112.72\times10^{-11} at h=0.05h = 0.05 is far above e−30e^{-30}), but never wrong in kind. The catch is on another axis. For oscillating problems, λ=iω\lambda = i\omega, ∣1/(1−iωh)∣=1/1+ω2h2<1\lvert 1/(1 - i\omega h)\rvert = 1/\sqrt{1 + \omega^2h^2} \lt 1, so backward Euler damps motion that should persist. Block E takes h=2π/64h = 2\pi/64 so 64 steps is one period: backward Euler keeps 0.73570.7357 of the amplitude per period, forward Euler gains to 1.35931.3593, the trapezoid keeps 1.00001.0000 and Heun 1.00071.0007.

Trapezoid. ∣R∣<1\lvert R\rvert \lt 1 for every z<0z \lt 0, so it never blows up, and it is neutral on the imaginary axis. But as z→−∞z \to -\infty, R→−1R \to -1, not 00. For z<−2z \lt -2 the sign flips each step and the amplitude barely shrinks: at z=−100z = -100 the exact solution has collapsed by 10−4410^{-44} while the trapezoid still has R=−0.9608R = -0.9608. This is ringing, the cost of averaging both ends.

Heun and midpoint. 1+z+z22=12((z+1)2+1)≥121 + z + \tfrac{z^2}{2} = \tfrac12\big((z+1)^2 + 1\big) \ge \tfrac12, so RR never goes negative, but it is at most 11 only for −2≤z≤0-2 \le z \le 0. The real stability limit is 2, the same as forward Euler’s, despite the higher order. A degree-2 polynomial must go to infinity for large ∣z∣\lvert z\rvert; more order buys accuracy near z=0z = 0, not stability far from it. Stability regions in the complex plane are the subject of 4.5 Stability Regions & Stiffness.

◆ Interactive Euler-family lab: the same decaying equation, five methods, one step size slider

Loading simulation…

Things to try, with ∣λ∣=10\lvert\lambda\rvert = 10 as in block D (the readout’s last-step value matches it):

  1. Sweep hh upward with only forward Euler ticked: the marker zz crosses R=0R = 0 at h=0.1h = 0.1 (sign flips begin), touches the lower dashed line at h=0.2h = 0.2, then leaves [−1,1][-1, 1].
  2. At h=0.25h = 0.25 tick Backward Euler and Trapezoid: the first stays on the axis, the second alternates in sign with 3.54×10−123.54\times10^{-12} at the end.
  3. Tick Heun and Midpoint together: the max-difference readout is zero to round-off (5.3×10−235.3\times10^{-23} at h=0.05h = 0.05).
  4. Raise ∣λ∣\lvert\lambda\rvert to 2020 at h=0.15h = 0.15 (z=−3z = -3): Heun blows up, the trapezoid and backward Euler stay bounded.
✅ Concept check

Forward Euler is applied to y' = -30y with h = 0.06. What does the numerical solution do?

✅ Concept check

Heun's method has order 2 and forward Euler order 1, yet on y' = lambda y with lambda lt 0 both are stable only for h up to 2/|lambda|. What is the reason?

✅ Concept check

One backward Euler step for y' = -y^2 from y_n = 0.5 with h = 2. What is y_{n+1}? (Give the positive root.)

Exercise: implicit steps by hand

Implement backward Euler twice, in closed form for a linear equation and with Newton’s method for a general one, then run both: a stiff linear problem with λh=−100\lambda h = -100, and a nonlinear decay.

🧪 Coding exerciseExercise: backward Euler, linear closed form and Newton for nonlinear f
⌘/Ctrl + Enter

Problems

✏️ Pen & paperThree regimes for one equation●○○

Take y′=−40yy' = -40y.

  1. For forward Euler, find the ranges of hh giving (a) monotone decay, (b) decay with sign flips, (c) growth.
  2. Take h=0.5h = 0.5. Find RR for backward Euler and for the trapezoid. How many steps does each need to bring ∣y∣\lvert y\rvert below 10−3 y010^{-3}\,y_0? Compare with the exact value of eλhe^{\lambda h}.
Show worked solution
  1. z=−40hz = -40h. Monotone decay: −1<z≤0-1 \lt z \le 0, so 0<h<0.0250 \lt h \lt 0.025. Sign flips with decay: −2<z<−1-2 \lt z \lt -1, so 0.025<h<0.050.025 \lt h \lt 0.05. At h=0.05h = 0.05, R=−1R = -1 (constant size). Growth: h>0.05h \gt 0.05.
  2. h=0.5h = 0.5 gives z=−20z = -20. Backward Euler: R=1/21=0.047619R = 1/21 = 0.047619. After 1, 2, 3 steps the factor is 0.04760.0476, 2.27×10−32.27\times10^{-3}, 1.08×10−41.08\times10^{-4}, so 3 steps. Trapezoid: R=(1−10)/(1+10)=−9/11=−0.8182R = (1 - 10)/(1 + 10) = -9/11 = -0.8182; we need 0.8182n<10−30.8182^n \lt 10^{-3}, n>ln⁡(10−3)/ln⁡(9/11)=34.42n \gt \ln(10^{-3})/\ln(9/11) = 34.42, so 35 steps, with alternating sign. The exact one-step factor is e−20=2.06×10−9e^{-20} = 2.06\times10^{-9}: both are bounded, neither is accurate, and the trapezoid rings.
✏️ Pen & paperA whole family from one parameter●●○

The θ\theta-method is yn+1=yn+h[(1−θ)fn+θfn+1]y_{n+1} = y_n + h\big[(1 - \theta) f_n + \theta f_{n+1}\big] with 0≤θ≤10 \le \theta \le 1.

  1. Show that θ=0\theta = 0, 11, 12\tfrac12 give forward Euler, backward Euler and the trapezoid, and find R(z)R(z) in general.
  2. Expand RR to order z2z^2 and show that only θ=12\theta = \tfrac12 gives order 2.
  3. Show that R(z)→−(1−θ)/θR(z) \to -(1-\theta)/\theta as z→−∞z \to -\infty and that ∣R∣≤1\lvert R\rvert \le 1 for all z<0z \lt 0 exactly when θ≥12\theta \ge \tfrac12. For θ<12\theta \lt \tfrac12 find the stability limit. Evaluate R(−10)R(-10) for θ=0.75\theta = 0.75 and 0.250.25.
Show worked solution
  1. On the test equation w=y+(1−θ)zy+θzww = y + (1-\theta) z y + \theta z w, so R(z)=1+(1−θ)z1−θzR(z) = \dfrac{1 + (1-\theta) z}{1 - \theta z}. Then θ=0\theta = 0 gives 1+z1 + z, θ=1\theta = 1 gives 1/(1−z)1/(1-z), and θ=12\theta = \tfrac12 gives (1+z/2)/(1−z/2)(1 + z/2)/(1 - z/2).
  2. 11−θz=1+θz+θ2z2+⋯\dfrac{1}{1-\theta z} = 1 + \theta z + \theta^2 z^2 + \cdots, so R=1+[θ+(1−θ)]z+[θ2+(1−θ)θ]z2+⋯=1+z+θz2+⋯R = 1 + \big[\theta + (1-\theta)\big] z + \big[\theta^2 + (1-\theta)\theta\big] z^2 + \cdots = 1 + z + \theta z^2 + \cdots. This matches ez=1+z+12z2e^z = 1 + z + \tfrac12 z^2 only if θ=12\theta = \tfrac12.
  3. Dividing by zz: R→−(1−θ)/θR \to -(1-\theta)/\theta, of magnitude at most 11 iff θ≥12\theta \ge \tfrac12. For θ<12\theta \lt \tfrac12, RR reaches −1-1 when 1+(1−θ)z=−(1−θz)1 + (1-\theta) z = -(1 - \theta z), i.e. 2+(1−2θ)z=02 + (1-2\theta) z = 0: the limit is z=−2/(1−2θ)z = -2/(1-2\theta) (−2-2 for θ=0\theta = 0, −4-4 for θ=0.25\theta = 0.25). Values: θ=0.75\theta = 0.75: R(−10)=(1−2.5)/(1+7.5)=−0.1765R(-10) = (1 - 2.5)/(1 + 7.5) = -0.1765; θ=0.25\theta = 0.25: R(−10)=(1−7.5)/(1+2.5)=−1.857R(-10) = (1 - 7.5)/(1 + 2.5) = -1.857, outside the limit −4-4 and unstable.
✏️ Pen & paperRinging against over-damping on a stiff mode●●●

A mode decays as y′=−500yy' = -500y, and you step it with h=0.1h = 0.1.

  1. Find zz, and RR for backward Euler and for the trapezoid. Find the limit of each RR as z→−∞z \to -\infty.
  2. How many steps does each take to bring the amplitude below 10−310^{-3} of its start? What does the exact solution do in one step?
  3. Which would you use if you only care about the slow modes of a system that also contains this fast one? How could you keep the trapezoid’s accuracy without its ringing?
Show worked solution
  1. z=−50z = -50. Backward Euler: R=1/51=0.019608R = 1/51 = 0.019608, with limit 00 as z→−∞z \to -\infty. Trapezoid: R=(1−25)/(1+25)=−24/26=−0.923077R = (1 - 25)/(1 + 25) = -24/26 = -0.923077, with limit −1-1.
  2. Backward Euler: 0.01960.0196 after one step, 3.84×10−43.84\times10^{-4} after two, so 2 steps. Trapezoid: n>ln⁡(10−3)/ln⁡(24/26)=86.30n \gt \ln(10^{-3})/\ln(24/26) = 86.30, so 87 steps, alternating in sign. The exact solution multiplies by e−50=1.93×10−22e^{-50} = 1.93\times10^{-22} in one step.
  3. Backward Euler: it kills the fast mode within a step or two and leaves the slow modes alone. The trapezoid leaves an alternating residue of the fast mode, visible as noise on a smooth solution, for about 8787 steps. To keep second order without ringing, use a method with R→0R \to 0 at −∞-\infty (see 4.5 Stability Regions & Stiffness), or take a few backward Euler steps first to damp the mode.

What comes next

4.3 Runge–Kutta Methods & Butcher Tableaux makes multi-slope sampling systematic, 4.4 Adaptive Step Size & RK45 chooses hh automatically, and 4.5 Stability Regions & Stiffness redraws stability in the complex plane.

Key Takeaways

  • Each method is a quadrature rule for ∫f dt\int f\,dt: left rectangle (forward Euler), right rectangle (backward Euler), trapezoid, midpoint with an Euler half step. Rules exact for lines give order 2, for constants order 1.
  • Implicit methods solve an equation per step: closed form if ff is linear, Newton with G′=1−hfyG' = 1 - h f_y otherwise. Fixed-point iteration needs ∣hfy∣<1\lvert h f_y\rvert \lt 1, which brings back the step restriction.
  • Heun is the trapezoid with an Euler predictor: order 2 survives, the implicit equation (and the denominator in RR) does not.
  • On y′=λyy' = \lambda y one step multiplies by R(z)R(z), z=hλz = h\lambda: 1+z1+z, 1/(1−z)1/(1-z), (1+z/2)/(1−z/2)(1+z/2)/(1-z/2), and 1+z+z2/21+z+z^2/2 for both Heun and midpoint. Order pp means RR matches eze^z through zpz^p.
  • Heun and midpoint agree on the test equation but not on nonlinear ff: for y′=−y2y' = -y^2 their error constants are −0.5-0.5 and −0.75-0.75 (units of h3h^3).
  • Forward Euler, Heun and midpoint are stable on the real axis only for h≤2/∣λ∣h \le 2/\lvert\lambda\rvert; forward Euler oscillates for h∣λ∣>1h\lvert\lambda\rvert \gt 1.
  • Backward Euler is stable for every hh but over-damps oscillations; the trapezoid is neutral on the imaginary axis but rings (R→−1R \to -1) on stiff modes.
  • Diffusion on a fine grid is a set of very fast decays: hence implicit diffusion in fluid codes.
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.

Where this leads

These later lessons build directly on this one: