Why this matters for simulation: Exponential integrators solve the stiff linear part of a fluid equation exactly.
By the end of this lesson you can…
Build a fundamental matrix $\Phi(t)$ from independent solutions and turn it into the flow map $e^{At} = \Phi(t)\Phi(0)^{-1}$.
State and check $e^{A(t+s)} = e^{At}e^{As}$, $(e^{At})^{-1} = e^{-At}$ and $\tfrac{d}{dt}e^{At} = Ae^{At}$, and give a $2\times2$ example where $e^{(A+B)t} \neq e^{At}e^{Bt}$.
Compute $e^{At}$ for a $2\times2$ matrix by diagonalization, by the nilpotent series, by a Jordan block (deriving the $t$ in the corner) and by the Cayley–Hamilton closed form.
Show that $e^{[[0,-\omega],[\omega,0]]t}$ is a rotation by $\omega t$, and use $\det e^{At} = e^{\operatorname{tr}(A)t}$ to say how a flow changes areas.
Predict when a truncated series or an eigendecomposition loses accuracy, and say what `scipy.linalg.expm` does instead.
Write the exponential Euler step and explain why it takes steps at which explicit Euler and RK4 blow up on a stiff linear part.
7.4 Complex & Repeated Eigenvalues ended with a promise: for a defective matrix the solution carries a factor t, and the claim eJt=eλt(10t1) would be derived later. This is later. The derivation is a special case of something bigger. Every case of 7.2 Linear Systems & the Eigenvalue Method and 7.3 Phase Portraits & the Trace–Determinant Plane is one formula, x(t)=eAtx0, and eAt is a single matrix that moves every starting point at once. 1.8 The Matrix Exponential: From Hops to Flows defined it and diagonalized it; here we make it the working tool: three ways to compute it, its determinant, how it fails numerically, and the one idea from it that fluid and flight codes use directly, the exponential integrator.
▶ VideoA blob of points carried by e^(At): the series, three routes to compute it, the area law, and a stiff step that is exactcaptions only
Transcript
Every solution is x of t equals e to the A t times x zero. Solutions side by side form a fundamental matrix phi, and e to the A t is phi of t times phi of zero inverse: the flow map, moving a whole blob at once. At t equals 1: 0.60042, 0.23254, minus 0.46509, minus 0.09721.
Flows compose, but the exponential of a sum is the product only for commuting matrices. For P with a one above the diagonal and Q with a one below, e to the P times e to the Q is 2, 1, 1, 1, yet e to the P plus Q differs by 0.54308.
The series defines it. For the rotation generator at t equals 1, the top left entry after 2, 4, 6, 8 terms is 0.5, 0.54167, 0.54028, 0.54030, near cosine 1. But the terms needed grow with t: 14, 29, 44, 72, 127 for t equals 1, 5, 10, 20, 40.
Three routes. If A is V lambda V inverse, the series sums to V e to the lambda t V inverse. If N squared is zero it stops: e to the N t is I plus N t, here 2, minus 2, 0.5, 0 at t equals 0.5, a shear.
A defective matrix is lambda I plus N, with N squared zero, and lambda I commutes with everything. So e to the A t is e to the lambda t times I plus N t: the t in the corner is the first term. The earlier example at 0.5: 0.18394, 0.18394, minus 0.18394, 0.55182.
For any two by two, Cayley-Hamilton gives a closed form with cosh, cos or t. For the rotation generator it is a rotation by omega t: Euler's formula, with J playing i. At t equals 1.5, that is 0.07074, minus 0.99749. The spiral adds a factor e to the minus 0.2 t.
Liouville: the determinant of e to the A t is e to the trace times t, since the determinant of phi has derivative T times itself. So the flow scales areas by e to the T t: the saddle by 2.71828, the spiral 0.67032, the node 0.04979.
The incompressible saddle has trace zero. At t equals 1 the disk becomes an ellipse with singular values 9.35647 and 0.10688, yet the area ratio is 1.00000. At t equals 4 it is a sliver 7663.87 long, area still 1. Euler's step gains area, 1.04.
For diagonal minus 20, minus 1, the true first entry is 2.0612 times 10 to the minus 9, but 150 terms give 5.6219 times 10 to the minus 9: the largest term is 4.3 times 10 to the 7, so rounding costs about 4 times 10 to the minus 9. expm is right.
The eigenvector route fails differently. For this nearly defective matrix, V has condition number 2 times 10 to the 12, so V e to the lambda V inverse is off by 2 times 10 to the minus 5 on an entry of 3678.8; expm by 9 times 10 to the minus 13.
Now the exponential integrator. For x prime equals A x plus N, freeze N over a step. The exact solution is e to the A h times x, plus A inverse times e to the A h minus I times N. The linear part is exact, so only N limits h.
A stiff test, diagonal minus 100, minus 1. Euler is stable only below h of 0.02, RK4 below 0.027853. At h equals 0.1 the errors are 3.5 times 10 to the 9 and 4.4 times 10 to the 24; exponential Euler's is 4 times 10 to the minus 16, but only first order.
In fluid codes diffusion is diagonal in Fourier modes, so e to the lambda k h is cheap; from memory, ETD methods use this. Stam instead treats diffusion implicitly. An exact incompressible flow keeps area, and a solver should try to. Next: nonhomogeneous systems.
Fundamental matrices and the flow map
Take A=(0−21−3). Its eigenpairs are −1 with (1,−1) and −2 with (1,−2) (check: A(1,−2)=(−2,4)). Put the two independent solutions side by side as columns:
Φ(t)=(e−t−e−te−2t−2e−2t).
This is a fundamental matrix: each column solves x′=Ax, so Φ′=AΦ, and the columns are independent at every t (they are at t=0, and by uniqueness stay so). Any solution is Φ(t)c, and matching x0 at t=0 forces c=Φ(0)−1x0. So
x(t)=eAtx0,eAt=Φ(t)Φ(0)−1.
Here Φ(0)−1=(2−11−1), giving eAt=(2e−t−e−2t−2e−t+2e−2te−t−e−2t−e−t+2e−2t), which at t=1 is (0.60042−0.465090.23254−0.09721) (block A matches it to expm). The matrix is independent of which fundamental matrix you started from: ΦC is another one and C cancels. It is the solution operator or flow map: feed it any point and it returns where that point is at time t. It is exactly the object a time-stepper approximates, one step at a time, with t=h.
Properties, each from the equation Φ′=AΦ or from uniqueness:
dtdeAt=AeAt (eAt is a fundamental matrix with value I at 0).
eA(t+s)=eAteAs: flowing for s then for t is flowing for t+s (both sides solve the equation in t with the same value at t=0). In particular (eAt)−1=e−At, so the flow can be run backwards and eAt is never singular.
Block A confirms them for this A: the semigroup gap and eAe−A−I are at rounding level (about 10−14), and the derivative check differs by 1.6×10−10, the finite-difference error.
e(A+B)t=eAteBt needs AB=BA.
The honest caveat. Take P=(0010) and Q=(0100), with PQ−QP=diag(1,−1)=0. Then eP=(1011), eQ=(1101), so ePeQ=(2111), while P+Q=(0110) has eigenvalues ±1 and eP+Q=(cosh1sinh1sinh1cosh1)=(1.543081.175201.175201.54308). The largest entry gap is 0.54308 (block A). This is the splitting error of 1.8 The Matrix Exponential: From Hops to Flows, made concrete.
The series, and how many terms it needs
Define eM=∑k≥0Mk/k! with M=At. For the rotation generator J=(01−10) at t=1 the answer is (cos1sin1−sin1cos1) with cos1=0.54030. The (1,1) entry of the partial sum after K=2,4,6,8 terms is 0.5,0.54167,0.54028,0.54030, with largest entry error 0.159,8.1×10−3,2.0×10−4,2.7×10−6 (block B): fast, because the terms 1/k! collapse. The collapse starts only after k≈∥Mt∥, because tk/k! first grows until k=t. Block B counts the terms until tk/k!<10−10 for Jt with ∥Jt∥=t:
t
1
5
10
20
40
terms needed
14
29
44
72
127
largest term
1
26.04
2755.7
4.3×107
1.5×1016
error of the sum
7.6×10−13
3.4×10−12
7.7×10−12
6.9×10−9
2.9
The cost grows linearly in ∥At∥, but the accuracy does not survive. Every term is rounded to about 10−16 of its own size, and the sum is a difference of huge terms of both signs, so the error is about 10−16 times the largest term: at t=40 the largest is 1.5×1016 and the “answer” is wrong by 2.9, on a rotation whose entries are at most 1. The series is a definition and a proof tool, not an algorithm for large ∥At∥. The sim’s readout shows the same effect live: the gap between its two series-based routes grows as you raise t.
Three routes by hand
Route 1: diagonalizable. If A=VΛV−1 then Ak=VΛkV−1 and the series sums to eAt=VeΛtV−1, with eΛt=diag(eλit). For the matrix above V=(1−11−2), which is Φ(0), so this is the formula Φ(t)Φ(0)−1: block C reproduces eAt at t=1 to rounding.
Route 2: nilpotent. If N2=0 the series stops: eNt=I+Nt. Example: N=(21−4−2) has N2=(4−42−2−8+8−4+4)=0, and e0.5N=I+0.5N=(20.5−20) (block C).
Route 3: Jordan block. A defective A with double eigenvalue λ is λI+N with N=A−λI, and Cayley–Hamilton gives N2=0 (as in the previous lesson). The matrix λI commutes with everything, in particular with N, so the caveat above does not bite: e(λI+N)t=eλIteNt, and the series for eλIt is eλtI. Hence
eAt=eλt(I+Nt),for J=(λ01λ):eJt=eλt(10t1).
There is the t in the corner: it is the first term of the nilpotent series, nothing more. For the previous lesson’s A=(−3−11−1), λ=−2 and N=A+2I=(−1−111), so eAt=e−2t(1−t−tt1+t). At t=0.5 this is e−1(0.5−0.50.51.5)=(0.18394−0.183940.183940.55182) (block C; the sim’s defective preset shows the same entries), and its second column applied to (0,1) is e−2t(t,1+t), the solution found there.
Route 4: a closed form for every 2×2. Put m=T/2 and B=A−mI. Then trB=0, so Cayley–Hamilton for B reads B2=−detBI=qI with q=m2−D (using detB=D−m2). Even powers of B are multiples of I, odd powers multiples of B, and summing the series for eBt gives
Real distinct, complex and repeated eigenvalues are the three lines; q=0 is the nilpotent route (with N=B), and the star node A=λI has B=0. Block C checks the formula against expm at t=1.3 for six matrices, including all three cases, and the largest gap is below 10−12. This is the formula the exercise below asks you to code.
The rotation generator and Euler’s formula
For Jω=(0ω−ω0): T=0, D=ω2, so m=0, q=−ω2 and the closed form gives
Directly from the series: Jω2=−ω2I, so the even terms give cos and the odd terms sin, which is Euler’s formula eiθ=cosθ+isinθ with i played by J1 (J12=−I). This is the center of 7.3 Phase Portraits & the Trace–Determinant Plane: closed circles, speed ω. For ω=1, t=1.5 block D prints (0.070740.99749−0.997490.07074) and the sim’s center preset shows the same matrix. The spiral (−0.21−1−0.2)=−0.2I+J1 is e−0.2t times that rotation (a scalar commutes), which at t=1 has entries 0.44236 and 0.68894 in the sim.
Liouville: what the flow does to area
Take w(t)=detΦ(t) with columns x1,x2. The derivative of a determinant is the sum of the determinants with one column differentiated: w′=det[Ax1,x2]+det[x1,Ax2]. For 2×2 matrices this sum equals (a11+a22)det[x1x2] (expand with x1=(p,q), x2=(r,s): the terms in b and c cancel and a and d each multiply ps−qr). So w′=Tw and, with Φ(0)=I,
deteAt=eTt,T=trA.
(For a diagonalizable A it is also ∏eλit=e(∑λi)t; the argument above is the general one, and the n×n version is Liouville’s formula, from memory.) Geometrically, eAt maps a region of area A to one of area eTtA: a flow with T=0 preserves area, however much it stretches. That is the incompressibility of 2.7 Divergence: Sources & Sinks: T=∇⋅u for a linear velocity field, the T=0 line of the phase-portrait plane. The sim carries a unit disk to its image, computes the area of the image with the shoelace formula, and compares it with deteAt and with eTt (a linear map scales every polygon’s area by exactly det, so the three agree to rounding).
◆ InteractiveFlow a blob by e^(At): matrix presets, a t slider, area against e^(Tt), singular values and three routes to e^(At)
Loading simulation…
What to try. (1) Incompressible saddle(122−1) at t=1: the disk becomes an ellipse with singular values 9.35647 and 0.10688 (e±5, since A is symmetric) and the area ratio is 1.00000. At t=4 the lengths are 7663.87 and 0.00013, yet the area ratio is still 1.00000: a thin sliver with the same area, which is how a strained flow smears a blob of dye into a filament. (2) Saddle(1210), T=1: at t=1 the ratio is 2.71828=e, with singular values 7.75164 and 0.35067. (3) Spiral (T=−0.4): ratio 0.67032 at t=1, singular values both 0.81873 (a rotation, no shape change). (4) Shear(0010): area 1.00000, singular values 1.61803 and 0.61803, a shear that is incompressible and not a rotation. (5) Node(−201−1), T=−3: ratio 0.04979=e−3.
A link to the time-steppers. The truncations of ehA do not respect the area law exactly. For the center (0−410) at h=0.1 block D prints det(I+hA)=1.04 for Euler (the area gain seen in 7.3 Phase Portraits & the Trace–Determinant Plane), 0.99999912 for the RK4 step (the Taylor polynomial of degree 4) and 1 for ehA. RK4 loses 9×10−7 of area per step, which is why it is so good and still not exactly conservative.
Computing it well
scipy.linalg.expm uses a Padé approximant combined with scaling and squaring (from memory, following Higham and Al-Mohy–Higham): choose s so that ∥At∥/2s is small, approximate eAt/2s by a rational function, and square the result s times. The sim does a simple version (halve until the norm is below 0.5, a 16-term Taylor series, square back). Scaling avoids the huge terms of the table: eM=(eM/2)2 needs a series for a small matrix only.
Where the plain series fails. Take M=diag(−20,−1), so e11M=e−20=2.0612×10−9. The series with 150 terms gives 5.6219×10−9, wrong by a factor 2.7, because its largest term is 2020/20!=4.3×107 and rounding of 10−16 of that is 4×10−9. For diag(−30,−1) it returns −3.07×10−5 for a quantity that is 9.4×10−14: negative and a billion times too large. expm gets both right (block E). Neither error is large in absolute terms, but for a stiff decaying mode they are the wrong sign or size.
Where the eigenvector route fails. Take A=(−10104−1+10−8), eigenvalues −1 and −1+10−8, a nearly defective non-normal matrix. The eigenvector matrix has condition number about 2×1012, and VeΛV−1 is off by 2.0×10−5 on an entry of size 3678.8 (exact value 104e−1(e10−8−1)/10−8, block E), while expm is off by 9×10−13. About eight of sixteen digits are gone (platform dependent), the same ill-conditioning the previous lesson measured as eigenvalues merge. For non-normal matrices, prefer expm.
Exponential integrators
For x′=Ax+N(x) with a stiff linear part A, freeze N at its current value Nn=N(xn) over one step. The solution of x′=Ax+Nn with x(0)=xn is
xn+1=eAhxn+A−1(eAh−I)Nn,
the exponential Euler step (check: Axn+1+Nn=AeAhxn+eAhNn=dhdxn+1). The linear part is advanced exactly for any h, so h is limited by how fast N changes, not by the fastest eigenvalue. The general formula for a time-varying forcing is Duhamel’s, in 7.6 Nonhomogeneous Linear Systems.
A stiff test.A=diag(−100,−1), constant N=(100,2), x(0)=0, exact solution (1−e−100t,2(1−e−t)), solved to t=1. Explicit Euler is stable only for h<2/100=0.02 and RK4 for h<2.7853/100=0.027853 (4.5 Stability Regions & Stiffness). Block F gives the largest error at t=1:
h
explicit Euler
RK4
exponential Euler
0.1
3.5×109
4.4×1024
4×10−16
1/35=0.02857
2.6×109
43.5
0
1/36=0.02778
9.9×108
0.66
1.6×10−15
Just above RK4’s limit (row 2) the fast mode grows; just below (row 3) it decays, slowly (0.66 is the damped remnant of an error that started at 1). Exponential Euler is exact to rounding at all three, because the forcing is constant. With a state-dependent N=(100,−x22) and x2(0)=1 it is only first order: its error at t=1 is 1.45×10−2,7.0×10−3,3.4×10−3,1.7×10−3 for h=0.1,0.05,0.025,0.0125 (ratios about 2), while RK4 diverges at h=0.1 and 0.05 (4.4×1024, 5.5×1022) but reaches 3.0×10−8 at h=0.025. So exponential Euler wins on stability, not on accuracy per step; higher-order exponential methods exist (from memory).
Computing it
🐍 PythonLab: properties, the series, three routes, rotation and determinant, where the series and eig fail, and the stiff test (blocks A to F)
⌘/Ctrl + Enter
Now build the closed form yourself and check it against the series-free expm. Everything graded is real.
🧪 Coding exerciseExercise: e^(At) for a 2x2 matrix in closed form, and the area of a flowed blob✓ passed
⌘/Ctrl + Enter
💡 Branch on q. For a repeated eigenvalue (q near 0) the formula is the nilpotent route. In blob_area_ratio, np.column_stack of cos and sin gives the vertices, and the image is points @ E.T.
Concept checks
✅ Concept check
$J = \begin{pmatrix}-2&1\\0&-2\end{pmatrix}$. What is the upper-right entry of $e^{Jt}$ at $t = 0.5$? (Give a number.)
Why: e^(Jt) = e^(-2t)(I + Nt), so the corner is t e^(-2t) = 0.5 e^(-1) = 0.18394, the same entry that appears in the defective example (block C and the sim).
✅ Concept check
A flow $\mathbf x' = A\mathbf x$ has $\operatorname{tr}A = -3$. By what factor does a blob's area change in time $t = 1$? (Give a number.)
Why: Area is multiplied by det e^(At) = e^(Tt) = e^(-3) = 0.04979, whatever the eigenvectors are. The sim's node preset shows the same ratio.
✅ Concept check
Why does summing the series for $e^{At}$ with $A = \mathrm{diag}(-30, -1)$, $t = 1$, return a wildly wrong value for the first entry?
Why: Convergence is guaranteed; accuracy is not. Scaling and squaring (expm) avoids the large terms.
Problems
✏️ Pen & paperA repeated eigenvalue●●○
Compute eAt for A=(31−11).
Show worked solution
T=4, D=3+1=4, so λ=2 twice. N=A−2I=(11−1−1)=0 and N2=(1−11−1−1+1−1+1)=0. Because 2I commutes with N, eAt=e2t(I+Nt)=e2t(1+tt−t1−t). Check: at t=0 it is I, and the derivative at 0 is 2I+N=A. (Hand computation; it is the matrix of the repeated-eigenvalue branch of the exercise.)
✏️ Pen & paperArea per turn of a spiral●●○
For A=(−0.53−3−0.5) find eAt, the period of the rotation, and the factors by which lengths and areas shrink per turn.
Show worked solution
A=−0.5I+3J1 with J1=(01−10), and the two commute, so eAt=e−0.5t(cos3tsin3t−sin3tcos3t). One turn takes t=2π/3=2.0944. The length factor is e−0.5⋅2.0944=0.35092. Here T=−1, so the area factor is e−2.0944=0.12316, which is the square of the length factor 0.35092, as it must be for a rotation-scaling. (Hand computation; not produced by the sim.)
✏️ Pen & paperThe oscillator, area and Euler●●●
(a) For A=(0−ω210) derive eAt and show deteAt=1. (b) Show that Euler’s matrix I+hA has determinant 1+h2ω2 for every h, and say what this does to a ring of initial conditions.
Show worked solution
(a) T=0, D=ω2, so m=0, q=−ω2 and eAt=cos(ωt)I+ωsinωtA=(cosωt−ωsinωtsin(ωt)/ωcosωt). Its determinant is cos2ωt+sin2ωt=1, as the area law says for T=0. (b) det(I+hA)=1⋅1−h⋅(−ω2h)=1+h2ω2. Each Euler step multiplies areas by this factor, so a ring of states is inflated step by step: the numerical flow is not area preserving, in contrast with the exact one (for ω=2, h=0.1 this is the 1.04 of block D).
Key Takeaways
A fundamental matrix Φ(t) has independent solutions as columns and Φ′=AΦ; eAt=Φ(t)Φ(0)−1 is the flow map x0↦x(t), with eA(t+s)=eAteAs, (eAt)−1=e−At and dtdeAt=AeAt.
e(A+B)t=eAteBt only if AB=BA: for P=(0010), Q=(0100) the largest entry gap is 0.54308.
Routes: VeΛtV−1; for N2=0, eNt=I+Nt; for a Jordan block eJt=eλt(I+Nt), because λI commutes with N, which derives the t in the corner; and for any 2×2, eAt=emt[cI+sB] from Cayley–Hamilton.
eJωt is rotation by ωt (Euler’s formula, from Jω2=−ω2I); the spiral is that rotation times eαt.
deteAt=etr(A)t because (detΦ)′=TdetΦ; T=0 flows preserve area (1.00000 even when the blob is stretched to a 7663.87-long sliver), and Euler’s det(I+hA)=1.04 does not.
The series needs about ∥At∥ terms and loses accuracy through cancellation (error 2.9 at t=40 for a rotation); the eigenvector route loses digits for nearly defective matrices (2.0×10−5 error); expm (Padé with scaling and squaring, from memory) avoids both.
Exponential Euler, xn+1=eAhxn+A−1(eAh−I)Nn, is exact for the linear part at any step: on the stiff test it is exact where Euler and RK4 blow up (h=0.1), but it is only first order in the nonlinear part. Next, forcing in general: 7.6 Nonhomogeneous Linear Systems.
Finished this lesson?
Mark it complete to track your progress (saved in this browser). Reading every section does this for you.