In 7.1 Systems & State Space you wrote down coupled systems and integrated them numerically; the only exact answers came from tricks. For the linear system there is a complete method, and it rests on one geometric fact from 1.5 Eigenvectors & Eigenvalues: a matrix has special lines it only stretches. The flow of cannot turn a state off such a line. Finding those lines solves the whole system, and the stretch factors say whether each direction grows or decays. This is also how a simulator’s linearised dynamics are analysed, so we finish with the link to fluids and with the numerical question that decides how small an RK4 step must be.
Only real, distinct eigenvalues appear in this lesson. Complex and repeated eigenvalues are the job of 7.4 Complex & Repeated Eigenvalues, and the classification of all the pictures is 7.3 Phase Portraits & the Trace–Determinant Plane.
Transcript
- For x prime equals A x, try the scalar solution's shape: x equals e to the lambda t times a constant vector v. The sides become lambda e to the lambda t v and e to the lambda t A v. Divide out the exponential: A v equals lambda v.
- Take A with rows 1, 2 and 3, 2. Trace 3 and determinant minus 4 give lambda minus 4 times lambda plus 1, so the eigenvalues are 4 and minus 1, with eigenvectors 2, 3 and 1, minus 1. Check: A times 2, 3 is 8, 12, which is 4 times 2, 3.
- On an eigenline, A x equals lambda x is parallel to x, so the flow cannot turn a state off the line. Start at 2, 3 on the lambda 4 line: x of 1 is e to the fourth times 2, 3, namely 109.196, 163.794, ratio 1.5, the slope of the line.
- Any combination of the two solutions solves the system. To reach every start, the eigenvectors must be independent. The Wronskian is e to the lambda one plus lambda two, times t, times det V. Here det V is minus 5, so W is minus 5 e to the 3 t, never zero.
- To fit x zero equals 1, 0, solve V c equals x zero, giving c equals 0.2, 0.6. Then x at 0.5 is 3.3195, 4.0695 and x at 1 is 22.0600, 32.5382. The lambda 4 term soon dwarfs the other, pulling the solution onto the unstable line.
- Now A with rows minus 1, 100 and 0, minus 2. Its eigenvectors, 1, 0 and 100, minus 1, are almost parallel. The condition number is 200.0 and c over x zero is 141.42. Both eigenvalues are negative, yet x one peaks at 25.0.
- Change to eigen-coordinates: x equals V y. Then y prime equals capital lambda y, with no shared variable, so each equation is scalar: y i equals c i e to the lambda i t. Here V inverse A V is diag 4, minus 1. All eigenvalues negative: every start decays.
- A second-order equation is a system for y and y prime. The determinant of C minus lambda I gives a lambda squared plus b lambda plus c, so the old roots are the eigenvalues. For 2 y double prime plus 7 y prime plus 3 y that is minus 3 and minus 0.5; y of 2 is 0.733280.
- At t equals 1 the eigen-solution matches expm to 1.7 times 10 to the minus 11: rounding. RK4 with h 0.1 is off by 2.0 times 10 to the minus 2, with h 0.01 by 2.7 times 10 to the minus 6: a ratio of 7.4 times 10 to the 3, near fourth order.
- Stiffness: eigenvalues minus 1 and minus 100. RK4 multiplies each decoupled mode by R of z, the degree-4 Taylor polynomial of e to the z, with z equal h lambda. The size of R reaches 1 at z equals minus 2.7853, so h must stay below 0.027853.
- Run to t equals 10. At h 0.025 and 0.0275, RK4 gives the exact 3.210 times 10 to the minus 5. At 0.0278 a trace of the fast mode survives, at 0.0279 it reaches 53.02, and at 0.03 it explodes to 3.4 times 10 to the 46.
- Linear stability of a flow is the same problem: perturb a base state and x prime equals A x, with A the Jacobian. A mode grows if the real part of lambda is positive. An inverted pendulum turns 10 to the minus 6 into 1.1013 times 10 to the minus 2 by t equals 10. From memory, Orr-Sommerfeld.
- To sum up. Trying e to the lambda t v gives A v equals lambda v. Eigenlines are never left, and V c equals x zero fits any start, though a large condition number warns of cancellation. Decoupling gives y prime equals capital lambda y. The fastest eigenvalue sets RK4's step. Next: complex and repeated eigenvalues.
Trying
For the solution is . For a system, try the same shape with a constant vector : . The left side is and the right side is . Since is never zero, divide it out:
So is a solution exactly when is an eigenpair, and must satisfy . (The condition rules out the useless solution .) Calculus has turned into algebra, the same move as in 5.2 Constant Coefficients & the Characteristic Equation, and it is no coincidence; see the companion-matrix section below.
Worked example. Take . The trace is and the determinant is , so , giving , . For : , whose rows say , so . For : gives . Check one directly: . Block A of the lab cell checks both and gets a residual .
Straight-line solutions
Start the system exactly on an eigenvector, . Then solves the ODE and matches , so by uniqueness (7.1 Systems & State Space) it is the solution. It is always a multiple of : the state never leaves the line through . The scalar only slides it along the line. If the point moves toward the origin and never reaches it; if it moves away. Because it also never crosses the origin to the other side.
Superposition: the general solution
Eigenvector solutions are special starts. Every other start is a mixture, and linearity does the mixing. Let . Differentiate term by term and use :
Any combination solves the ODE for any constants. To reach every starting point we need two independent solutions, which for two vectors means . (Eigenvectors for distinct eigenvalues are always independent, from 1.5 Eigenvectors & Eigenvalues.) This is the Wronskian idea of 5.1 Second-Order Linear Equations: Structure in vector form: , which is zero for no unless . For our example , so ; block A prints . Also is the trace of , so : Abel’s formula again.
Fitting initial conditions:
Put in at : , that is
This is a linear solve, the Gaussian elimination you already know, and are the coordinates of in the eigenbasis. For the example with : and give , , and
At this is and at it is . The saddle preset of the sim starts here: its readout shows and, scrubbed to and , those two states. The big arrow is the component and it soon dwarfs the other: the solution is pulled onto the unstable line.
Try the two negative preset with the default start : , , , and . The component has half-life , the component . By the state is and the fast coordinate has shrunk from to , a factor , while has only gone from to . The fast mode dies first, so every curve bends onto the slow eigenline before heading in.
When the eigenbasis is nearly degenerate
The numbers are only as good as the solve. Take : eigenvalues and , eigenvectors and , almost parallel. For the solve gives : two big terms cancelling to a unit vector. With unit-length eigenvector columns, block B measures and , so a relative error in may reach about in (the standard perturbation bound, from memory; the 141 is measured here, the 200 is its ceiling). Both eigenvalues are negative, yet peaks at at before decaying, a transient of 25 times the starting size. Eigenvalues alone do not tell you the transient. The last preset of the sim shows it: large and the sum small.
Decoupling
Change coordinates to the eigenbasis: (so , the coordinates had at ). Substituting, , and multiplying by ,
This is of 1.6 Change of Basis & Diagonalization (there and ), now put to work. The two equations and share no variable, so each is a scalar problem with the answer . The middle panel of the sim plots exactly those; tick log scale and each becomes a straight line of slope . Block A prints . Decoupling is the whole method: a coupled problem that is a diagonal problem in the right coordinates.
Growth, decay, and a preview of stability
Each mode follows . If every then every mode decays and from every start: the origin is stable. The sim reports a half-life per mode. If some then a generic start grows without bound, and only the starts with for that mode escape it (in the saddle preset, the line along ). We connect and do not yet classify: 7.3 Phase Portraits & the Trace–Determinant Plane names these pictures (nodes, saddles) and handles complex eigenvalues, where oscillation enters. The transient above is a warning that decay in the long run is not the same as small along the way.
The saddle shows the knife edge. A start exactly on the line decays like , but any error in the start, however tiny, puts a small on the mode, and that mode then outgrows the decaying one by the factor . A numerical method has such errors at every step (rounding, plus truncation error that is not parallel to the line), so a simulation cannot hold a trajectory on an unstable manifold for long. A rounding error of would be overtaken after roughly time units (a back-of-envelope estimate, not run in the sim). In the sim, a start dragged close to that line snaps onto it, so the exact curve has (the readout shows it) while RK4 has no such guarantee.
Connection to the second-order lesson
Write as a system with :
Then , which is the characteristic equation of 5.2 Constant Coefficients & the Characteristic Equation, so the roots are the eigenvalues. An eigenvector is : , because solves the quadratic. That is the vector of for , so the first component of the eigen-solution is the old solution. Verify with that lesson’s example , , : block C finds the eigenvalues and , equal to the roots of . Solving gives (check: ), so , and block C reproduces from the eigen-solution and from RK4 with steps.
Numerics: eigen-solution, expm and RK4
Fix for the saddle example. The eigen-solution gives in exact arithmetic. Block A compares it with scipy’s expm(A t) @ x0 (the matrix exponential, the subject of 7.5 The Matrix Exponential & Fundamental Matrices) and with RK4. The gap to expm is , which is rounding and no more. RK4 (4.3 Runge–Kutta Methods & Butcher Tableaux) with is off by , and with by : a ratio of for a tenfold smaller step, near the of a fourth-order method. At the gaps are and . The sim’s right panel draws the RK4 error and the readout gives the gap at the scrub time.
Stiffness preview: why the fastest eigenvalue sets the step
Let . Its eigenvalues are (vector ) and (vector ), and for both modes have . The fast mode is below by , yet it still controls an explicit solver. Decouple: RK4 on multiplies each step by with (the degree-4 Taylor polynomial of , from 4.3 Runge–Kutta Methods & Butcher Tableaux). The fast mode stays bounded only if . On the negative real axis at (block D bisects it), so the limit is . The slow mode would tolerate far more: at block D gets against .
Now run it. Block D takes with near the limit; the exact is :
| (steps) | from RK4 | |
|---|---|---|
| 0.025 (400) | 0.64844 | |
| 0.0275 (364) | 0.94415 | |
| 0.0278 (360) | 0.98873 | |
| 0.0279 (358) | 1.01213 | |
| 0.028 (357) | 1.02413 | |
| 0.03 (333) | 1.38102 |
The factors are for the actual step , so they differ slightly from . Below the limit the answer is exact to the digits shown (the fast mode has decayed away). At , is still stable, but the fast mode is only decaying slowly, so a trace of it is left at . Just above the limit the fast amplitude grows by to percent per step ( and in the table) and, after hundreds of steps, becomes visible; at it explodes. This is the same story as the diffusion matrix of 7.1 Systems & State Space: the stable step is dictated by the fastest, most uninteresting mode, here about steps per unit of time to follow a mode that needs few. Implicit methods are not bound by this limit for decaying modes (from memory); 4.5 Stability Regions & Stiffness works that out. In the sim, pick the widely separated preset and drag upward: the amplification factor in the readout crosses and the RK4 curve diverges.
Computing it
Now build the method yourself. The exercise matrix is the saddle example; a second test uses with , whose state at is .
Concept checks
Problems
Solve , .
Show worked solution
Trace , determinant : . For : , . For : , . Solve , : , . So . Check at : . Check the ODE on the first component: and . Both eigenvalues are positive, so every non-zero start grows.
For , , : write the companion matrix, find its eigenvalues and eigenvectors, and solve.
Show worked solution
, , the characteristic polynomial, so . The eigenvectors are : and (check ). The conditions are and . The second gives , and then gives , . So ; check .
A system has eigenvalues and . Using for , find the largest stable RK4 step and the number of steps to reach . Then find the step at which a single step of the slow mode, , is within of , and compare step counts.
Show worked solution
The fast mode needs , so and steps. For the slow mode alone, solve numerically: (at the ratio is , a error, too big; at it is ). That needs about steps, so stability forces about times more steps than accuracy of the slow mode asks for. That is stiffness. (These numbers come from the formula and a root-finder, not from the sim or lab.)
Key Takeaways
- Trying in gives : exponential solutions come from eigenpairs.
- A trajectory that starts on an eigenline stays on it, moving in or out along the line according to the sign of ; off the lines the curve bends toward the slower-decaying or faster-growing direction.
- The general solution is , valid when (so never vanishes). Example: and .
- Initial conditions are a linear solve ; nearly parallel eigenvectors give a large condition number (200 in our example), big cancelling coefficients and transient growth (25 times) despite negative eigenvalues.
- Decoupling with gives : independent scalar equations. All eigenvalues negative means decay from every start.
- The characteristic equation of is for its companion matrix, so the eigenvalues are the old roots ( and in our check).
- The eigen-solution matches
expmto , and RK4’s gap shrinks like . The step limit of RK4 is : for , confirmed by the blow-up between and . - Linear stability of a flow is the same eigenvalue problem for a perturbation; real parts decide growth (from memory for the full fluid cases). Next: complex and repeated eigenvalues in 7.4 Complex & Repeated Eigenvalues, the matrix exponential in 7.5 The Matrix Exponential & Fundamental Matrices.