Almost everything so far had one unknown. A fluid solver has millions: a velocity at every cell of the grid, all pushing on each other. A flight simulator has twelve or so (from memory for a rigid aircraft: three each for position, velocity, attitude and rotation rates). The trick that makes this manageable is the one you saw in 5.7 Higher-Order Linear Equations: stack the unknowns into a state vector and write one equation, . This lesson builds that language and shows that the RK4 you wrote for a scalar in 4.3 Runge–Kutta Methods & Butcher Tableaux carries over line for line. It also introduces the picture the rest of the module lives in, phase space. 1.1 Vectors, Linear Combinations & Span supplied vectors as arrows and lists; here a vector is a state.
Transcript
- Almost everything so far had one unknown. Now take two tanks, 100 and 50 litres, with pumps moving brine back and forth. The salt in tank 1 depends on tank 2, and the other way round, so neither equation solves alone. It is a coupled system: x prime equals A x plus b.
- Start both tanks with fresh water and integrate. At 30 minutes the tanks hold 194.115 and 78.018 grams, and at 60 minutes 260.211 and 122.707. The equilibrium solves A x equals minus b, giving 300 and 150 grams: 3 grams per litre, like the inflow.
- Any nth order equation is a system in disguise. Stack y, y prime and y double prime into a state vector. Each row says one is the derivative of the next; only the last carries the physics. The exact y of 1 is 0.7474195, and RK4 on the system prints 0.7474207.
- The phase space has one axis per component, so a point is a complete state. The vector field gives the velocity at every point, and a solution is a curve tangent to the arrows. At 0, 0 the field is 12, 0, so the curve leaves horizontally. Equilibria are where the arrows vanish.
- Trajectories of an autonomous system cannot cross. If two met at a point, both would solve the same initial value problem there, and uniqueness gives one solution, so they are the same curve. Loops can close, but a trajectory inside a loop is trapped inside it.
- Here is one solution two ways. The time series of the damped oscillator shows when. The phase plane drops time and shows where the state can go, spiralling into the origin. At t equals 10 the exact state is x equals minus 0.336852 and v equals 0.185346.
- The whole solver fits in a few lines. Make x an array and let f return an array: plus and scalar multiples act componentwise, so scalar RK4 becomes RK4 for a system. A step costs four evaluations of f, whatever n is. The order-4 proof carries over, from memory, so measure it.
- Is it really fourth order? Measure it. On the damped oscillator the RK4 error at t equals 10 falls by 16.11, 16.05 and 16.02 per halving of h. Two to the fourth is 16. Euler's ratios, 2.64, 2.29 and 2.13, slide toward 2: first order. Measured, not assumed.
- Predator-prey has a conserved quantity. Dividing the equations separates the variables, and integrating gives H equals x minus log x plus y minus log y. Its derivative along solutions is zero. At 2, 1 it equals 2.306853, so each orbit is a level curve of H.
- That makes H a drift detector. RK4 to t equals 60 drifts by minus 6.888 times 10 to the minus 5 at h 0.2, and minus 6.047 times 10 to the minus 9 at h 0.025. Euler drifts by plus 0.468, spiralling outward. H is blind to timing, so a small drift is necessary, not sufficient.
- Take a ring of six cells with heat flowing between neighbours: six coupled ODEs, u prime equals D u. With all the heat in cell 3, at t equals 5 the cells hold 0.1385, 0.1933, 0.2234, 0.1933, 0.1385 and 0.1129. By t equals 20 every cell is between 0.1664 and 0.1669.
- This is the method of lines. A flight simulator has about twelve states, from memory. A grid of 100 cubed cells with three velocity components has three million unknowns, an illustration, and still one state vector. From memory, Stam's stable fluids takes an implicit step, because the explicit step is limited by delta x squared.
- To sum up. A system is a state vector and one equation. Higher order equations become systems, and trajectories in phase space cannot cross. RK4 for systems is the same code, with order 4 measured. A conserved quantity detects drift. Next: linear systems, where eigenvalues solve x prime equals A x exactly.
Why one equation is not enough
Coupled tanks. Tank 1 holds 100 L and tank 2 holds 50 L. Brine at 3 g/L flows into tank 1 at 4 L/min. A pump moves 6 L/min from tank 1 to tank 2, another moves 2 L/min back, and tank 2 drains 4 L/min. Volumes stay constant (tank 1: in , out ; tank 2: in , out ). Let be grams of salt. Each pipe carries concentration times flow:
Numerically, and . Neither equation can be solved alone: needs and vice versa. That is a coupled system. In matrix form with and . Start both tanks with fresh water. Block A of the lab cell integrates this: at min the tanks hold g, which is and g/L, and at they hold g, and g/L. Both concentrations climb toward the inflow’s g/L, tank 1 first. The equilibrium, where , solves . The second row gives , so ; the first then reads , so and g. Check: and . In concentrations that is g/L in both tanks, the same as the inflow, which is the only way the salt in the tanks can stop changing. The cell also solves the same system exactly (the matrix exponential, a later lesson) and the gap is below .
Predator-prey, as a teaser. Let be prey and predators, scaled so every rate is 1: and . Prey grow unless eaten, predators starve unless they eat. Populations oscillate: when prey are plentiful predators boom, then prey crash. Honest limits: this model has no carrying capacity and no randomness, and its perfect closed loops are fragile (from memory, change the equations slightly and the loops become spirals or a limit cycle; 7.8 Conservative Systems & Lyapunov Functions and 15.2 Predator–Prey & Competing Species return to this). We use it because it is nonlinear, has no closed-form , and has a conserved quantity that makes a test for a solver.
Vector notation and the state vector
Write and , where returns numbers. If does not appear, the system is autonomous: . Any non-autonomous system becomes autonomous by adding a state with , so autonomous loses no generality.
Higher order becomes first order. For set . Then
The first equations just say “the derivative of each is the next one”; only the last carries the physics. For a linear equation this is the companion matrix of 5.7 Higher-Order Linear Equations, which proved its eigenvalues are the characteristic roots; the new point is that may be nonlinear. The pendulum is .
Worked example (third order). with , . The state is :
The roots of give . The initial conditions say , , , solved by (check each row). So . Block D runs RK4 on the system with and prints : the error is . The initial condition of a system is a vector, which is why an nth-order ODE needs initial values.
Phase space and the vector field
The phase space is the space of all states, one axis per component. A single point is a complete snapshot: tank 1 holds 194 g and tank 2 holds 78 g, nothing else is needed to say what happens next. The vector field attaches to each point the velocity with which the state moves there: arrows, exactly like the direction fields of 3.2 Direction Fields & Euler's Method Revisited, but now an arrow is an actual velocity vector, not a slope. A solution is a curve everywhere tangent to the arrows, the trajectory. At the tank field is : the first tank gains salt, the second has none to receive yet, so the curve leaves horizontally. In the sim, the short black arrow is at the dot.
Trajectories of an autonomous system cannot cross. Suppose two trajectories met at a point . Both are solutions of the initial value problem , there (shift the clock of one so they pass at the same instant; this is legal only because has no ). The uniqueness theorem of 3.7 Existence & Uniqueness says that problem has one solution when is smooth. (That lesson did the scalar case; the vector version needs the same Lipschitz idea, quoted from memory.) So the two curves are the same curve. They can close into a loop or approach an equilibrium, but not cross. In one dimension this meant solutions could not pass through each other; in the plane it means a trajectory that starts inside a closed loop is trapped inside it, and that is what organises the pictures in 7.3 Phase Portraits & the Trace–Determinant Plane.
Equilibria are the points where and the arrows vanish. Solving is an algebra problem and finds them before any integration: for the tanks; and for predator-prey (from and ); for the damped oscillator; and , , that is , for the pendulum. The sim marks them with hollow circles (for the pendulum, those at and ). Whether curves near an equilibrium approach it, circle it or leave it is the subject of the next lessons.
Two views of one solution. The time series plots and against . The phase-plane curve plots against with time removed. Time is recoverable only from the arrows’ lengths. The time series shows when; the phase plane shows where it can go, and for a periodic orbit it collapses an endless oscillation into one closed loop.
The sim integrates every seed with the same vector stepper. Choose coupled tanks, keep the three default seeds and scrub the active seed (the starting-empty one) to : the readout shows and g/L, the numbers above. Choose predator-prey and scrub the active seed around: its time series oscillate while the phase curve is one closed loop that never meets the other seed’s loop. The damped oscillator (, ) spirals into the origin, and the pendulum shows the closed loops of small swings with open waves of full rotations outside them. Switch the method to Euler on the pendulum (seed , , ): the energy panel climbs, and the readout gives a change of to . The separatrix between swinging and rotating is (the pendulum balanced at the top, at rest), so Euler has pumped the swing to the brink of going over the top, which no real pendulum with no forcing does. RK4 at the same changes by . The same step size, the same equations, a different physical conclusion: the integrator is part of the model.
Solving systems numerically
Here is the whole solver. It is written once, for any system:
def rk4(f, x0, T, h):
x = np.array(x0, float); t = 0.0
for _ in range(round(T / h)):
k1 = f(t, x); k2 = f(t + h/2, x + h/2*k1); k3 = f(t + h/2, x + h/2*k2); k4 = f(t + h, x + h*k3)
x = x + h/6*(k1 + 2*k2 + 2*k3 + k4); t += h
return x
Cost. One RK4 step calls four times, so the price of a step is four evaluations of the right-hand side plus a few vector additions, whatever is. For the six-cell ring below, is a small matrix-vector product; for a grid solver it is a sweep over every cell. The error constant and the cost per step are the two numbers to weigh when choosing .
Observed order. Use the damped oscillator with , , which has an exact solution: , . Block B measures the norm of the state error at . RK4 with gives errors , , , . Each halving divides the error by , , . Order predicts , and : fourth order, measured, not assumed. Euler on the same problem with has errors , , , and ratios , , , sliding toward : first order. The sim’s readout reports this error for the oscillator: choose it with the seed , , and drag .
A conserved quantity as a drift detector. For predator-prey, separates into , which integrates to
Check directly: . So each orbit lies on a level curve of , and a numerical solution that leaves it is wrong by exactly the amount it left. Start at , where . Block C reports . RK4 to with drifts by , , , , ratios , , : not yet at these step sizes, but falling toward it. Euler is very different: to , drifts by and by . Positive drift means the orbit spirals outward, the telltale of Euler on a centre (the pendulum energy behaves like this too; RK4 at and over from changes it by and , shown in the sim’s readout). The slow, systematic leak that RK4 has is the topic of 4.7 Energy & Symplectic Integrators.
What the detector cannot see. depends only on where the state is, not when. A solution that stays exactly on the right orbit but runs it slightly too fast has zero drift and is still wrong. So a small drift is necessary, not sufficient. Block B’s comparison with an exact solution is the stronger test when one exists.
Computing it
Now build the solver yourself, and convert a third-order ODE. The test equation with , , has the exact solution (all three roots are , and these are the initial values of ), so should be .
Concept checks
Problems
Write as . Then make it autonomous, and state how many initial values the IVP needs.
Show worked solution
With : . To remove , add with , , and write in : the autonomous system in four variables. The IVP needs three initial values, : the dimension of the original state (the fourth is fixed at ).
For the tank system, find at and at . Then find the nullcline and say what it means for the direction of the arrows.
Show worked solution
At : , so the curve leaves in the direction. At : and , an equilibrium, with concentrations g/L equal to the inflow. The nullcline is , the line : there the arrow is purely horizontal ( momentarily constant). Below that line (, tank 2 relatively dilute) and arrows point up; above it they point down. So every curve is pushed vertically toward that line, and the equilibrium sits on it ().
Show that is constant along solutions of , , and find , the equilibrium value.
Show worked solution
. Factor : . At , . The start has , above , and everywhere (each of , has minimum at ), so the level curves around are closed loops: no orbit can escape, in line with the sim.
Key Takeaways
- A coupled system has one unknown per state; a model like the two tanks gives and approaches the equilibrium g.
- An nth-order ODE is a first-order system for ; only the last equation carries the physics, and nonlinear right-hand sides work the same way. Adding as a state makes any system autonomous.
- Phase space is the space of states, the vector field gives the velocity there, and trajectories of an autonomous system cannot cross, by uniqueness of the IVP.
- A time series and a phase-plane curve are two views of one solution; time is hidden in the phase plane.
- Scalar RK4 is vector RK4 with arrays; the measured error ratio per halving of was for RK4 and for Euler.
- A conserved quantity detects drift off the orbit (RK4 about at , Euler at ) but is blind to errors along the orbit.
- The method of lines turns a grid of cells into an -dimensional linear system whose total is conserved. Next, 7.2 Linear Systems & the Eigenvalue Method solves exactly, with the eigenvectors of 1.5 Eigenvectors & Eigenvalues and the linear theory of 5.1 Second-Order Linear Equations: Structure in a new role.