In Jos Stam’s fluid solver every grid cell stores a number, and the number changes in time and depends on where the cell sits. The unknown is no longer a state marched by Euler or RK4 but a function , and its governing equation contains derivatives with respect to both variables: a partial differential equation, a PDE. 3.1 What Is a Differential Equation? told ODE from PDE in a paragraph. This lesson earns the rest: it derives most PDEs of physics from one bookkeeping rule, sorts the second-order ones into three families (equilibrium, diffusion, waves), and shows that running heat flow backwards defeats every computer. It uses the divergence theorem of 2.10 Green, Stokes & the Divergence Theorem and the boundary conditions of 8.1 Two-Point Boundary Value Problems.
Transcript
- In Stam's fluid solver every grid cell stores a number that depends on where the cell sits, and on time. The unknown is no longer a state y of t but a function u of x and t, and its equation holds derivatives in both: a partial differential equation. Three things change. The solution is a function: u equals f of x minus c t solves u t plus c u x equals zero for any f. The data live on whole curves and surfaces, not at one point. And there is new vocabulary: the order, linear or not, homogeneous or not.
- Most PDEs start with one sentence: a conserved quantity in a region changes only by flow through the boundary or production inside. Let u be the density, F the flux and S a source, on the interval a to b. The balance reads d by dt of the integral of u equals flux in minus flux out plus the integral of S. Since the interval is arbitrary, the integrand vanishes: u t plus F x equals S. Choose F equal c u and this is advection. Choose F equal minus D u x, Fick's law, and it is the heat equation. Let heat settle, so u t is zero, and Laplace appears.
- Now one Gaussian bump under four equations. Advection with c equal 1 starts the bump at 0.25 and slides it: at t equal 0.4 the peak is still 1.0000, now at x equal 0.65, and the total is unchanged. Heat, with D equal 0.01 and insulated ends, smooths it: the peak falls to 0.7071 at t equal 0.16 and to 0.4924 at t equal 0.5, while the total stays 0.14180. Zero flux, so nothing escapes.
- The wave equation, with fixed ends and released at rest, splits the bump into two half-height copies: peak 0.5000 at t equal 0.2. At 0.5 each copy meets its upside-down reflection and the string is flat, peak 0.0000, though not at rest. At t equal 1 the bump is back upside down, minimum minus 1.0000, and at t equal 2 it is upright again, peak 1.0000. Laplace has no time. With the bump held on the top edge, u at the centre is 0.0579, and at 0.5, 0.9 it is 0.3601: the boundary leaks smoothly inward.
- Why do the types differ? Try u equal f of alpha x plus beta y. The second-order part collapses to a bracket times f double prime, and where the bracket vanishes a kink can ride along unchanged: a characteristic. The discriminant b squared minus a c counts them. Positive: two directions, hyperbolic. Zero: one, parabolic. Negative: none, elliptic. Laplace gives minus 1, the heat equation 0. The wave equation u t t equals 4 u x x gives 4, with slopes plus or minus 0.5. The next row gives 1.25, with slopes 2.6180 and 0.3820.
- The type is pointwise, and it can change from place to place. The Tricomi equation y u x x plus u y y has discriminant minus y: elliptic above the axis, hyperbolic below, 5000 cells each on a 100 by 100 map. Steady small disturbances of compressible flow have discriminant M squared minus 1: elliptic when subsonic, hyperbolic when supersonic. At Mach 2 the slopes are plus or minus 0.5774, the Mach lines. With an invented Mach field, a supersonic pocket in a subsonic stream, the hyperbolic region covers 1372 of ten thousand cells, 13.72 percent, against the exact pi ln 2 over 16, 13.61 percent.
- Each type needs different data. Heat needs the initial profile and one condition at each end. Waves also need the initial velocity. Laplace needs one condition at every boundary point. A problem is well posed, Hadamard said, from memory, if a solution exists, is unique, and depends continuously on the data. The first two can fail through bad data: Neumann data all round Laplace's equation need a zero net flux. A computer needs the third, because every stored number carries round-off.
- Condition three hurts. On zero to pi, a mode sine k x is multiplied by e to the minus k squared T going forward, and by e to the plus k squared T going backward. At T equal 0.1 and k equal 30 the forward factor is 8.19 times ten to the minus 40. The backward factor is 1.220 times ten to the 39. Noise of ten to the minus 8 becomes 1.22 times ten to the 31, while the signal grows by only 1.1052. Only modes with k up to 13 come back with noise below 1. A large enough k beats any bound: ill posed, for every method.
- Where does each equation go next? Hyperbolic advection is solved along characteristics, and ends on Stam's semi-Lagrangian step. Parabolic heat is derived again with physical constants, and meets an explicit stability limit, r at most one half. Elliptic Laplace is solved on a rectangle and a disk, then by iteration, and pressure in a fluid is exactly such a problem. Stam's solver runs all three types in one loop: advect, diffuse, project.
- One question is left. Where do the modes sine k x and the rates k squared come from? Separation of variables. Put u equal X of x times T of t into u t equals u x x, and T prime over T equals X double prime over X. The sides depend on different variables, so both equal a constant, minus lambda. That leaves X double prime equals minus lambda X, with X zero at both ends, and it has a nonzero solution only for special lambda, the eigenvalues. Here lambda equals k squared, with no upper limit: running time backward has no safe limit either. That is lesson 8.2, on eigenvalue problems and Sturm-Liouville theory, and it is not yet written.
From ODE to PDE: what changes
Three things change when a second variable arrives.
The solution is a function. The general solution of carries one constant. The general solution of is with an arbitrary function (chain rule: , ). Even has the general solution .
Data live on curves and surfaces. An ODE is pinned down by values at one or two points. A PDE needs whole functions: the initial profile along and boundary values along and for all time. For equilibrium problems the data wrap around the entire boundary.
There is more vocabulary. Subscripts are partial derivatives, (2.1 Functions of Several Variables & Partial Derivatives). The order is the highest derivative that appears. An equation is linear when and its derivatives appear only to the first power, times known functions of the independent variables, and homogeneous when is also a solution. Linear equations obey superposition, the property behind every series solution to come.
| equation | name | order | linear? | homogeneous? |
|---|---|---|---|---|
| advection (transport) | 1 | yes | yes | |
| heat with a source | 2 | yes | no, unless | |
| wave | 2 | yes | yes | |
| Laplace | 2 | yes | yes | |
| Burgers (inviscid) | 1 | no: |
Conservation laws: where PDEs come from
Most PDEs here start with one sentence: a conserved quantity in a region changes only by flow through the boundary or production inside.
Let be the density (kilograms of dye per metre of pipe), the flux, the amount crossing per second, positive to the right, and a source per metre per second. For the interval , the control volume,
Now use and, for smooth , move the derivative inside the integral:
A continuous integrand nonzero at one point would keep one sign on a small interval there, with a nonzero integral. So the integrand is zero:
In three dimensions the box is a region with outward normal , and the divergence theorem (2.7 Divergence: Sources & Sinks: net outflow per unit volume) converts the boundary integral into a volume integral:
One equation, two unknowns. The missing piece is a constitutive law saying how the flux depends on the density.
- Advection. Stuff riding a current of speed : everything within of a point crosses it in time , so and .
- Diffusion. Stuff spreads from crowded to empty at a rate proportional to the slope, Fick’s law for dye and Fourier’s law for heat (empirical, from memory): , . The minus sign makes the flow run downhill, and for constant
- Both at once. gives (9.10 First-Order PDEs: Advection & Characteristics). A nonlinear flux gives Burgers’ .
- Waves. With and , the pair , are two coupled conservation laws, and eliminating and gives . That a string obeys this is from memory; 9.4 The Wave Equation & Vibrating Strings (lesson 9.4) derives it.
- Equilibrium. Let 2D heat flow settle: in leaves the Laplace equation , with no time in it.
The integral balance is more fundamental: where jumps the differential form is meaningless but the box balance holds, which is how shocks are handled in 11.12 Compressible Flow & Shocks.
One bump, four equations
The first panel of the lab starts each equation from the same Gaussian bump on (the advection bump starts at to leave room), with exact formulas and no time stepping: advection is the shift on a periodic interval, heat and wave are series with modes, and Laplace holds the bump on the top edge of the unit square.
With and the sim prints:
- Heat (insulated ends): the peak falls to at and at while the total stays (trapezoid on points): zero flux, nothing escapes.
- Wave (fixed ends, released at rest): the bump splits into two half-height copies, peak at (copies at and ). At each copy meets its upside-down reflection at a wall and the string is flat, peak , though not at rest. At the bump is back upside down, minimum ; at it is upright, peak .
- Advection: at the peak is still , at , with the total unchanged.
- Laplace: no time. and : the boundary leaks smoothly into the interior.
Diffusion smooths and forgets, waves carry shape, equilibrium spreads boundary data inward. The classification explains why.
Classifying second-order linear PDEs
Take two independent variables (for time-dependent problems is ) and constants :
The in front of is a convention. Only the second-order part decides the type (stated, not proved). To see why, ask which sharp features a solution can carry. Try with possibly kinked. Then , , , and the second-order part collapses to
If the bracket vanishes for some direction , then every solves the pure second-order equation, so a kink can ride along the lines and persist. These lines are the characteristics. With the bracket vanishes when , whose discriminant is . The number of real directions decides the type:
The names come from the conic , dashed in panel 2. The characteristic curves have slope (if one direction is vertical).
| equation | type | of characteristics | ||||
|---|---|---|---|---|---|---|
| Laplace | 1 | 0 | 1 | elliptic | none | |
| heat , | 1 | 0 | 0 | parabolic | (lines const) | |
| wave , | 4 | 0 | hyperbolic | |||
| 1 | 1.5 | 1 | hyperbolic | and | ||
| 1 | 1 | 1 | parabolic |
Every row is a preset of panel 2. In the wave row , so solve it: shapes moving at speed .
What each type means
Hyperbolic: waves. Features travel along two families of characteristics at finite speed. Nothing is smoothed (the wave bump kept height ), and a kink stays a kink.
Parabolic: diffusion. One characteristic direction, const: no travelling features, only a direction in time. Information moves forward only, and sharp features die at once. The spreading Gaussian is positive everywhere for any , so strictly diffusion has infinite signal speed, with a vanishing tail (from memory; lesson 9.9 derives it).
Elliptic: equilibrium. No real characteristics, so no kink survives and no direction is special. Every boundary point influences every interior point at once, the interior is smooth, and the solution cannot exceed its boundary values (the maximum principle of 9.6 Laplace's Equation: Rectangles & Disks).
Data each type needs
Count time derivatives as for ODEs (5.1 Second-Order Linear Equations: Structure): one needs an initial value, two need value and velocity.
| type | example | data | problem |
|---|---|---|---|
| parabolic | heat | , one condition at each end for all | initial-boundary value |
| hyperbolic | wave | and , one condition at each end | initial-boundary value (on an infinite line, Cauchy) |
| elliptic | Laplace | one condition at every boundary point | boundary value |
The boundary conditions are the three kinds from 8.1 Two-Point Boundary Value Problems, now with a flux reading. Dirichlet holds . Neumann holds , prescribing Fick’s flux ; is zero flux, why the insulated rod kept its total. Robin, , makes the flux proportional to the value (Newton cooling). A Cauchy problem gives and its normal derivative on a curve and marches away from it.
The type can change from place to place
Classification is pointwise. The Tricomi equation has , , , so : elliptic above the -axis, hyperbolic below. Panel 3 maps in cells: elliptic, hyperbolic.
A physical example, from memory: steady small disturbances of compressible flow obey , so : elliptic for subsonic flow, hyperbolic for supersonic. For panel 2 gives slopes , the Mach lines at (). Panel 3’s second map has an invented Mach field , a supersonic pocket in a subsonic stream, hyperbolic inside the circle of radius . That is of the square; the cell count gives of , (cells are classified at their centres). A solver needs two kinds of method in two regions.
Well-posedness and the backward heat equation
Hadamard (from memory) called a problem well posed if (1) a solution exists, (2) it is unique, (3) it depends continuously on the data. A computer needs the third: every stored number carries round-off.
The first two can fail through badly chosen data. Prescribe the flux all round a region for Laplace’s equation: the divergence theorem gives , so there is no solution unless the net flux is zero, and then it is unique only up to an added constant.
Condition (3) is the one that hurts. Work on with at both ends and . A single mode has , so . Running forward a time multiplies the mode by . Recovering the past from the present, the backward heat equation, multiplies it by .
Take data at : the forward solution of the initial profile , plus noise . Panel 4 prints:
- forward, : the factor is ;
- backward: , so the noise becomes , while the signal grows only by ;
- the noise reaches size at (and at for );
- at only modes with come back with noise below , since .
A large enough beats any bound, so no accuracy of the data guarantees any accuracy of the answer: the problem is ill posed.
Where each equation goes next
9.10 First-Order PDEs: Advection & Characteristics (hyperbolic) solves along characteristics and ends on Stam’s semi-Lagrangian step. 9.2 The Heat (Diffusion) Equation (parabolic) redoes the heat derivation with physical constants and finds the explicit limit . 9.6 Laplace's Equation: Rectangles & Disks (elliptic) solves the rectangle and the disk, then meets 10.8 Elliptic Solvers: Jacobi, Gauss–Seidel & SOR. For numerics see 10.1 Finite Differences: Discretizing Derivatives, 10.2 Explicit Methods for the Heat Equation and 10.6 Advection Schemes: Upwind, Lax–Wendroff & Beyond; in fluids, 11.11 Stam's Stable Fluids, Step by Step runs all three types in one loop. 9.11 Nonlinear Waves: Burgers' Equation & Shocks (lesson 9.11) returns to Burgers’ equation and shocks.
Computing it
It prints the total at the start and after both runs, and peaks (no cell centre sits exactly at ), after diffusive steps and after advective ones; then the discriminants and the factor . The advective bump has moved exactly cells, so an exact shift would have kept its peak at . The upwind flux smears it instead, a numerical error that 10.6 Advection Schemes: Upwind, Lax–Wendroff & Beyond analyses. The total is exact, the shape is not.
Concept checks
Problems
(a) For find the type and two directions for which solves it for every . (b) Where is elliptic, parabolic, hyperbolic?
Show worked solution
(a) , , : , hyperbolic. The bracket is , zero for and , so and solve it. Check the second: .
(b) , , : . Elliptic for , parabolic on , hyperbolic for .
Dye of density is carried down a pipe at m/s and decays at the rate , s. (a) Write the PDE. (b) Find the steady profile with and the fraction left m downstream.
Show worked solution
(a) and , so .
(b) Steady means , so (metres). At the fraction is .
On with and noise : (a) the noise size after at ; (b) the at which it reaches ; (c) the largest keeping the noise at most for .
Show worked solution
(a) , so the noise is . (b) . (c) , so : the largest mode is . Panel 4 with prints all three.
Key Takeaways
- A PDE’s general solution contains arbitrary functions, and its data live on curves or surfaces.
- A control volume gives , the divergence theorem gives , and a flux law closes it: is advection, is heat, steady heat is Laplace. Finite volumes keep the total to round-off.
- counts real characteristic directions: two (hyperbolic, waves), one (parabolic, diffusion), none (elliptic, equilibrium). The type can change with position.
- Elliptic problems need boundary data all around, parabolic ones an initial profile plus boundary conditions, hyperbolic ones also an initial velocity.
- Well posed means a solution exists, is unique and depends continuously on the data. Backward heat fails the third: noise grows by over , for any method.
- Stam’s advect, diffuse and project steps are a hyperbolic, a parabolic and an elliptic problem in one loop.
Where do the modes and rates come from? The next row in the backlog, 8.2 on eigenvalue problems and Sturm-Liouville theory (8.2 Eigenvalue Problems & Sturm–Liouville Theory), answers that, and separation of variables needs it. Substituting into gives ; the sides depend on different variables, so both equal a constant . That leaves with , which has a nonzero solution only for special , the eigenvalues, as does. Here with no upper limit, which is why running time backward has no safe limit either.