Almost everything in simulation is nonlinear: drag grows with the square of speed, a pendulum’s restoring force goes like , and the Navier–Stokes equations contain the velocity multiplied by its own derivatives. Yet nearly every tool we have for analyzing simulations is linear: matrices, eigenvalues, stability factors like . The bridge between the two worlds is the oldest idea in calculus: zoom in far enough on anything smooth and it looks linear. In Calc I that gave you the tangent line. In 2.1 Functions of Several Variables & Partial Derivatives you learned to measure slopes of a surface along the coordinate directions. This lesson glues those slopes together into a tangent plane, then into a matrix, the Jacobian, which is the honest derivative of a map from to . Along the way we derive the multivariable chain rule, and from it the single most-used operator in fluid dynamics: the material derivative.
Transcript
- Calculus has one big idea: if you zoom in far enough on anything smooth, it looks linear. This lesson takes that idea into several dimensions.
- Start with what you know from Calc One. Here is a curved graph and one point on it. The tangent line touches the curve there with the same slope.
- Now zoom in on the point. The more we magnify, the less the curve bends away from its tangent line, until the two are indistinguishable. Locally, the function is linear: f of x zero plus h is about f of x zero plus slope times h.
- Functions of two variables have graphs that are surfaces. Here is a dome, z equals f of x and y, and a point on it.
- Slice the surface in the x direction and you get a curve whose slope is the partial derivative f sub x. Slice it in the y direction and the slope is f sub y.
- Two tangent lines through the same point determine a plane: the tangent plane. Its equation just adds the two slopes, each times how far you step in that direction.
- Walk around it. The plane hugs the dome near the point, and peels away from it only as you move farther off.
- Zoom in, and the dome and the plane become indistinguishable. That is linear approximation in two variables: near a point, a smooth surface is a plane.
- This is what differentiable means. The error between the surface and its tangent plane must shrink faster than your distance from the point. For smooth functions it shrinks like the distance squared.
- A surface with a sharp crease or a cone tip has no tangent plane there. Zooming in never makes it flat, so it is not differentiable, even if both partial derivatives happen to exist.
- Next, let us see what happens when the inputs themselves are moving: the chain rule.
Zoom in: the tangent line, revisited
Recall the one-variable picture. If is differentiable at , then for a small step ,
and the error is small compared with : as . For a smooth function, Taylor’s theorem says more precisely that the error is about . Halve the step, and the error drops by a factor of four.
That is all “the tangent line is the best linear approximation” means. Geometrically: magnify the graph around by a factor . The curve’s deviation from its tangent line, measured in magnified units, is about , which vanishes as you zoom. Smooth curves are locally straight.
Tangent planes
Now take a function of two variables, , whose graph is a surface. Pick a point on the ground and the point above it on the surface.
Any non-vertical plane through that point has an equation of the form
for some slopes and . Which plane is tangent? Slice everything with the vertical plane . The surface becomes the curve , whose slope at is the partial derivative . The plane becomes the line , with slope . Tangency demands the slopes agree, so . Slicing with gives in the same way.
The formula reads naturally: start at the known height, then add (slope in ) × (step in ) plus (slope in ) × (step in ). Each partial derivative handles its own direction, and for small steps the two contributions simply add. That additivity is what “linear” means, and it’s the reason everything that follows works.
Worked example: a tangent plane and an estimate
Find the tangent plane to at , and use it to estimate .
- Height: .
- Slopes: , so ; and , so .
- Tangent plane: .
- Estimate: with and , .
The exact value is , so the error is . Where does it come from? Expand exactly around : because is quadratic, , and the leftover is . The error is quadratic in the step, exactly like the tangent line. You’ll see the general version (with second derivatives, the Hessian) in 2.4 Taylor Series in One and Several Variables.
Differentials
Writing for small changes in the inputs and for the resulting change predicted by the linearization gives the compact total differential
Engineers use this constantly for sensitivity and error propagation: if the inputs are uncertain by and , the output is uncertain by about in the worst case.
Let .
- Find the tangent plane at .
- Use it to estimate and compare with the exact value.
- Is the error roughly the size you’d expect for a step of length about ?
Show worked solution
- , , . The tangent plane is .
- . Exactly, . The error is about .
- Yes. The step has length , and the error should be of order (second derivatives) × (step)², i.e. about times a modest constant. Indeed the second-order Taylor term is , which accounts for almost all of the error.
Differentiability: what “smooth” really requires
In one variable, “the derivative exists” and “the tangent line is a good approximation” are the same statement. In two variables they are not, and the difference is worth a few minutes.
In words: the linearization’s error must shrink faster than the distance , and it must do so as from every direction at once, not just along the axes. Partial derivatives only test two directions, so they can exist even when the function is badly behaved in between.
The classic counterexample. Let for and . On both axes is identically zero, so . But on the diagonal , no matter how close you get to the origin. The function isn’t even continuous at the origin, let alone well approximated by the plane . Partial derivatives exist; differentiability fails.
Fortunately, there’s a simple test that covers every function you’ll meet in practice:
Theorem. If and exist near and are continuous at , then is differentiable at .
Polynomials, exponentials, sines and cosines, and their sums, products, quotients (away from zero denominators) and compositions all qualify. Trouble lives at kinks, creases, cone tips and jumps: , at the origin, where , and a shock wave in a compressible flow.
The multivariable chain rule
The second video covers the rest of the lesson: the chain rule, the Jacobian and the material derivative. Watch it now for the big picture, then work through the details below.
Transcript
- Here is a temperature field, T equals x squared times y. Red is hot, blue is cold. A probe moves through it along a circle, so its x and y are both functions of time.
- The probe's reading changes for two reasons: it moves in x, and it moves in y. How fast does the reading change?
- Add the two contributions. The rate in x times how sensitive T is to x, plus the rate in y times how sensitive T is to y. That is the multivariable chain rule.
- A tree diagram keeps the bookkeeping straight. T depends on x and y, which each depend on t. Multiply along each branch from top to bottom, then add the branches.
- Now a whole map of the plane to itself, F of x and y, with two output components. A matrix would keep grid lines straight. This map bends them.
- Pick a point p and a small square window around it, of half width h. On the left is its image under the map. On the right we magnify that image by one over h, so it always fills the same box.
- For a big window the image is clearly curved. Overlay, in dashed amber, what a single matrix would do to the same window. They disagree.
- Now shrink the window. Watch the cyan grid straighten out and slide onto the amber one. Zoomed in far enough, the nonlinear map is a linear map.
- That matrix is the Jacobian. Exactly as in lesson one point two, its columns are where the tiny steps in x and in y land: the partial derivatives of F with respect to x and to y.
- So the best linear approximation of a map is F of p plus delta, approximately F of p plus the Jacobian times delta. Same formula as the tangent line, with a matrix in place of the slope.
- Composing two maps multiplies their Jacobians, which is the chain rule in matrix form. And the determinant of the Jacobian tells you how much the map scales tiny areas.
- Last idea, and the most important one for fluids. Wind blows to the right through a temperature field. A thermometer can sit still, or it can ride along with a parcel of air.
- First, a steady field: the temperature at each fixed place never changes, so the probe's reading is constant. But the parcel moves through warm and cold regions, so its temperature changes anyway.
- Now let the wind carry the warm and cold air along with it. The parcel now rides with its own air and its reading stays constant, while the fixed probe sees the pattern sweep past.
- The chain rule explains both cases. The rate felt by the parcel is the local rate at a fixed point plus the velocity times the spatial rates of change. This is the material derivative, D T D t.
- In shorthand, D D t equals partial d t plus u dot del. Every fluid equation, from advection to Navier Stokes, is written with it, because Newton's laws apply to parcels, not to fixed points in space.
- To recap. Smooth means locally linear. For a surface that is the tangent plane, for a map it is the Jacobian matrix, and following a moving parcel gives the material derivative.
- Next time, we pack the partial derivatives of a single function into one vector, the gradient, and find out why it always points uphill.
Suppose a temperature probe moves through a room. The temperature field is and the probe’s position at time is . Its reading is the composite function . How fast does the reading change?
Over a short time , the probe moves by and . Because is differentiable, its change is given by the linearization plus a small error:
Divide by and let . The error term vanishes (this is exactly where differentiability, not just existence of partials, is needed), and we get
Each input contributes (sensitivity of the output to that input) × (rate at which that input changes), and the contributions add.
Tree diagrams
When variables depend on variables that depend on other variables, draw a tree. Put the final output at the top, its direct inputs below, and so on down to the independent variables. Label each branch with the corresponding partial (or ordinary) derivative:
The rule: to differentiate the top with respect to a bottom variable, find every path from top to that variable, multiply the derivatives along each path, and add over paths. Here there are two paths from to , giving the two terms above.
Worked example: a probe on a circle
Let and let the probe follow , . Find , and its value at .
The partials are and ; the path derivatives are and . The chain rule gives
Check by substituting first: , and the product rule gives , the same thing. At , with : . The probe is cooling at that instant.
Substituting first worked here, but it often doesn’t: in a simulation, is a grid of numbers and the path comes from an ODE solver. The chain rule only needs the local partials and velocities, which is exactly what a solver has.
Two intermediate variables, two independent ones
If with and , the tree has two leaves on each branch, and . Following paths:
Worked example: polar coordinates
Express and in terms of and , where , .
The branch derivatives are , , , . So
Sanity check with : ✓, and ✓ (the distance from the origin doesn’t change as you go around a circle). Formulas like these are how the Laplacian gets rewritten in polar coordinates for flow around a cylinder.
Let with and .
- Draw the tree diagram.
- Use the chain rule to find and at .
- Check your answers by substituting first.
Show worked solution
- branches to and ; each of and branches to and . There are two paths from to and two from to .
- At : , . Then and . The branch derivatives are , , , . So and .
- . Then , which at is ✓. And , which at is ✓.
The dynamic pressure of an airflow is , where is the air density and the airspeed. A wind-tunnel run has and .
- Compute and its differential in terms of and .
- Show that the relative error satisfies .
- If density is known to within 1% and speed to within 2%, what is the worst-case relative error in ? Which measurement should you improve first?
Show worked solution
- . The partials are and , so (numerically, ).
- Divide by : .
- Worst case, the errors add: , i.e. about Pa. The speed error counts double because is squared, so improving the airspeed measurement pays off first.
The Jacobian matrix: the derivative of a map
So far the output has been a single number. Now let the output be a vector too. A map ,
moves every point of the plane somewhere else. In 1.2 Matrices as Transformations you pictured linear maps by watching the grid deform: lines stayed straight, parallel and evenly spaced. A nonlinear map bends the grid into curves. Examples are everywhere: the change from polar to Cartesian coordinates, a camera’s lens distortion, the curvilinear mesh wrapped around an airfoil, and the flow map that sends each fluid particle’s starting position to its position one time unit later.
Linearize each component separately, using its own tangent plane. For a small displacement from :
Stack these two lines and you get a matrix–vector product:
So the Jacobian is to a map what the slope is to a curve: the linear transformation the map looks like when you zoom in. A single-output function is the special case , where the Jacobian is the row from the tangent plane.
Things to try:
- With the wavy warp, set near its maximum. In the zoom window the cyan grid (the true image of a small square of grid around ) is visibly curved, and it disagrees with the amber parallelogram grid (the image under ).
- Slowly shrink . The cyan grid straightens and lands on the amber one. Watch the readout: error / h heads to zero (that’s differentiability), and error / h² settles to a constant (that’s the quadratic error of a smooth map).
- Drag the point around (you can click anywhere). The Jacobian changes from place to place: a nonlinear map is a different matrix at every point.
- Switch to the squaring map and drag toward the origin. The determinant goes to zero, and the zoom window collapses toward a point: the linearization stops being invertible there.
- Switch to the swirl. The determinant stays at exactly 1 everywhere, even where the grid is wildly sheared. Hold that thought for the fluids callout below.
Dragging uses Newton’s method to find which maps to where your mouse is, and Newton’s method works by repeatedly solving . The Jacobian is doing double duty.
Worked example: the polar coordinate map
Find the Jacobian of and interpret its columns and determinant.
Differentiate each component with respect to each input:
- Column 1, , is the unit vector pointing radially outward: increasing moves you straight away from the origin at unit speed.
- Column 2, , points around the circle with length : increasing by moves you an arc length , farther when you’re farther out.
- The determinant is the area scaling factor (lesson 1.4 Determinants as Volume Scaling): a tiny rectangle maps to a curved patch of area about . That’s the famous in polar integrals, which you’ll use in 2.5 Multiple Integrals.
The chain rule as matrix multiplication
Compose two maps: first , then . Near , looks like the matrix ; near , looks like . Composing linear maps multiplies their matrices (lesson 1.2), so
This is the general chain rule, and every formula in the previous section is a special case. For the probe, has the Jacobian and has the Jacobian ; their product is the matrix . The tree diagram’s “multiply along paths, add over paths” is just the row-times-column rule of matrix multiplication.
Worked example: composing two maps
Let and . Find the Jacobian of at two ways.
Via the chain rule. at . Next, , and at . Multiply, right to left:
Directly. , whose Jacobian is , equal to at ✓. Note that had to be evaluated at , not at : forgetting that is the most common chain-rule mistake.
Let . (If you know complex numbers, this is .)
- Compute and .
- At , approximately what is the area of the image of a tiny square of side centered at ?
- Find the Jacobian of the local inverse of near .
- Where is not locally invertible?
Show worked solution
- and .
- At , . The square has area , so its image has area about . (The image is close to a square rotated by and scaled by in each direction: .)
- Undoing near must undo its linearization, so by the chain rule () the inverse’s Jacobian at is .
- Where , i.e. only at the origin. There the map crushes every direction (both columns are zero), and indeed is two-to-one near the origin: .
The material derivative
Here is the payoff for fluids. A fluid has a temperature field that varies in space and time, , and a velocity field . Follow one fluid parcel, a tiny blob of fluid carried along by the flow. Its path satisfies
What rate of change of temperature does the parcel feel? It’s the derivative of the composite . The tree has three paths from down to : through , through , and directly. The chain rule gives
This rate gets its own notation, the material derivative:
(The vector of partial derivatives is the gradient; next lesson, 2.3 Gradient & Directional Derivatives, is all about it. For now, is just shorthand for .)
The two terms answer different questions, as the end of the video shows:
- is what a fixed thermometer reads: the change at one place in space.
- is what a thermometer riding with the fluid reads.
The difference, , is the change caused purely by being carried to a place with a different temperature.
Worked examples
Steady field, moving parcel. The air temperature along a valley is °C ( in meters, unchanging in time), and the wind blows along it at m/s. Then , but °C/s. Every parcel warms quickly, even though no thermometer bolted to the ground sees any change.
Frozen pattern, carried by the wind. If parcels keep their temperature (no heating, no diffusion), then , which rearranges to
This is the advection equation: a fixed probe sees temperature change only because the wind sweeps a different part of the pattern past it. For a uniform wind in 1D, any solves it, a pattern sliding rightward at speed . You’ll solve it properly in 9.10 First-Order PDEs: Advection & Characteristics.
A full computation. Let and . At the point and : , , , and . So
The field is cooling everywhere at 1 degree per second, but this parcel is being carried toward warmer fluid fast enough that it warms at 2 degrees per second.
- A dye concentration sits in a uniform flow . Show that and interpret.
- Consider the rigid rotation . It is steady: . Compute the parcel acceleration , component by component, and interpret it.
Show worked solution
- , , . Then . The dye pattern (circles centered at ) is simply carried along by the flow, so each parcel keeps its concentration.
- Write with , . Then , , , , and , and . So : each parcel accelerates toward the center with magnitude , exactly the centripetal acceleration of circular motion with . The velocity at each fixed point never changes, but every parcel is constantly turning. This is why alone can’t be the acceleration in Newton’s law.
Linearization in code
First, a numerical check of the chain rule: differentiate the probe’s reading by finite differences and compare with the chain-rule formula.
Now the main exercise. In a real code you rarely have formulas for the partial derivatives: implicit ODE solvers and Newton’s method need the Jacobian of whatever function you hand them, so they estimate it by finite differences, one column at a time. Column is the change in caused by nudging input , which is exactly the geometric meaning of a Jacobian column.
Once it passes, look at the printed errors in the solution: each halving of divides the error by about 4. That’s the signature of an error, the same order you’ll learn to read off log–log plots in 4.1 Local Error, Global Error & Order.
Finally, the material derivative in action. We trace a parcel through a time-dependent temperature field with RK4, then compare the rate of change of its temperature with .
Notice that a fixed thermometer would always read a cooling rate of exactly degrees per second, while the parcel’s rate swings positive and negative as it circulates between warm and cold regions.
Where linearization goes next
You will use the Jacobian constantly for the rest of the course:
- Stability of equilibria. Near an equilibrium of a nonlinear system , the deviation obeys , a linear system. Its eigenvalues decide whether the equilibrium is stable (7.7 Nonlinear Systems & Linearization).
- Implicit solvers. Backward Euler for a stiff system needs to solve for , which is done with Newton’s method, driven by the matrix (4.5 Stability Regions & Stiffness).
- Error analysis. Every finite-difference formula is a Taylor expansion, the higher-order version of the tangent plane (2.4 Taylor Series in One and Several Variables).
Key Takeaways
- Smooth means locally linear. Zoomed in, a differentiable curve is a line, a surface is a plane, and a map of the plane is a matrix.
- The tangent plane at is ; its error is quadratic in the step for smooth .
- Differentiable means the linearization’s error shrinks faster than the distance, from every direction. Existing partials are not enough; continuous partials are.
- Chain rule: multiply derivatives along each path of the tree diagram, then add over paths. In matrix form, .
- The Jacobian is the matrix of first partials. Its columns are where tiny steps along each input axis go, and is the local area scaling factor.
- The material derivative is the chain rule along a particle path: local change plus change from being carried somewhere else.
- Fluid equations use because Newton’s laws apply to parcels. The term is the nonlinearity of Navier–Stokes.