8.3 Fourier Series ended with a loose thread. We wrote and checked it numerically, but the square wave’s partial sums kept an overshoot: at , and still about at . This lesson says what "" means in that formula, why the overshoot cannot be removed by adding terms, and what to do about it. A spectral fluid solver stores each field as a truncated series, and the series next to a sharp front is the one studied here. 10.10 The Finite Volume Method ended with a related symptom in a different scheme; we will be careful about how far the relationship goes.
Transcript
- Last time the square wave's partial sums kept an overshoot. So what does the equals sign mean? Three things. Mean-square: the root mean square error goes to zero for every finite-energy function, jumps included. Pointwise: the sum approaches a number at each x separately. Uniform: the worst error over all x goes to zero. A jump rules that out, because every partial sum is continuous. Even the mean-square rate is slow: about 2 over pi root N plus 1, 0.1006 at 39 terms, 0.0563 at 127.
- Look next to the jump. The one-term sum peaks at 1.2732, three terms 1.2004, five terms 1.1884. At 39 terms the peak is 1.17919, at 127 terms 1.17901. As a share of the jump, which has size 2, that is 13.66 percent, 10.02, 9.42, 8.96, and 8.95. The peaks fall quickly, then stall.
- Why 1.179? Differentiate the sum: 2 over pi times sine of N plus 1 times x, over sine x. It first vanishes at x equals pi over N plus 1, the peak. The height is 2 over pi times the integral of sine s over s from 0 to pi, which is Si of pi. That is 1.851937, so the limit is 1.17898: an overshoot of 8.95 percent of the jump. At 5 terms, by hand, 1.18836.
- Why does it narrow but not shrink? The bump is one fixed shape, squeezed horizontally by the factor N plus 1. Its peak sits at pi over N plus 1: 0.07854 at 39 terms, 0.02454 at 127. At any fixed x above zero the bump passes by and the sum tends to 1. But at the moving peak the error is always 0.179: uniform convergence fails, drawn as a picture. It is no numerical artifact; it is what finitely many smooth waves do to a step.
- Two cures. Fejér averages the first N plus 1 partial sums, multiplying each term by 1 minus k over N plus 1. Its kernel is never negative, so it cannot overshoot: the maximum is 0.84883 at 3 terms, 0.98409 at 39, 0.99503 at 127. Lanczos averages over a window as wide as the bump, a sinc factor. That leaves a 1.19 percent overshoot, maximum 1.0237.
- How fast does the error fall? A jump in the m-th derivative gives rms error falling like N to the minus m plus a half: a straight line on a log-log plot. The square wave measures minus 0.487, the triangle minus 1.461, and the quartic, whose third derivative jumps, minus 3.453. A smooth function like exp of cosine x beats every power: a factor of 100 per two modes, 3.9 times 10 to the minus 3 at 3 terms, 3.2 times 10 to the minus 5 at 5.
- Now the exponential filter of spectral codes, with exponent p. A high order leaves most modes alone. At p equals 8 it keeps the front sharp and most of the ringing, 7.82 percent, while the error away from the jump falls to 3.7 times 10 to the minus 6. At p equals 2 the ringing is gone, the maximum is exactly 1.0000, but the front is smeared and the rms error is 0.1410. The plain sum has the smallest rms error, 0.0563: the ringing is the price of that optimality.
- Computers sample. With M points and the trapezoid rule, the coefficients become the discrete Fourier transform, an orthogonal change of basis. The catch is aliasing: on the grid, mode k and mode k plus M are the same samples, so modes beyond the Nyquist limit land on slower ones. A jump leaks about 1 over M into every retained mode. The direct sum costs M squared; splitting into even and odd samples gives the fast Fourier transform, M log 2 M. For M equal to 1024, roughly 100 times fewer steps.
- Spectral fluid solvers live on these facts. They use the FFT on a periodic box. A product of two fields creates modes up to twice the highest retained, so the 2 thirds rule zeroes the top third after each product. A shock rings, so codes filter every step or add viscosity, and the linear filter smears smooth regions too. The ringing resembles Lax-Wendroff's, 1.2129 and minus 0.2074 on a pulse of height 1, but not in mechanism: that is a dispersion error building up over time steps, while Gibbs sits in a frozen snapshot.
- To recap. Mean-square convergence always holds, pointwise gives the midpoint at a jump, uniform fails. The Gibbs bump rises to 1.17898, 8.95 percent of the jump, and narrows without shrinking. Smoothness sets the rate. Fejér is monotone but first order; high-order filters keep most of the ringing. Next, special functions: a pipe or a sphere gives Bessel functions and Legendre polynomials, and we ask which of these rules survive.
Three meanings of “converges”
A series can fail or succeed in three different ways.
Mean-square (L2) convergence means , where the root-mean-square error is the square root of the average of over a period. By Parseval (8.3 Fourier Series) the energy missed by is the tail of the coefficient sum, which goes to zero for every of finite energy, jumps included (completeness of sines and cosines; from memory). For the square wave the tail is , which an integral comparison (Calc II) approximates by , so
At this gives (the sim reads ) and at it gives (the sim reads ): the error goes to zero, but only like .
Pointwise convergence means a number, at each separately. Uniform convergence means the worst error over all goes to zero. Uniform implies pointwise, not the reverse. A Calc II fact decides the matter: a uniform limit of continuous functions is continuous. Every partial sum is continuous, so the series of a function with a jump cannot converge uniformly. Conversely, if is finite, the Weierstrass -test gives uniform convergence: the triangle wave has , summable, so it converges uniformly; the square wave has , which is not.
The Dirichlet theorem and the midpoint
What does the series do at a single point? Insert into and sum the geometric series inside the integral:
The partial sum is averaged against the Dirichlet kernel : a spike of height at with total weight , surrounded by sign-changing ripples. The ripples are the story of this lesson.
Dirichlet’s theorem (from memory, not proved here): if and are piecewise continuous, then at every . At a point where is continuous this is . At a jump it is the midpoint of the two sides. For the square wave at the midpoint is , and every term vanishes there, so the sim reads for every ; for the sawtooth at its jump it also reads , the average of .
Away from the jump convergence is slow but sure. At the square-wave sum is the Leibniz series , whose error is about half the first omitted term, . The sim gives , and then at and at , errors of and , matching and .
The Gibbs overshoot, derived
Why does the maximum stay near ? Differentiate the square-wave sum, for odd :
This vanishes first at , so the first maximum of sits there, and
using and for small . Since , the limit is . The bump rises above : an overshoot of about of the jump of size .
The sim’s maxima of at are , which as percentages of the jump are . The peak sits at : for and for . By hand at , peak at : , the sim’s value.
What to try. (1) Square wave, no filter, zoom on, . (2) The sawtooth: the same percentage of its jump , approached slowly ( at , at ) because is still rising under the bump. (3) Each filter. (4) Triangle and quartic: the slope of the rms line.
Why it narrows but does not shrink. The bump is a fixed shape squeezed horizontally by the factor . At any fixed it eventually passes by and (pointwise convergence), but at the error is always : the failure of uniform convergence, with a picture. The overshoot belongs to the representation, not to any numerical method: it is what approximating a step by a finite number of smooth waves does. It occurs, with the same , at every jump of every piecewise smooth function (from memory; the sim checks the square wave and, with a slope correction, the sawtooth).
Cure one: average away the ringing (Fejér)
Average the ripples of away. The Cesàro (Fejér) mean is the average of the first partial sums,
The term of frequency appears in for all , i.e. in of the sums, so the mean is the ordinary partial sum with each term multiplied by a sigma factor
Averaging the kernels likewise gives the Fejér kernel , which is never negative and has the same total weight as (from memory). A positively weighted average of values of lies between the smallest and largest of them: no overshoot is possible. The sim agrees: for the square wave the Fejér sum peaks at for , for and for , always under . For continuous it converges uniformly (Fejér’s theorem; from memory).
Cure two: a moving average (Lanczos), and other sigma factors
The Gibbs bump has width , so average over a window of half-width . Calc II gives , so the windowed average is again the partial sum with
This is the Lanczos sigma factor. Two others: the raised cosine and the exponential filter of even order , used in spectral codes ( makes the last mode round-off small; from memory). Each is a one-line change.
At on the square wave the sim gives:
| filter | max of sum | overshoot | rms error | worst error at least from a jump |
|---|---|---|---|---|
| none | ||||
| Fejér | none | |||
| Lanczos | ||||
| raised cosine | ||||
| exponential, | ||||
| exponential, | none | below |
Three lessons sit in this table. First, the plain sum has the smallest rms error: it is the orthogonal projection of onto the first modes (1.1 Vectors, Linear Combinations & Span), and any filter moves away from it. The ringing is the price of that optimality. Second, all but Fejér buy accuracy away from the jump: the worst error at least from every jump is plain but to filtered. Third, the order- exponential filter leaves most modes alone and keeps the front sharp, and with it most of the ringing (); the order- version kills the ringing by smearing the front.
Convergence rate against smoothness
The previous lessons said a jump in the -th derivative gives coefficients of size . The tail energy is about (integral comparison), so the rms error falls like . The sim fits the slope of against over to :
| function | jump in | predicted slope | measured |
|---|---|---|---|
| square wave | () | ||
| triangle | () | ||
| quartic | () | ||
| none | faster than any power | rms at , at |
For the smooth function the error drops by about a factor per two extra modes: this spectral (faster than any power) convergence is what spectral methods exploit. At the worst error is for the square wave and for the quartic.
Now the filters again. A smooth keeps an error of about . For Fejér, is of order even for , so the error cannot beat however smooth is: the measured slope is for the triangle, the quartic and alike. For Lanczos and the raised cosine, , so the best slope is (measured on the quartic and on ). For the order-8 exponential the best is : the sim measures on and keeps the full rates and on the triangle and quartic. The filter’s order is the order at which vanishes at .
Worked example: the Fejér mean by hand
At the sigma factors are and (the even terms are zero). At , and , so the Fejér sum is , which the sim reports as the maximum of the Fejér sum at . (By coincidence the plain has the same value.)
From series to samples: the DFT and the FFT
A simulation samples at equally spaced points and applies the trapezoid rule (2.5 Multiple Integrals) to . The result is the discrete Fourier transform
It is a change of orthogonal basis: is if is a multiple of and otherwise (a geometric sum). The same sum shows the catch: on the grid and are the same samples, so and every mode beyond (the Nyquist limit) is aliased onto a slower one. A smooth spectrum loses little; a jump, with , leaks about into every retained mode.
The direct sum costs multiplications. Splitting the samples into even and odd writes , with and the DFTs of the two half-size sequences. Repeating on each half gives levels of operations: the fast Fourier transform (Cooley and Tukey, 1965; from memory). For that is steps against , about times fewer.
Computing it
Now implement the sigma factors and the filtered sum yourself.
Concept checks
Problems
Find where the first maximum of the square-wave sum occurs and its value, and compare with the formula .
Show worked solution
. The first positive zero is , i.e. , which equals for . The value is . (The next critical point at is a minimum, .)
Use Parseval and an integral comparison to show that the rms error of for the square wave is about . Evaluate at .
Show worked solution
The mean square error is . The odd integers above are apart, so the sum is about . The mean square error is and the rms error ; at that is , against the sim’s .
(a) The previous lesson’s series for on has . Is the periodic extension continuous? Does its derivative jump? Estimate the rms error of .
(b) Compute the DFT of and verify Parseval’s identity .
Show worked solution
(a) The periodic extension of is continuous, since both ends equal , but its slope jumps from at to at : a corner, so and , consistent with the formula. The mean square error is , so the rms error is about , the slope of the triangle. (A direct numerical check at gives about against the estimate .)
(b) With : . . . . Then and . Energy is conserved by the change of basis, as in the continuous case.
Key Takeaways
- Mean-square convergence holds for any finite-energy (square wave: rms error at ); pointwise convergence holds by Dirichlet’s theorem, to the midpoint at a jump; uniform convergence fails at a jump.
- Gibbs: the first maximum sits at with limiting height , an overshoot of of the jump ( at ). It narrows but never shrinks.
- Fejér averaging () uses a positive kernel and cannot overshoot, but is only first-order accurate; the Lanczos factor leaves and the raised cosine .
- The plain sum is the orthogonal projection, best in mean square ( at against for Lanczos); filters improve the error away from the jump from to or better.
- A jump in the -th derivative gives rms error : measured slopes , , for jumps in , , . A filter of order caps the rate at .
- The DFT is a quadrature rule and an orthogonal change of basis, with aliasing beyond Nyquist; the FFT costs instead of .
- Spectral fluid codes filter and dealias for these reasons; their Gibbs ringing is the same kind of symptom as Lax-Wendroff ringing, not the same mechanism.
What comes next
Everything so far used sines and cosines, the right eigenfunctions for a constant-coefficient equation on a straight interval. A pipe or a sphere gives Sturm-Liouville problems (8.2 Eigenvalue Problems & Sturm–Liouville Theory) whose eigenfunctions are Bessel functions and Legendre polynomials. The next lesson, 8.6 Orthogonal Polynomials & Special Functions, builds series in those bases and introduces the Chebyshev polynomials that spectral methods use near walls. I expect mean-square convergence and the smoothness-rate rule to survive, but I will not assume it: the lesson will check what carries over (from memory: the Chebyshev case reduces to a cosine series by , which is where the FFT comes back in).