Part 0 · The Toolkit — Chapter 0.9

Fourier, Delta Functions, and Probability

The uncertainty principle, three parts before quantum mechanics, as a theorem about integrals.

Tools you'll need  — Chapter 0.2: integration by parts, the Gaussian integral $\int\ee^{-ax^{2}}\dd x=\sqrt{\pi/a}$ and its completed-square cousin, and Problem 4 (the transform of a Gaussian), which §6 cashes in explicitly. Chapter 0.3: Euler's formula, and Taylor expansion to second order with control of the remainder. Chapter 0.5: inner products, Cauchy–Schwarz with its equality condition, orthonormal bases and the coefficient formula $c_i=\avg{e_i,v}$, completeness $\sum_i\ket{e_i}\bra{e_i}=\hat I$, unitarity, and Dirac notation. Chapter 0.7: the divergence theorem and the diffusion equation. Chapter 0.8: that $\ee^{\lambda t}$ is the eigenfunction of $\dd/\dd t$, the driven oscillator, and the Duhamel/integrating-factor solution. Sections 3 and 6 each cash a specific theorem from Chapters 0.2 and 0.5. Nothing here is decoration.

1 · Fourier series as an orthonormal basis

Let's begin by giving functions the same treatment vectors got in Chapter 0.5. The plan for this section is short: define an overlap between two functions, check it is a genuine inner product, and then choose a basis. Chapter 0.5's table of inner-product spaces already had the row we need, and we skipped past it at the time. It was functions on an interval, with

Nothing new is being asserted here. This is the same definition Chapter 0.5 gave, with the vectors now being functions. Take the three requirements in turn.

• Conjugate symmetry, because $\overline{\int\bar g f}=\int\bar f g$.
• Linearity in the second slot, because integration is linear.
• Positive definiteness, because $\int\abs f^{2}\ge0$ and vanishes only for $f\equiv0$.

Chapter 0.4 warned that a space of functions was coming, and that it would be infinite-dimensional. Here it is.

Everything Chapter 0.5 proved about inner-product spaces is therefore available to us. What we do not yet have is a basis. We are about to choose one, and the choice is not a matter of taste. §3 shows that the functions below are precisely the eigenfunctions of $\dd/\dd x$.

1.1 · The orthogonality relation, by direct integration

Work on the interval $[-L/2,L/2]$ and define the wavenumbers

These are exactly the $k$ for which $\ee^{\ii kx}$ returns to its starting value after one interval length. Euler from Chapter 0.3 shows it in one line: $\ee^{\ii k_n(x+L)}=\ee^{\ii k_nx}\ee^{2\pi\ii n}=\ee^{\ii k_nx}$.

We want to know whether two of these waves are perpendicular, so compute their inner product. There are two cases. For $m=n$ the integrand is $\ee^{-\ii k_nx}\ee^{\ii k_nx}=1$, and the integral is just the length $L$. For $m\neq n$, write $\kappa = k_n-k_m = 2\pi(n-m)/L$, which is not zero, and integrate directly:

because $n-m$ is a non-zero integer and $\sin$ of an integer multiple of $\pi$ vanishes. (The middle step used $\ee^{\ii\theta}-\ee^{-\ii\theta}=2\ii\sin\theta$, which is Euler's formula subtracted from its own conjugate.) Both cases together:

That is the entire technical content of Fourier series. Divide by $\sqrt L$ to normalise and the functions

That is an orthonormal set in exactly Chapter 0.5's sense $\avg{e_i,e_j}=\delta_{ij}$, verified by integration rather than assumed.

1.2 · Coefficients are inner products

Chapter 0.5 proved that in an orthonormal basis, finding coordinates requires no work at all: $v=\sum_i\avg{e_i,v}e_i$. Transcribe it. Suppose $f=\sum_n c_ne_n$. Take $\avg{e_m,\cdot}$ of both sides and use (0.9.5):

Not one integral had to be solved and not one linear system inverted. The orthogonality did all the work. Now write that out and fold the $1/\sqrt L$ into the coefficient, and we arrive at the standard form:

1.3 · Completeness — quoted, and flagged as quoted

1.4 · A worked expansion, and an honest failure

Take $L=2\pi$, so $k_n=n$, and expand the square wave

Since $f$ is real and odd, it is easier to work with sines directly. Write $f=\sum_{n\ge1}b_n\sin nx$, where $\int_{-\pi}^{\pi}\sin mx\sin nx\,\dd x=\pi\delta_{mn}$. That orthogonality is the same computation as (0.9.4), rewritten with Euler.

Now find the coefficients. The integrand $f(x)\sin nx$ is a product of two odd functions, hence even, so the integral over $[-\pi,\pi]$ is twice the integral over $[0,\pi]$, where $f=+1$:

The bracket is $2$ for odd $n$ and $0$ for even $n$, so

Let's look at what that last line is saying. A discontinuous function, built entirely out of smooth ones. That should bother you slightly, and it should. No finite partial sum is discontinuous, so the discontinuity is manufactured only in the limit. The manufacturing process leaves a permanent scar.

The Gibbs phenomenon. Near the jump at $x=0$, every partial sum overshoots. Not by less and less: by the same amount, forever. The partial sum $S_M$ peaks at $1.178980\ldots$ rather than $1$, and as $M$ grows the overshoot narrows and slides toward the jump but never shrinks. Since the jump has size $2$, the overshoot is $8.949\%$ of the jump, and that number is a constant of nature for every jump discontinuity in every Fourier series. The grind box derives it.

This is a genuine failure of pointwise convergence, and it is worth naming as one rather than apologising for it. It is also not a failure of $L^{2}$ convergence: the overshoot region shrinks in width fast enough that $\int\abs{f-S_M}^{2}\to0$ regardless. The two notions of convergence disagree, and the one physics needs is the one that survives.

Grind box — where $1.178980$ comes from

Take the partial sum of (0.9.10) to $M$ terms (odd $n$ only) and differentiate it. The sum of cosines telescopes into something closed:

$$S_M'(x) = \frac4\pi\sum_{n\ \mathrm{odd}}^{M}\cos nx = \frac{2\sin(2Nx)}{\pi\sin x},$$

where $M=2N-1$. The closed form follows from summing the geometric series $\sum_{j=0}^{N-1}\ee^{\ii(2j+1)x}$ and taking real parts. Now integrate from $0$ to get back the partial sum itself:

$$S_M(x) = \frac{2}{\pi}\int_0^{x}\frac{\sin(2Nt)}{\sin t}\,\dd t.$$

The first maximum is at the first zero of the integrand's numerator past the origin, i.e. $x^{*}=\pi/2N$. For large $N$ that is small, so $\sin t\approx t$ throughout the range, and substituting $u=2Nt$:

$$S_M(x^{*}) \;\longrightarrow\; \frac{2}{\pi}\int_0^{\pi}\frac{\sin u}{u}\,\dd u = \frac{2}{\pi}\,\mathrm{Si}(\pi) = 1.178979\ldots$$

The number $\mathrm{Si}(\pi)=1.851937\ldots$ is not elementary. It is a value of the sine integral, and it is perfectly definite. Numerically, the peak of $S_M$ is $1.181302$ at $M=11$, $1.179103$ at $M=51$, $1.178988$ at $M=201$, $1.178980$ at $M=1001$. It is converging, and not to $1$.

Why it matters beyond aesthetics. To band-limit a signal is to truncate its Fourier content, and that is what every detector, every lens and every digital filter does. Any time you do it, you ring at the edges by $9\%$ of the step. Sharp edges and finite bandwidth are incompatible, and §6 makes that a theorem.

2 · From series to transform

Fourier series need a box. Physics on the whole line does not have one, so we take $L\to\infty$ and watch what survives.

2.1 · The limit, taken carefully

The allowed wavenumbers (0.9.2) are spaced by

So as the box grows, the modes crowd together, and in the limit they form a continuum. The sum over $n$ in (0.9.7) is going to become an integral over $k$. That works only if we first arrange for the sum to look like a Riemann sum, which means every term has to carry a factor of $\Delta k$.

Manufacturing that factor is the whole job of this subsection. Start by defining, for a function on the whole line,

and notice that the coefficient in (0.9.7) is $a_n = \sqrt{2\pi}\,\tilde f(k_n)/L$. Our goal is to get the series into Riemann-sum form, so substitute that expression for $a_n$ back into it:

Look at what appeared. The factor $2\pi/L$ that had to be manufactured is exactly the mode spacing (0.9.11). So the right-hand side is a Riemann sum, of the kind Chapter 0.2 defined the integral to be the limit of. Let $L\to\infty$ and the sum becomes that integral:

The two boxed formulas are the Fourier transform pair. Let's compare them with Chapter 0.5's $v=\sum_i\avg{e_i,v}e_i$. Then (0.9.12) is the coefficient $\avg{e_k,f}$, and (0.9.14) is the expansion, with the sum over a discrete index replaced by an integral over a continuous one. It is the same equation.

What has changed is that the basis is now labelled by a real number instead of an integer. That one change is the difficulty Chapter 4.5 has to work to make legitimate. The functions $\ee^{\ii kx}$ do not belong to the space they are supposed to be a basis of, because $\int\abs{\ee^{\ii kx}}^{2}\dd x=\int1\,\dd x=\infty$. Chapter 0.5's warning box said this would happen. §5 gives it the tool it needs.

2.2 · The convention, chosen and held

The $2\pi$ has to go somewhere and there is no canonical place to put it. We have chosen the symmetric convention: a factor $1/\sqrt{2\pi}$ on each of (0.9.12) and (0.9.14), and the sign $\ee^{-\ii kx}$ going forward, $\ee^{+\ii kx}$ coming back. We will hold it for the entire book.

Other texts choose differently, and this is a real and recurring source of confusion. Factors of $2\pi$ and $\sqrt{2\pi}$ appear and disappear between references for no mathematical reason whatsoever. Here are the three conventions you will meet.

Convention	Forward	Inverse	Convolution theorem
Symmetric (ours)	$\frac{1}{\sqrt{2\pi}}\!\int\! f\ee^{-\ii kx}\dd x$	$\frac{1}{\sqrt{2\pi}}\!\int\!\tilde f\ee^{\ii kx}\dd k$	$\sqrt{2\pi}\,\tilde f\tilde g$
Asymmetric	$\int f\ee^{-\ii kx}\dd x$	$\frac{1}{2\pi}\!\int\!\tilde f\ee^{\ii kx}\dd k$	$\tilde f\tilde g$
Ordinary frequency	$\int f\ee^{-2\pi\ii\nu x}\dd x$	$\int\tilde f\ee^{2\pi\ii\nu x}\dd\nu$	$\tilde f\tilde g$

The symmetric one is chosen here for one reason: it makes the transform unitary, which is the next result, and unitary is the property Chapter 4.6 needs.

One further wrinkle is worth flagging now, because it catches people. In time-dependent problems physics usually writes $\ee^{-\ii\omega t}$ for the forward direction in time, which is the opposite sign from our $\ee^{-\ii kx}$ in space. The reason is that it makes a plane wave read $\ee^{\ii(kx-\omega t)}$. That is a convention about which way waves travel rather than a different transform. We will flag it again when it first matters.

2.3 · Plancherel: the transform is a rotation

Chapter 0.5's Parseval identity said that in an orthonormal basis, the squared length of a vector is the sum of the squared magnitudes of its coordinates. We want to know what that statement becomes on the whole line, so take it into the box first and then to the limit.

In the box, expand $\int\abs f^{2}$ using (0.9.7), then collapse the double sum with the orthogonality (0.9.4). Only the diagonal terms survive:

Now we want the right-hand side in terms of the transform rather than the coefficients, so substitute $a_n=\sqrt{2\pi}\,\tilde f(k_n)/L$ and manufacture the mode spacing again:

Now look at the two ends of that chain. The left-hand side of the first line never depended on $L$ at all, so it needs no limit taken, and the right-hand side has just become an integral. Reading them together, we have proved Plancherel's theorem:

Let's read that structurally rather than as a formula. It says the map $f\mapsto\tilde f$ preserves norms. A norm-preserving linear map on an inner-product space preserves the inner product as well, because the inner product is recoverable from the norm. To see that, expand $\norm{f+g}^{2}$ and $\norm{f+\ii g}^{2}$ and solve. So the transform preserves overlaps too:

3 · Why Fourier, actually: the derivative becomes multiplication

Now the reason the subject exists. Everything so far could have been said about any orthonormal basis. This section says what is special about this one.

3.1 · The derivative theorem

We want to know what the transform does to a derivative, so transform $f'$ and integrate by parts, which by Chapter 0.2 is the product rule run backwards:

The boundary term is dropped, and the assumption behind dropping it should be said out loud: $f(x)\to0$ as $\abs x\to\infty$. Since $\abs{\ee^{-\ii kx}}=1$ exactly, the oscillating factor cannot help. The decay has to come from $f$ itself.

That assumption is worth translating, because it is a physical statement in disguise. It says the disturbance is localised. The wave packet is somewhere. The field dies away. The concentration is finite. It is the same hypothesis Chapter 0.2 flagged when it derived integration by parts, and granting it here leaves

and by iterating, $\widetilde{f^{(m)}}=(\ii k)^{m}\tilde f$. Differentiation has become multiplication.

3.2 · Why: the basis diagonalises $\dd/\dd x$

Do not accept (0.9.20) as a computational trick. It is the spectral theorem. Chapter 0.8 observed that exponentials are the eigenfunctions of the derivative, so write that observation in the form this chapter needs:

The operator $\dd/\dd x$ acting on the function $\ee^{\ii kx}$ returns the same function times the number $\ii k$. That is Chapter 0.5's $Av=\lambda v$ with $A=\dd/\dd x$, $v=\ee^{\ii kx}$ and $\lambda=\ii k$. So the Fourier basis is the eigenbasis of the derivative operator.

Which is why (0.9.20) is nothing but Chapter 0.5's observation that an operator, written in its own eigenbasis, is a list of numbers. Here the list is $\{\ii k\}$, indexed by a continuum rather than by an integer.

3.3 · A demonstration: the driven oscillator

Chapter 0.8 fought the damped driven oscillator with trial solutions and complex amplitudes:

Our goal is to turn that into an equation with no derivatives in it at all, so transform in $t$ with conjugate variable $\omega$, using (0.9.20) twice. Each time derivative becomes a factor $\ii\omega$:

The differential equation has become an algebraic equation, for each $\omega$ separately, and it is solved by division:

The function $\chi$ is the transfer function. It also goes by response function and by susceptibility, three names for one object. Everything Chapter 0.8 extracted by hand is now readable straight off it.

• The response amplitude is $\abs\chi$, which peaks where the denominator is smallest. That peak is resonance.
• The phase lag is $-\arg\chi$, which passes through $\pi/2$ at $\omega=\omega_0$.

One more reading is available if we look closely at the peak. Near resonance the denominator is approximately $2\omega_0(\omega_0-\omega)+2\ii\gamma\omega_0$, so

That is the Lorentzian of Chapter 0.8, with half-width-at-half-maximum $\gamma$, now derived rather than fitted. Problem 4 turns this same shape into the natural linewidth of an excited state, and the connection runs through §6.

Grind box — what "transform the equation" is really doing, and two traps

It is a change of basis, and only that. In Chapter 0.5's language, an equation $\mathcal Ly=F$ in the position basis becomes, after inserting completeness, an equation between components in the $k$ basis. Since $\mathcal L$ is diagonal there, the matrix equation $\sum_j L_{ij}y_j=F_i$ degenerates to $L_{ii}y_i=F_i$ with no sum at all. That is one equation per mode, each solved by dividing. There is no new mathematics in §3. There is a change of coordinates.

Trap 1: initial conditions. Transforming in $t$ over the whole line quietly assumes the solution exists for all time and decays at both ends. That is the right thing to want for a steady-state response, and the wrong thing for an initial-value problem, where the system is switched on at $t=0$ and you are told $y(0)$ and $\dot y(0)$. For those, the correct tool is the Laplace transform. It is the same idea with $\ii\omega\to s$ and the integral run over $[0,\infty)$, and that change of limits produces the initial data as boundary terms in the integration by parts instead of discarding them. Everything in this chapter has a Laplace twin.

Trap 2: $P(\ii k)$ can vanish. Dividing by $P(\ii k)$ is illegal where $P$ has a zero on the real axis, which is exactly the undamped resonance $\gamma=0$, $\omega=\omega_0$. The division then fails because the answer genuinely does not exist: an undamped oscillator driven exactly at resonance has no bounded steady state, it grows linearly forever. The formula's failure is reporting a physical fact. When damping is present, $P$ has no real zeros and the division is safe. That is why $\gamma\gt0$ was doing more work in Chapter 0.8 than it looked.

4 · Convolution and Green's functions

Definition. The convolution of two functions is

Read it as: at each point $x$, take a weighted average of $f$, with the weights given by $g$ centred on $x$ and reflected. The operation is symmetric. Substituting $u=x-y$ turns (0.9.26) into $\int g(u)f(x-u)\dd u$, so $f*g=g*f$. It is bilinear as well, which makes it a genuine product on the space of functions.

4.1 · The convolution theorem, derived

Transform (0.9.26) and split the exponential using $x=y+(x-y)$:

The second line substituted $u=x-y$ at fixed $y$. That is a shift, so $\dd u=\dd x$ and the limits are unchanged. The double integral now factorises, because the integrand is a product of a function of $y$ alone and a function of $u$ alone:

The stray $\sqrt{2\pi}$ is the price of the symmetric convention, and the table in §2.2 lists what it becomes in the others. Get it wrong and every amplitude in Part V is off by a constant factor, so it is worth one moment of care now.

Now the headline. Convolution in $x$ is multiplication in $k$. An operation that costs a full integral at every single point becomes a pointwise product.

4.2 · Linear time-invariant systems, and Green's functions

Here is why (0.9.28) is one of the most load-bearing facts in physics. Consider any system that is linear, meaning that doubling the input doubles the output and that inputs superpose, and time-invariant, meaning that its behaviour does not depend on what o'clock it is. Feed it an input $F(t)$ and call the output $y(t)$.

Let $G(t)$ be the output produced by a single sharp unit impulse at $t=0$. That output carries two names, the impulse response and the Green's function. Now put the two hypotheses to work. Time-invariance says an impulse at $t'$ produces $G(t-t')$. Linearity says a general input, regarded as a dense sequence of impulses of strength $F(t')\dd t'$, produces the superposition of the individual responses:

That is the whole content of linear response theory, and it is a convolution. Transform it with (0.9.28) and it becomes $\tilde y=\sqrt{2\pi}\,\tilde G\tilde F$. Now set that beside (0.9.24) and read off that the transfer function is $\chi=\sqrt{2\pi}\,\tilde G$. The transfer function is the transformed Green's function. Two objects, one thing.

Chapter 0.8 already built one of these and said so. Its integrating-factor solution of $\dot y+\lambda y=F(t)$ (Chapter 0.8 calls the rate constant $k$, but here $k$ is taken, so it is $\lambda$) was

with $G(u)=\ee^{-\lambda u}$ for $u\gt0$ and $G(u)=0$ for $u\lt0$. The upper limit $t$ in Chapter 0.8's formula and the vanishing of $G$ for negative argument are the same statement, and that statement is causality: the response cannot precede the impulse. Duhamel's formula was a convolution all along.

5 · The Dirac delta, handled honestly

Two debts have been quietly accumulating. §4 needed "a single sharp unit impulse" and did not define one. §2 needed a completeness relation for a continuum of basis functions and did not have one. The same object settles both debts, and that object is not a function.

That last sentence is the whole difficulty of this section, so this is the plan. First we define the thing by what it does rather than by what it is. Then we show it is the narrow-bump limit Chapter 0.2 was already drawing. Then we derive its rules, and at each step you will see why the "not a function" caveat has to be respected rather than merely noted.

5.1 · Define it by what it does

Do not ask what value $\delta(x)$ takes at each point. There is no answer, and §7's warning box shows in detail why the question is malformed. Ask instead what the object does to other functions. The Dirac delta is defined by

for every sufficiently well-behaved test function $f$. That is the definition, complete.

Now notice what kind of object that makes $\delta$. It is a rule that eats a function and returns a number, and it does so linearly, since $\int\delta(\alpha f+\beta g)=\alpha f(0)+\beta g(0)$. A linear map from functions to numbers is a linear functional, which is precisely Chapter 0.5's definition of a bra. Objects defined this way are called distributions, and $\delta$ is the simplest non-trivial one.

Shifting the argument gives the general sifting property, by substituting $u=x-a$:

The delta reaches into a function and pulls out its value at one point, discarding everything else. Every use of $\delta$ in this book is that sentence.

5.2 · It is the limit Chapter 0.2 was drawing

Take the normalised Gaussian, whose total area is $1$ by Chapter 0.2's $\int\ee^{-ax^{2}}\dd x=\sqrt{\pi/a}$:

This is exactly the curve in Chapter 0.2's width figure. It is the one whose measured area stayed pinned at $1.00000$ while the peak rose as $1/\sigma$, and whose caption said that pushing $\sigma$ to its minimum was watching the Dirac delta form. It was. Here is that claim made precise. For continuous $f$,

Proof. What we have to drive to zero is the difference between the two sides. Since $\int\delta_\sigma=1$, we may write that difference as

Fix $\varepsilon\gt0$. By continuity there is an $\eta\gt0$ with $\abs{f(x)-f(0)}\lt\varepsilon$ for $\abs x\lt\eta$. That part of the integral is therefore smaller than $\varepsilon\int\delta_\sigma=\varepsilon$. Outside $\abs x\lt\eta$, the factor $\delta_\sigma$ is at most $\ee^{-\eta^{2}/2\sigma^{2}}/(\sigma\sqrt{2\pi})$, which goes to zero faster than any power as $\sigma\to0$, so (for bounded $f$) that part vanishes in the limit. Hence the whole thing is below $2\varepsilon$ for small enough $\sigma$, and $\varepsilon$ was arbitrary. $\blacksquare$

Two things are worth extracting from that proof, and the first is the one that will keep you out of trouble.

The limit is taken after integrating, never before. Take it before and you get $\lim_\sigma\delta_\sigma(x)$ pointwise, which is $0$ for $x\neq0$ and $+\infty$ at $x=0$. That object carries no information at all: it is zero almost everywhere, so any integral of it against anything is zero, and the one point where it is not zero contributes nothing. The order of the two operations is what the delta is. This is why every rule in §5.4 is proved by testing both sides against an $f$, rather than by comparing values point by point, and it is why the delta lives inside an integral sign and nowhere else.

The Gaussian was not special. Any normalised bump that narrows will do. Chapter 0.2's grind box on differentiating under the integral sign exhibited another one, $2x\ee^{-x^{2}/a^{2}}/a^{2}$ on the half-line, and called it "the Dirac delta being born". It was.

5.3 · The Fourier representation, and what it really is

The single most-used identity in Parts IV–VII is

which cannot be true as it stands, since the integrand has modulus $1$ everywhere and the integral does not converge. Here is the derivation that makes it respectable, and it uses nothing beyond Chapter 0.2. There are three steps. Insert a convergence factor $\ee^{-\epsilon k^{2}}$ with $\epsilon\gt0$, do the now-honest Gaussian integral with $b=\ii x$, and take $\epsilon\to0$ at the end:

The regulated integral is a normalised Gaussian of width $\sqrt{2\epsilon}$. That is precisely (0.9.33), which tends to $\delta$ by §5.2. So (0.9.36) is true in the only sense in which it was ever meant: as a limit of honest integrals, evaluated under an integral sign.

The manoeuvre was to insert a regulator, compute, and then remove the regulator. It is not a dodge to be embarrassed about. It is the same move that will be called renormalisation in Chapter 5.10, met here in its harmless form.

Now let's see what (0.9.36) actually is. Chapter 0.5's completeness relation reads $\sum_n\ket{e_n}\bra{e_n}=\hat I$. In function language, with $e_n(x)$ the components, that says $\sum_n e_n(x)\overline{e_n(y)}=\delta(x-y)$, since acting on $f$ it has to return $f$. So put in the Fourier basis (0.9.5), $e_n=\ee^{\ii k_nx}/\sqrt L$, and manufacture the mode spacing one last time:

So (0.9.36) is the completeness relation $\sum_n\ket{e_n}\bra{e_n}=\hat I$ in continuous disguise. That is the honest reading, and it explains why the delta appears everywhere the Fourier transform does: every time you insert the identity in a continuous basis, a delta drops out. In Dirac notation the whole of §2 compresses to $\avg{x|k}=\ee^{\ii kx}/\sqrt{2\pi}$ and $\int\ket{k}\bra{k}\dd k=\hat I$, and (0.9.36) is $\avg{x|y}=\delta(x-y)$.

5.4 · The rules, derived

Scaling. Claim: $\delta(ax)=\delta(x)/\abs a$ for $a\neq0$. Test it against an arbitrary $f$ and substitute $u=ax$. If $a\gt0$ then $\dd x=\dd u/a$ and the limits keep their order:

If $a\lt0$ the substitution reverses the limits, and swapping them back supplies a minus sign, so the answer is $-f(0)/a=f(0)/\abs a$. Both cases give $f(0)/\abs a$, which is what $\int\big[\delta(x)/\abs a\big]f(x)\dd x$ gives. Since the two functionals agree on every test function, they are equal:

Note the consequence: $\delta$ is not dimensionless. If $x$ carries units of length then $\delta(x)$ carries units of $1/\text{length}$, so that (0.9.31) balances. Checking that in an unfamiliar formula catches an impressive number of errors.

The derivative. We would like $\delta'$ to obey integration by parts. So define it to, discarding the boundary term (test functions vanish at infinity):

This is how every distribution is differentiated, and it is why distributions are so useful. They can always be differentiated, as many times as you like, with no smoothness required of anything. The step function has a derivative, and that derivative is $\delta$. Then $\delta$ has a derivative of its own, and so on forever. Chapter 5.2's field equations lean on this constantly.

The three-dimensional delta, and Chapter 0.7's loose end. Define $\delta^{3}(\vv r)=\delta(x)\delta(y)\delta(z)$, so that $\int\delta^{3}(\vv r)f(\vv r)\,\dd^{3}r=f(\vv 0)$. Chapter 0.7 left a warning here. The Coulomb potential $1/r$ satisfies $\nabla^{2}(1/r)=0$ everywhere the computation is legal, and yet a point charge is manifestly a source. Both halves are true, and the resolution is a distribution.

Take the two halves in turn. Direct differentiation away from the origin gives $\nabla^{2}(1/r)=0$ for $r\neq0$. Now instead integrate over a ball of any radius $R$ and use the divergence theorem of Chapter 0.7, with $\nabla(1/r)=-\hat{\vv r}/r^{2}$:

An object that vanishes everywhere except one point yet integrates to $-4\pi$ over every ball containing that point is, by (0.9.31), exactly $-4\pi$ times a delta:

Chapter 0.7's loose end is tied. Now read (0.9.43) once more in the language of §4. It says that $-1/4\pi r$ is the Green's function of the Laplacian, the response to a unit point source. Poisson's equation $\nabla^{2}\phi=-\rho/\epsilon_0$ is therefore solved by convolution with it, and that convolution is

That is Coulomb's law, recovered as a special case of §4.2 rather than posited.

Grind box — distributions, said properly

The construction that makes all of §5 rigorous is due to Schwartz and is worth seeing once, in outline, because it explains exactly which manipulations are safe.

Test functions. Fix a space $\mathcal D$ of very well-behaved functions. They are infinitely differentiable, and either compactly supported or, in the Schwartz space $\mathcal S$, decaying faster than any power along with all their derivatives. $\mathcal S$ is the natural home for Fourier analysis, because (0.9.20) and its partner $\widetilde{xf}=\ii\,\dd\tilde f/\dd k$ trade decay for smoothness in both directions, so the transform maps $\mathcal S$ onto itself.

Distributions. A distribution is a continuous linear map $T:\mathcal D\to\C$. Ordinary functions embed: $g$ becomes $T_g[f]=\int gf$. Some distributions, like $\delta$, are not of that form for any $g$, and that is the precise sense of "$\delta$ is not a function".

Everything is defined by moving the operation onto the test function. Derivative: $T'[f]\equiv -T[f']$. Scaling and translation: as in §5.4. Fourier transform: $\tilde T[f]\equiv T[\tilde f]$. Applied to $\delta$, that last one gives $\tilde\delta[f]=\delta[\tilde f]=\tilde f(0)=\frac{1}{\sqrt{2\pi}}\int f$, so $\tilde\delta=1/\sqrt{2\pi}$, a constant. A perfectly sharp spike in $x$ has perfectly flat content in $k$. Hold that thought for §6.

What is not defined. There is no general product of two distributions. The recipe above works because differentiating or transforming a test function gives another test function. Multiplying two distributions has no such move available, and $\delta^{2}$ is genuinely undefined. Not hard, not subtle, undefined. See the warning box.

6 · The bandwidth theorem

This is what the chapter was for. Here is the plan, in plain English, before we start. We give a function a width and give its transform a width, using the ordinary statistical definition of spread. Then we show the two widths cannot both be small, and we get the exact number they are bounded by.

6.1 · Defining a width

Let $f$ be normalised, $\int\abs f^{2}\dd x=1$, which by Plancherel (0.9.17) means $\int\abs{\tilde f}^{2}\dd k=1$ as well. Then $\abs f^{2}$ and $\abs{\tilde f}^{2}$ are both non-negative and both integrate to one: they are probability densities, whether or not anybody intends them as such. So give each the standard deviation of §7:

with $\avg x=\int x\abs f^{2}\dd x$ and $\avg k=\int k\abs{\tilde f}^{2}\dd k$.

We may assume $\avg x=\avg k=0$. That is not a convenience granted to ourselves. It is a fact, and it takes two lines.

• Replacing $f(x)$ by $f(x+x_0)$ shifts $\abs f^{2}$ bodily and changes $\tilde f$ only by the phase $\ee^{\ii kx_0}$. So $\Delta x$ and $\Delta k$ are both untouched, while $\avg x$ moves by $-x_0$.
• Replacing $f(x)$ by $\ee^{-\ii k_0x}f(x)$ leaves $\abs f^{2}$ alone and shifts $\tilde f(k)$ to $\tilde f(k+k_0)$. That moves $\avg k$ by $-k_0$, again with $\Delta k$ untouched.

So both means can be driven to zero without changing either spread, and we do so.

6.2 · The move that makes it work

The difficulty is that $\Delta x$ is an integral over $x$ while $\Delta k$ is an integral over $k$. They live in different places, so as things stand there is nothing to compare. Our goal is to get both of them onto the same axis, and §2 and §3 together do exactly that. Use the derivative theorem (0.9.20) first, then Plancherel (0.9.17):

Let's pause on that line, because it is the crux. The spread in $k$ has become a statement about $f$ alone: it is the size of the derivative of $f$. In hindsight the shape of it makes sense. A function with a lot of high-$k$ content is a function that wiggles, and a function that wiggles has a big derivative. What we have now is an identity rather than an intuition. Both integrals run over $x$, so we may compare them.

6.3 · Cauchy–Schwarz

Apply Chapter 0.5's inequality $\abs{\avg{u,v}}^{2}\le\avg{u,u}\avg{v,v}$ in the function space with the inner product (0.9.1), to the two vectors

Their norms are exactly the two quantities we want, by (0.9.45) and (0.9.46):

So the theorem is proved as soon as we can put a lower bound on the left-hand side. Call it $I=\int x\,\bar f f'\,\dd x$ (the $x$ is real, so the conjugation does nothing to it) and add $I$ to its own conjugate. The two terms assemble into a derivative by the product rule:

Now integrate by parts (Chapter 0.2). The boundary term drops because $x\abs{f(x)}^{2}\to0$ at both ends. That is a hypothesis rather than a triviality, and the grind box justifies it:

That is the whole calculation. The normalisation $\int\abs f^{2}=1$, which looked like tidiness, has turned into the number on the right-hand side.

6.4 · The theorem

A complex number is at least as big as its real part in absolute value, so from (0.9.50),

Feed that into (0.9.48) and take the square root of both sides:

Done. Let's list what went into it, because the shortness of the list is the point.

• The definition of the transform (§2).
• Plancherel (§2).
• The derivative theorem (§3).
• Cauchy–Schwarz (Chapter 0.5).
• Integration by parts (Chapter 0.2).
• The normalisation.

There is nothing else in it.

6.5 · The Gaussian saturates it

We want to know which functions actually achieve the bound, so run the argument backwards from its two inequalities. Chapter 0.5 recorded the equality condition for Cauchy–Schwarz: equality holds exactly when the two vectors are parallel. Here that reads $f'=\lambda\,x f$ for some constant $\lambda$.

The other inequality was (0.9.51), and equality there needs $\abs I=\abs{\mathrm{Re}\,I}$, which forces $I$ to be real. Since $I=\lambda\int x^{2}\abs f^{2}=\lambda(\Delta x)^{2}$, that in turn forces $\lambda$ to be real. Now solve $f'/f=\lambda x$ by separation (Chapter 0.8). It gives $\ln f=\tfrac12\lambda x^{2}+\text{const}$, so

The minimisers are exactly the Gaussians, and nothing else. This is stronger than the claim that Gaussians happen to be one example. The derivation ran backwards from the equality condition and produced them uniquely.

Now verify the value directly, cashing Chapter 0.2's Problem 4. Write $\lambda=-1/2\sigma^{2}$, so $f=C\ee^{-x^{2}/4\sigma^{2}}$ and $\abs f^{2}\propto\ee^{-x^{2}/2\sigma^{2}}$, which by Chapter 0.2's Worked example 2 has $\Delta x=\sigma$. Its transform, by $\int\ee^{-ax^{2}}\ee^{-\ii kx}\dd x=\sqrt{\pi/a}\;\ee^{-k^{2}/4a}$ with $a=1/4\sigma^{2}$, is

We now have the same function described both ways, so the two widths can finally be multiplied together. Squaring the transform gives $\abs{\tilde f}^{2}\propto\ee^{-2\sigma^{2}k^{2}}=\ee^{-k^{2}/2(1/2\sigma)^{2}}$, and matching that against the standard Gaussian shape gives $\Delta k=1/2\sigma$. Therefore

The bound is attained, so (0.9.52) cannot be improved. Squeeze the Gaussian and its transform broadens by exactly the reciprocal factor. The product does not move. That is a thing you should now watch happen.

shape Gaussian square pulse two humps

width

w = 0.60

Δx = 0.600

Δk = 0.833

Δx·Δk = 0.500

Gaussian: the bound is attained. Drive the slider — the panels change, the product does not.

Grind box — the hypotheses, said out loud

The boundary term. We assumed $x\abs{f(x)}^{2}\to0$ at $\pm\infty$. Here is why it is automatic whenever the theorem's quantities are finite. By Cauchy–Schwarz, $\int\abs{x}\abs f^{2}\le(\int x^{2}\abs f^{2})^{1/2}(\int\abs f^{2})^{1/2}=\Delta x\lt\infty$, so $x\abs f^{2}$ is integrable. Its derivative is $\abs f^{2}+2x\,\mathrm{Re}(\bar f f')$, whose second piece is bounded in integral by $2\Delta x\,\Delta k\lt\infty$, so the derivative is integrable too. A function whose derivative is integrable has limits at $\pm\infty$. A function that is itself integrable and has limits must have those limits equal to zero. So the boundary term vanishes for exactly the functions for which the statement has content.

When the spreads are infinite. If $\Delta x=\infty$ or $\Delta k=\infty$ the inequality is vacuously true, and this is not a rare corner: any function with a jump discontinuity has $\Delta k=\infty$ (Problem 1), and any function with power-law tails $\abs f^{2}\sim\abs x^{-3}$ has $\Delta x=\infty$. The theorem is sharp only among functions smooth enough and localised enough to have both moments. Outside that class one uses different width measures: the width at half maximum, the width containing $90\%$ of the power, the reciprocal of the peak height. All of them obey inequalities of the same shape with different constants, and none of those constants is $\tfrac12$.

Why the standard deviation, then? Because it is the measure for which the constant is exactly $\tfrac12$ and the minimiser is exactly the Gaussian, and because it is the one Chapter 4.9 needs. There, $(\Delta x)^{2}$ is a variance of measurement outcomes, and it has to be that and nothing else.

7 · Probability, and why it is the same subject

Probability enters this chapter for a structural reason and not a curricular one: the central limit theorem is a Fourier argument, and the object that makes it one is a Fourier transform under another name.

7.1 · The apparatus, briefly

A random variable $X$ taking real values is described by a probability density $p(x)\ge0$ with $\int p=1$, meaning $\Pr(a\le X\le b)=\int_a^bp$. Its cumulative distribution is $F(x)=\int_{-\infty}^{x}p(u)\dd u$.

By the fundamental theorem of calculus (Chapter 0.2), those two are related by $F'=p$. The density is the derivative of the CDF, and that one sentence is what makes both objects worth having. Next, the expectation of any function is

which is linear in $g$ because integration is. The mean is $\mu=\mathbb E[X]$, the variance is $\mathrm{Var}(X)=\mathbb E[(X-\mu)^{2}]$, and expanding the square with linearity gives the form you actually compute with:

The standard deviation is $\sigma=\sqrt{\mathrm{Var}}$. That is the same quantity §6 called a spread, computed from the same integral. And the Gaussian density is

whose normalising constant is Chapter 0.2's $\int\ee^{-ax^{2}}\dd x=\sqrt{\pi/a}$ with $a=1/2\sigma^{2}$, and whose variance is $\sigma^{2}$ by Chapter 0.2's Worked example 2. Nothing in (0.9.58) is quoted.

7.2 · The characteristic function is a Fourier transform

Define the characteristic function of $X$:

It is the Fourier transform of the density, up to the convention's constant and a sign. Three properties follow at once, and all three get used below.

• It always exists, because $\abs{\ee^{\ii tx}}=1$ and $p$ is integrable. Moments are not like that. A moment can be infinite, and the grind box at the end of §7 turns on a case where one is.
• It determines $p$ uniquely, because the transform is invertible by (0.9.14).
• Its Taylor coefficients are the moments, because differentiating under the integral (Chapter 0.2) brings down factors of $\ii x$.

Writing that last property out to second order:

7.3 · Independent sums convolve, so characteristic functions multiply

Let $X$ and $Y$ be independent, with densities $p_X,p_Y$, and set $Z=X+Y$. Independence means the joint density factorises, $p(x,y)=p_X(x)p_Y(y)$. Then

and differentiating with respect to $z$ under the integral, using $F'=p$ on the inner integral:

Adding independent random variables convolves their densities. That is a surprising fact with a one-line derivation, and it is the bridge we came here for, because §4's convolution theorem now applies to probability.

So transform (0.9.62), then translate the result back into characteristic functions with (0.9.59). The two stray factors of $\sqrt{2\pi}$ cancel against the one in (0.9.28), and what is left is

A hard operation on densities, an integral at every point, has become multiplication of two ordinary functions. This is (0.9.28) and nothing more, and it is the entire reason the central limit theorem is provable.

7.4 · Variance adds, and the $1/\sqrt n$

Take $\mu_X=\mu_Y=0$ without loss of generality (subtract the means). Then

and the cross term factorises under independence, $\mathbb E[XY]=\iint xy\,p_X(x)p_Y(y)=\mathbb E[X]\,\mathbb E[Y]=0$. Hence

Variances add. Standard deviations do not. Everything below is bookkeeping on that one line.

Take $n$ independent copies, each of variance $\sigma^{2}$. Their sum has variance $n\sigma^{2}$. And since $\mathrm{Var}(cX)=c^{2}\mathrm{Var}(X)$ follows directly from the definition, the mean $\bar X_n=\frac1n\sum X_i$ has

The standard error falls as $1/\sqrt n$, and the exponent $\tfrac12$ is not a convention or an empirical rule. It is the square root of "variances add", which is (0.9.65), which is the vanishing of a cross term.

7.5 · The central limit theorem, sketched honestly

Let $X_1,X_2,\dots$ be independent with the same distribution, mean $\mu$, finite variance $\sigma^{2}$. Standardise:

Write $Y=X-\mu$, so $\mathbb E[Y]=0$ and $\mathbb E[Y^{2}]=\sigma^{2}$. By (0.9.63), and since $\varphi_{cY}(t)=\varphi_Y(ct)$ straight from the definition,

Take logarithms, which turns the power into a product and is the move that makes the whole thing work. Expand $\varphi_Y$ to second order using (0.9.60) with $\mu_Y=0$, then expand $\ln(1+w)=w-\tfrac12w^{2}+\cdots$ from Chapter 0.3:

Watch what happened, because it is the entire theorem. The quadratic term carried exactly one power of $1/n$, so multiplying by $n$ left it finite and unchanged. Every higher term carried a higher power of $1/\sqrt n$, so multiplying by $n$ still left it going to zero. The quadratic term is the unique survivor. It survives not because we truncated anything, but because the scaling by $\sqrt n$ in (0.9.67) was chosen precisely to let it survive. Hence

All that remains is to recognise the answer. By Chapter 0.2's Gaussian integral with $a=\tfrac12$ and $b=\ii t$, the function $\ee^{-t^{2}/2}$ is the characteristic function of the standard normal density $\ee^{-x^{2}/2}/\sqrt{2\pi}$, as you can check by doing that integral. So the limit is a Gaussian. And it is a Gaussian because $\ee^{-t^{2}/2}$ is the fixed point of "square and rescale", which is what (0.9.68) does.

What is not proved here. Two things, and here they are.

• The remainder $O(n^{-3/2})$ needs control uniform in $n$. Doing that properly requires either a third moment or a truncation argument, and the honest general statement is Lindeberg's.
• Convergence of characteristic functions implies convergence of distributions. That is Lévy's continuity theorem, and we are quoting it.

Neither gap affects the mechanism. The Gaussian appears because the quadratic term is the only one that survives the $\sqrt n$ scaling, and that part is fully derived above.

Grind box — when the central limit theorem is false

The hypothesis $\sigma^{2}\lt\infty$ is doing real work. Drop it and the conclusion fails, in an instructive way.

Take the Cauchy density $p(x)=\dfrac{1}{\pi(1+x^{2})}$, whose tails fall as $x^{-2}$. That makes $\int x^{2}p\,\dd x$ diverge, so there is no variance to speak of. Its characteristic function is $\varphi(t)=\ee^{-\abs t}$ (⚑ a contour integral, quoted here, and the only place in this book that uses complex analysis, which Chapter 5.4 builds). Then by (0.9.63) the mean of $n$ samples has

$$\varphi_{\bar X_n}(t) = \left[\ee^{-\abs{t}/n}\right]^{n} = \ee^{-\abs t},$$

the characteristic function of a single Cauchy sample. Averaging a million Cauchy observations gives you exactly the precision of one observation. The $1/\sqrt n$ is gone, because (0.9.65) never applied: there was no variance to add. Note also that $\ee^{-\abs t}$ is not differentiable at $t=0$. The failure of the mean to exist is visible as a corner in the characteristic function, exactly where (0.9.60) would have read it off.

This is not a pathology invented to embarrass the theorem. Cauchy tails arise from ratios of normal variables, from resonance line shapes (the Lorentzian of (0.9.25) is a Cauchy density), and in any setting where the largest single contribution is comparable to the sum of the rest. When you see a sample mean that refuses to stabilise as data accumulate, this is the first thing to suspect.

8 · Worked examples

9 · Your turn