Part 0 · The Toolkit — Chapter 0.1

What a Derivative Really Is

The single most important reframe in the book, and it happens on page one.

Tools you'll need  · None. This is the ground floor.

1 · The question that forces the idea

Physics is almost entirely the study of things that change. The first thing we need, then, is a way of saying how fast something changes. That sounds trivial until you try to be precise about it, and this section is about where the trouble lies.

Suppose a particle's position is $x(t)$. Between times $t$ and $t+h$ it moves a distance $x(t+h)-x(t)$. Divide that by the time it took and you have the average velocity over the stretch:

There is no controversy in that. The trouble starts when you ask for the velocity right now, at the single instant $t$.

The obvious move is to set $h=0$ in (0.1.1). Do it and you get $0/0$, which is not a number. Yet an object clearly does have a speed at each moment. Your speedometer reads something. So there is a real quantity here, and the naive algebra cannot reach it.

The way out is to refuse to set $h=0$ at all. We ask instead what the ratio approaches as $h$ shrinks. That evasive-sounding manoeuvre is the whole of differential calculus.

It works for a reason worth naming now. The numerator and the denominator both go to zero, and the numerator does it no more slowly than the denominator does. Their ratio therefore settles down to something finite rather than blowing up.

separation h

h = 1.585

Δf/Δx = 0.69868 → f′(a) = 1.04030

2 · The definition, stated carefully

Let's write the definition down, and then spend a moment on the one clause in it that carries all the weight.

The derivative of $f$ at the point $a$ is

whenever that limit exists.

That last phrase is doing real work, and it is not a formality. It means there is a single number $L$ such that the quotient can be forced as close to $L$ as you like, just by making $h$ small enough. It means one thing more as well, and that is the part which matters. The same $L$ has to come out whether you approach from the left or from the right.

3 · The reframe: derivative as linear approximation

Our goal in this section is to get the definition into a form that mentions no graph at all. The route is two lines of algebra applied to (0.1.2), and we will take them one at a time.

Saying that the quotient tends to $f'(a)$ is the same as saying that the difference between the quotient and $f'(a)$ tends to zero. Give that difference a name, $\varepsilon(h)$:

What we want in the end is a statement about $f(a+h)$ itself rather than about a quotient, so multiply that line through by $h$ and move the terms around:

Read (0.1.4) slowly, because it is the sentence this entire book is built on. Here is what it says. Near $a$, the function $f$ is a constant plus a linear function of the displacement. The error you make by stopping there is not merely small. It is small compared to $h$ itself.

That last clause is the sharp part. Lots of approximations have small errors. This one has an error that vanishes faster than the thing you are approximating.

And $f'(a)$ is the unique number with that property. No other choice of coefficient makes the leftover shrink faster than linearly. Uniqueness is what lets us turn the whole thing around and use it as the definition:

3.1 · Seeing it: local flatness

The claim in (0.1.4) is one you can watch happen, and the figure below lets you do it.

The reasoning behind the figure runs like this. If the error really does die faster than $h$, then zooming in on the graph near $a$ must make the curve and its tangent line converge. Not merely look similar. They should become indistinguishable in a way that keeps improving the harder you look.

half-window

h = ±4.0e-1

error/h = 7.99e-2

error/h² = 0.2008 (½f″(a) = 0.1707)

4 · The rules, derived

Almost every differentiation rule you were once asked to memorise falls out of (0.1.4) in a line or two. We will take three of them here, in the order sum, product, chain.

Deriving them once is worth more than memorising them ten times. The reason is practical rather than moral. The derivations are the part that carries over into settings where the memorised forms mean nothing.

4.1 · Sum

Linear approximations add, and that is the whole of it. If $f \approx f(a)+f'(a)h$ and $g\approx g(a)+g'(a)h$, then $f+g \approx (f+g)(a) + \big(f'(a)+g'(a)\big)h$.

The errors still need checking, since the strong sense of §3 is what we are claiming. Each of the two error terms is smaller than $h$, so their sum is smaller than $h$ as well. Hence $(f+g)' = f'+g'$, and the same argument gives $(cf)'=cf'$.

4.2 · Product — with a picture

Let $A = uv$, and picture it as the area of a rectangle with sides $u$ and $v$. That picture is exact rather than decorative, and it will hand us the answer before we do any algebra.

Nudge $x$ by $h$. The sides grow by $\Delta u$ and $\Delta v$, so the area grows by three pieces: a strip along the top, a strip along the side, and a tiny square in the corner where the two growths overlap.

We are after a rate of change, so divide by $h$ and see which of the three pieces survive the division.

The first two terms give $u\,\Delta v/h + v\,\Delta u/h \to uv' + vu'$. The corner term is $\Delta u \cdot \Delta v / h$, and since $\Delta u \approx u'h$ and $\Delta v\approx v'h$, that is roughly $u'v'h$, which goes to zero. The corner is second order and it dies. What is left is

The corner square is the entire reason the product rule looks the way it does.

It is also your first meeting with a move you will make hundreds of times before this book is over: second-order terms die. It is worth noticing that this is a move rather than a law of nature. In Chapter 3.4 we transport a vector around a closed loop, and the second-order terms that refuse to die there are precisely the curvature of spacetime.

4.3 · Chain rule — where the reframe pays

This is the derivation that justifies everything said in §3, so let's go slowly.

We want $\frac{\dd}{\dd x}f(g(x))$ at $x=a$. Write $b=g(a)$ for the point at which the outer function gets evaluated.

Start by nudging $x$ by $h$ and asking how far $g$ moves. The linear approximation of $g$ at $a$ answers that directly:

So the inner function hands the outer one a displacement of size $k$. Our next step is to feed that displacement into $f$, using the linear approximation of $f$ at $b$:

The second line put in $k = g'(a)h$ plus something smaller than $h$, which sweeps every error into a single bucket measured against $h$. That is now exactly the shape of (0.1.4), so we can read off whatever multiplies $h$:

Notice what actually happened. We composed two linear approximations, and linear maps compose by multiplication. That is the whole content of the chain rule.

The same sentence keeps working as the setting gets harder. In several variables the linear maps are matrices, and matrices compose by matrix multiplication, so the chain rule becomes $\mathrm{D}(f\circ g) = \mathrm{D}f\,\mathrm{D}g$, a product of Jacobians (Chapter 0.6). In general relativity it becomes the transformation law that defines a tensor (Chapter 2.4). It is the same statement each time. Try getting there from "slope times slope".

Grind box — quotient rule, and the fine print on the chain rule

Quotient. Write $u/v = u\cdot v^{-1}$ and combine the product rule with the chain rule applied to $w\mapsto w^{-1}$, whose derivative is $-w^{-2}$:

$$\left(\frac{u}{v}\right)' = u'v^{-1} + u\cdot(-v^{-2})v' = \frac{u'v - uv'}{v^{2}}.$$

There is no reason to memorise this separately.

Fine print. The chain-rule argument above has a gap. Suppose $k=0$ for some sequence of $h\to0$. That is a perfectly legal thing for $g$ to do, and a constant $g$ does it at every step. The step "$f(b+k)=f(b)+f'(b)k+o(k)$" then divides by something that may vanish.

The clean repair is to define

$$\Phi(k) = \begin{cases} \dfrac{f(b+k)-f(b)}{k}, & k \neq 0\\[4pt] f'(b), & k=0\end{cases}$$

which is continuous at $k=0$ precisely because $f$ is differentiable there. Then $f(b+k)-f(b) = \Phi(k)\,k$ holds for all $k$, including $k=0$, and

$$\frac{f(g(a+h))-f(g(a))}{h} = \Phi(k)\cdot\frac{k}{h} \;\longrightarrow\; f'(b)\,g'(a).$$

No division by zero anywhere. This is the kind of hole that does not matter for any function you will meet in physics. It is still worth seeing once that somebody noticed it and plugged it.

5 · Where $\ee$ comes from

The exponential is not a function anyone chose out of fondness. It is forced, and this section shows what forces it.

Begin by differentiating $b^{x}$ straight from the definition, leaving the base $b$ unspecified:

Something remarkable is visible already, before any base has been picked. Whatever $b$ is, the derivative of $b^x$ comes out proportional to $b^x$ itself. Exponentials are the functions whose rate of growth is set by their current size.

All that distinguishes one base from another is the constant of proportionality $L(b)$, an awkward leftover sitting out in front. It depends on the base, with $L(2)\approx0.693$ and $L(10)\approx2.303$. The natural move is to ask for the base that makes the leftover disappear.

Before taking that base for granted, let's check that it exists and that only one does. For rational $r$, $L(b^{r}) = \lim_{h\to0}(b^{rh}-1)/h = r\,L(b)$, and continuity fills in the irrationals. So $L$ is a continuous, strictly increasing function of the base on $(0,\infty)$, with $L(1)=0$ and $L(b)\to\infty$ as $b\to\infty$. A continuous increasing function that starts at $0$ and runs to infinity takes the value $1$ exactly once. That base is what we call $\ee$.

So this is the definition of $\ee$: the base for which the function is its own derivative. The numerical value $2.71828\ldots$ is a consequence rather than a starting point.

The formula you already know falls straight out of it. Setting $L(\ee)=1$ means $(\ee^{h}-1)/h\to1$, which says $\ee^{h}\approx 1+h$ for small $h$, so $\ee \approx (1+h)^{1/h}$. Put $h=1/n$ and you have the familiar $\ee=\lim_{n\to\infty}(1+1/n)^{n}$.

The logarithm follows in one line, by the chain rule. Start from $\ee^{\ln x}=x$ and differentiate both sides. That gives $\ee^{\ln x}\cdot(\ln x)' = 1$, and therefore $(\ln x)' = 1/x$.

6 · Trigonometric derivatives

Everything in this section rests on one genuinely geometric input, so let's isolate that input before spending it.

In the unit circle, compare three areas that sit one inside the next: the triangle with vertices at the centre, at $(1,0)$ and at the point on the circle at angle $h$; the circular sector spanning the same angle; and the triangle cut off by the tangent line at $(1,0)$, which pokes out beyond the circle. For $0 \lt h \lt \pi/2$ their areas are $\tfrac12\sin h$, $\tfrac12 h$ and $\tfrac12\tan h$, so the comparison gives $\sin h \lt h \lt \tan h$.

What the derivative is going to need is a statement about $h/\sin h$, so divide the whole chain through by $\sin h$:

and that limit follows because $\cos h\to1$, which leaves the middle quantity trapped between two things both heading for $1$.

This is the point at which radians enter physics permanently. Measure angles in degrees instead and the limit comes out as $\pi/180$, so every formula downstream inherits the litter.

Grind box — from the squeeze to $\sin' = \cos$

We need a companion limit first, for the cosine terms that are about to appear. Using $1-\cos h = \dfrac{\sin^{2}h}{1+\cos h}$,

$$\frac{1-\cos h}{h} = \frac{\sin h}{h}\cdot\frac{\sin h}{1+\cos h} \;\longrightarrow\; 1\cdot\frac{0}{2} = 0.$$

With both limits in hand, expand the difference quotient for $\sin$ using the angle-addition identity:

$$\begin{aligned} \frac{\sin(x+h)-\sin x}{h} &= \frac{\sin x\cos h + \cos x \sin h - \sin x}{h}\\[4pt] &= \sin x\,\frac{\cos h - 1}{h} \;+\; \cos x\,\frac{\sin h}{h}\\[4pt] &\longrightarrow\; \sin x\cdot 0 + \cos x\cdot 1 \;=\; \cos x. \end{aligned}$$

The same manipulation on $\cos$ gives $\cos' = -\sin$. Now note what follows from doing it twice: $\dv{^{2}}{x^{2}}\sin x = -\sin x$. That is a function proportional to minus its own second derivative, which is the harmonic oscillator equation. It is about to become the most reused equation in this book (Chapters 0.8, 4.8, 5.3, 7.4).

7 · Higher derivatives, and what they are for

Differentiate again and you get $f''$, the rate at which the rate is changing. The first three of these are so common in physics that they have names: position, velocity $\dot x$, and acceleration $\ddot x$.

Newton's second law is a statement about the second derivative, and that is exactly why mechanics is hard. You are told the acceleration and asked for the position. Getting there means integrating twice, and each integration picks up a constant. The two constants encode where the thing started and how fast it was going.

Geometrically, $f''$ is concavity. The sharper statement is the one the zoom figure already showed you numerically, which is that $f''$ is the leading error in linearisation,

Keep going in that direction and you get Taylor's theorem. That is Chapter 0.3, and it more or less runs the rest of physics.

7.1 · Notation, and one deliberate abuse

Four notations are in common use and you will need to read all four. They are not rivals. Each one is convenient for a different job.

Form	Written	Best for
Lagrange	$f'(x)$, $f''(x)$	Pure functions of one variable
Leibniz	$\dv{f}{x}$, $\dfrac{\dd^{2}f}{\dd x^{2}}$	Anything where you must track with respect to what
Newton	$\dot x$, $\ddot x$	Derivatives with respect to time, universally, in mechanics
Operator	$\mathrm{D}f$, $\partial_{x} f$	When the derivative itself is the object of study, as in QM and QFT

8 · Worked examples

9 · Your turn