Part 0 · The Toolkit — Chapter 0.4

Vector Spaces and Linear Maps

The matrix is not the point. The matrix is a shadow the point casts once you pick coordinates.

Tools you'll need  — Chapter 0.1: linearisation, and the characterisation of the exponential as the function equal to its own derivative (we use it once, in §6). Nothing else. Everything about vectors is built here from scratch.

1 · What a vector actually is

Let's start by asking what the word "vector" is actually doing, because the answer decides how far everything later will reach.

You were probably told that a vector is an arrow, a thing with magnitude and direction. That description is a special case of a special case. It will strand you in Chapter 4.2, where the vectors are functions and there is not an arrow in sight.

Here is the actual definition. A vector space over a field of scalars $\mathbb{F}$ (for us always $\R$ or $\C$) is a set $V$ carrying two operations. The first is addition of two elements of $V$. The second is multiplication of an element of $V$ by a scalar. Both operations must land back inside $V$, and both must satisfy eight rules:

That is the whole definition. Notice what is not in it. There is no length, no angle, no coordinates and no arrows. Those all come later and they cost extra. Length and angle are Chapter 0.5. Coordinates are §2 of this chapter, and they are always a choice.

The axioms are not decoration. They do work, and here are two examples of that work, both of which you have used a thousand times without noticing that they need proving.

The scalar $0$ annihilates. Nothing in (0.4.1) says that $0\,v$ is the zero vector. The symbol $0$ on the left is a number and the one on the right is a vector, and those are different objects, so the statement has to be earned.

By (S4), $0v = (0+0)v = 0v+0v$. Add $-(0v)$ to both sides, use (A4) and (A3), and what is left is $0 = 0v$.

The same style of argument gives $(-1)v = -v$. Compute $v + (-1)v = 1v + (-1)v = \big(1+(-1)\big)v = 0v = 0$. By (A4), the thing you add to $v$ to get $0$ is exactly what the symbol "$-v$" means.

1.1 · The examples that are not arrows

The six examples below have almost nothing in common as objects. They have everything in common as vector spaces, and that is the whole point of the definition.

1. Tuples, $\R^{n}$. Add componentwise, scale componentwise. This is the familiar one, and it is the only one that looks like arrows.

2. Polynomials of degree at most $n$. Call this space $P_{n}$. The sum of two polynomials of degree $\le n$ again has degree $\le n$, and so does a scalar multiple of one. The zero polynomial is the zero vector. All eight axioms hold because they hold coefficient by coefficient.

There is a trap here worth stopping on. Polynomials of degree exactly $n$ do not form a vector space. Take $x^{2}$ and $-x^{2}+x$, both of degree exactly $2$. Their sum is not of degree $2$, and the zero polynomial is missing from the set entirely. Closure is a real condition, not a formality.

3. The solutions of a linear differential equation. This one matters more than the other five, so let's do it properly. Take the general second-order linear equation:

We want to know whether the solution set is closed under addition and scaling, so take two solutions $y_{1}$ and $y_{2}$ and feed a combination of them into $L$. The only facts we need are that differentiation is itself additive and that it commutes with constants:

Both terms vanished, so the combination solves the equation as well. The solution set is closed under addition and scaling, which makes it a vector space.

That statement has a name in physics. It is called superposition, and it does a great deal of work. It is why you may add two solutions of the wave equation and get a solution. It is why normal modes work (Chapter 0.8). It is why a plucked string can be decomposed into harmonics (Chapter 0.9).

Now notice what it is not. It is not a physical principle. It is the observation that $L$ is a linear map, together with the fact that the set of things a linear map sends to zero is a vector space.

The matching trap is worth seeing now, because it explains a rule you were once made to memorise. Suppose the equation is inhomogeneous, meaning $L[y]=f$ with $f\neq0$. Then the solution set is not a vector space at all, since the sum of two solutions gives $2f$ rather than $f$.

Something almost as good survives. Let $y_{p}$ be any one solution and let $y$ be any other. Then $L[y-y_{p}] = f-f = 0$, so the difference solves the homogeneous equation. That gives general solution = particular solution + (vector space of homogeneous solutions).

Picture it as a plane sitting inside a larger space, shifted off the origin. The "arbitrary constants" you were told to add are coordinates on that shifted plane.

4. Functions on an interval. Take all real-valued functions on $[a,b]$ and add them pointwise, so that $(f+g)(x)\equiv f(x)+g(x)$ and $(cf)(x)\equiv c\,f(x)$. Every axiom holds pointwise, because it holds for real numbers. This space is infinite-dimensional in the sense §2 is about to make precise, and it is the arena of Chapters 0.9 and 4.3.

5. Matrices. The $m\times n$ real matrices form a vector space under entrywise addition. Read that twice, because matrices are themselves vectors.

If you are carrying around the idea that a vector is a column of numbers, you cannot even parse that sentence. You will need to parse it in Chapter 6.1, where the vector space in question is a space of matrices and its elements are the generators of a symmetry group.

6. Quantum states. The states of a quantum system form a vector space over $\C$, and not in any metaphorical sense. The superposition principle is precisely the assertion that the axioms (0.4.1) hold for physical states. This is why Chapter 4.2 is a linear algebra chapter with physics vocabulary sprinkled on top.

2 · Span, independence, basis, dimension

We now want to describe an abstract vector by a list of numbers, because a list of numbers is something we can compute with. The machinery of this section does that. Just as importantly, it tells us that the list is not the vector.

Three definitions, in the order they are needed. A linear combination of $v_{1},\dots,v_{k}$ is any vector $c_{1}v_{1}+\cdots+c_{k}v_{k}$. The span of a set is the collection of all its linear combinations, and the span is itself a vector space, since a sum of two linear combinations is a linear combination and so is a multiple of one.

The third definition rules out redundancy. The set $\{v_{1},\dots,v_{k}\}$ is linearly independent if

In words, the only way to build the zero vector out of them is to use none of them. Put the two ideas together and you have the object we are after. A basis is a set that is linearly independent and that also spans the whole space.

2.1 · Coordinates exist, and they are unique

Let $\{e_{1},\dots,e_{n}\}$ be a basis of $V$. Because it spans, every $v\in V$ can be written as $v=\sum_{j}x_{j}e_{j}$ for some scalars $x_{j}$. What we need to show is that those scalars are uniquely determined by $v$, since otherwise the phrase "the coordinates of $v$" would not mean anything. So suppose $v$ had two expansions, and watch what independence forces:

The last step is exactly the definition of independence (0.4.4), applied to the coefficients $c_{j}=x_{j}-y_{j}$.

The converse is just as quick. Suppose every vector has a unique expansion, and apply that to $v=0$. The zero vector certainly has the expansion with all coefficients zero, and uniqueness says there is no other one. That is independence.

So the two conditions are one condition, which is worth stating on its own:

The scalars $x_{j}$ are the coordinates of $v$ in the basis $\{e_{j}\}$. We will write them as a column and give that column its own symbol, deliberately different from the symbol for the vector itself:

Throughout this chapter, $v$ is the vector and $\vv{x}$ is its coordinate list. They are not the same kind of thing. Keeping them typographically apart is not pedantry. It is the entire subject.

2.2 · Dimension

The dimension of $V$ is the number of elements in a basis. Before that can count as a definition, we need to know that two different bases of the same space cannot have different sizes. They cannot. The proof is a swapping argument worth seeing once, and it is in the grind box below. Granting it, the examples fall out:

Space	A basis	Dimension
$\R^{n}$	$e_{1},\dots,e_{n}$ (the standard columns)	$n$
$P_{3}$, polynomials of degree $\le3$	$1,\ x,\ x^{2},\ x^{3}$	$4$
$2\times2$ real matrices	the four matrices with a single $1$	$4$
Solutions of $y''+\omega^{2}y=0$	$\cos\omega t,\ \sin\omega t$	$2$
Functions on $[a,b]$	no finite basis exists	$\infty$

The polynomial entry deserves a word, since it is the one people wave through. Why are $1,x,x^{2},x^{3}$ independent? Suppose $c_{0}+c_{1}x+c_{2}x^{2}+c_{3}x^{3}=0$ as a function, meaning true for every $x$. Then that polynomial has infinitely many roots. But a nonzero polynomial of degree $\le3$ has at most three roots. So it must be the zero polynomial, and all $c_{k}=0$.

The independence of $\cos$ and $\sin$ is Problem 1, and it is proved the same way, by evaluating somewhere convenient.

Grind box — why every basis has the same size (the exchange lemma)

Lemma. If $\{w_{1},\dots,w_{m}\}$ spans $V$ and $\{v_{1},\dots,v_{k}\}$ is independent, then $k\le m$. An independent set can never be larger than a spanning set.

Proof. We swap the $v$'s in one at a time, keeping a spanning set at every step.

Since the $w$'s span, we may write $v_{1}=\sum_{j}c_{j}w_{j}$. Not every $c_{j}$ is zero, because that would make $v_{1}=0$, and a set containing $0$ is never independent. (Take $c_{1}=1$ and the rest zero in (0.4.4).) Relabel so that $c_{1}\neq0$, and then solve for $w_{1}$:

$$w_{1} = \frac{1}{c_{1}}\Big(v_{1} - \sum_{j\ge2}c_{j}w_{j}\Big),$$

so anything expressible with $w_{1}$ is expressible with $v_{1}$ instead, and $\{v_{1},w_{2},\dots,w_{m}\}$ still spans $V$.

Now suppose we have got as far as $\{v_{1},\dots,v_{r},w_{r+1},\dots,w_{m}\}$ spanning, with $r\lt k$. Expand the next one:

$$v_{r+1} = \sum_{i\le r} a_{i}v_{i} + \sum_{j\gt r} b_{j}w_{j}.$$

Ask what would happen if every $b_{j}$ vanished. Then $v_{r+1}$ would be a linear combination of $v_{1},\dots,v_{r}$, and $a_{1}v_{1}+\cdots+a_{r}v_{r}-v_{r+1}=0$ would be a nontrivial vanishing combination, since the coefficient of $v_{r+1}$ is $-1\neq0$. That contradicts independence. So some $b_{j}\neq0$. Relabel it as $b_{r+1}$, solve for $w_{r+1}$ as above, and swap it out. The new set still spans.

Each step consumes one $w$. If $k\gt m$ we would run out of $w$'s while $v$'s remained. But the step above proved that some $b_{j}$ with $j\gt r$ is nonzero, and that requires at least one $w$ to still be present. Contradiction. Hence $k\le m$. $\blacksquare$

Consequence. Let $B_{1}$ and $B_{2}$ be bases with $n_{1}$ and $n_{2}$ elements. $B_{1}$ is independent and $B_{2}$ spans, so $n_{1}\le n_{2}$. $B_{2}$ is independent and $B_{1}$ spans, so $n_{2}\le n_{1}$. Therefore $n_{1}=n_{2}$, and dimension is a property of the space, not of the description. $\blacksquare$

A corollary we need in §5. In an $n$-dimensional space, any $n$ independent vectors already form a basis. Suppose they failed to span. Then there would be some $u$ outside their span, and $\{v_{1},\dots,v_{n},u\}$ would be independent. To see that, take a vanishing combination $\sum c_{i}v_{i}+cu=0$. It must have $c=0$, since otherwise $u=-\frac1c\sum c_iv_i$ would lie in the span, and then all $c_{i}=0$ by independence of the $v$'s. That gives $n+1$ independent vectors in a space spanned by $n$, contradicting the lemma.

3 · Linear maps, and why matrix multiplication looks the way it does

We have the objects. Now we need the maps between them, and the only maps worth having are the ones that respect the two operations the objects were defined by. A map $T:V\to W$ between vector spaces is linear if it does exactly that:

Here are some examples, deliberately not all of them matrices. Rotation of the plane is linear. So is differentiation $y\mapsto y'$, acting on polynomials or on smooth functions. So is the operator $L$ of (0.4.2). So is definite integration $f\mapsto\int_{a}^{b}f$, which is a linear map from a function space to $\R$. And in Chapter 4.2, so is every physical observable.

3.1 · A linear map is determined by what it does to a basis

A linear map looks at first like an enormous amount of data, since it has to say what happens to every one of infinitely many vectors. We want to show that it is far less than that.

Let $\{e_{1},\dots,e_{n}\}$ be a basis of $V$ and let $T:V\to W$ be linear. Take any $v\in V$, expand it in coordinates as in (0.4.6), then apply additivity $n-1$ times and homogeneity to each term:

Let's read what that says. If you know the $n$ vectors $T(e_{j})$, you know $T$ completely. Its value on every one of the infinitely many vectors in $V$ is then forced.

The converse holds too, and it is worth spelling out. Choose any $n$ vectors $w_{1},\dots,w_{n}$ in $W$, whatever you like, and define $T(v)\equiv\sum_{j}x_{j}w_{j}$. That is a legitimate definition precisely because the coordinates $x_{j}$ are unique, which is what (0.4.5) was for. And it is linear because coordinates add and scale.

So linear maps $V\to W$ correspond exactly to arbitrary lists of $n$ vectors in $W$. Nothing is left over on either side.

3.2 · The matrix, defined

Those $n$ vectors $T(e_{j})$ are still abstract objects, and what we want is numbers. So pick a basis $\{f_{1},\dots,f_{m}\}$ of the target space $W$ as well. Each $T(e_{j})$ lives in $W$, so it has coordinates there, and we give those coordinates a name:

The $m\times n$ array of numbers $T_{ij}$ is the matrix of $T$ in the bases $\{e_{j}\},\{f_{i}\}$. Here is the sentence to remember, in words: the $j$-th column of the matrix is the coordinate list of the image of the $j$-th basis vector.

The two indices are doing different jobs. The first labels which component in the target space you are looking at. The second labels which basis vector went in.

Now let's find out what the array does to coordinates, since that is what we will spend it on. Combine (0.4.8) with (0.4.9) and swap the order of two finite sums:

That is the row-times-column rule for a matrix acting on a column vector. We did not postulate it. It is what (0.4.9) forces.

3.3 · Composition, and the punchline

Here is the moment this section exists for. We want to know which array describes doing one map and then another, so set two maps up end to end. Let $T:U\to V$ and $S:V\to W$, with bases $\{u_{k}\}$ of $U$, $\{v_{j}\}$ of $V$ and $\{w_{i}\}$ of $W$. Each map gets its matrix from (0.4.9):

The composite map $S\circ T:U\to W$ means "do $T$, then $S$". It is linear, which you can check by composing the definitions, so it has a matrix of its own. By (0.4.9) that matrix is read off from what $S\circ T$ does to the basis $\{u_{k}\}$. So let's compute exactly that:

Now compare that last line with the definition (0.4.9), and read off the matrix of the composite:

There it is. Matrix multiplication is not a convention someone chose. It is what composition of linear maps does to the arrays. Every strange feature of the rule is now explained rather than memorised:

• Why rows times columns. The summed index $j$ labels a basis of the middle space $V$, which is where the output of $T$ lands and where the input of $S$ comes from. It has to be summed because (0.4.12) feeds every component of $T(u_{k})$ through $S$. The free indices $i$ and $k$ label the two ends.
• Why the shapes must match. An $m\times n$ times an $n\times p$ is legal and nothing else is, because the middle dimension is the dimension of the middle space. Mismatched shapes are not a typing error. They are an attempt to compose maps that do not connect.
• Why $ST$ means "first $T$". Because $(S\circ T)(u)=S(T(u))$ and the function nearest the argument acts first. The reversed-looking order is inherited from function notation, which is in turn inherited from writing $f(x)$ rather than $(x)f$.
• Why it is associative for free. $(RS)T = R(ST)$ requires no computation whatsoever. Composition of functions is associative, since both sides mean "do $T$, then $S$, then $R$". People verify this with triple sums. There is no need.
• Why it need not commute. Doing $T$ then $S$ is a different operation from doing $S$ then $T$. See the warning box below, and then most of twentieth-century physics.

The repeated index $j$ in (0.4.13), summed over while the free indices survive, is your first contraction. In Chapter 2.4 it gets a notation of its own, Einstein's convention, in which a repeated index is silently summed. From there it never stops.

4 · Change of basis

Two people describe the same space with different bases. What is the relationship between their arrays? This is the most mechanical section of the chapter, and it is also the one with the longest reach. It is the prototype of every transformation law in relativity.

Let $\{e_{j}\}$ and $\{e'_{j}\}$ both be bases of $V$. Each new basis vector is itself a vector, so it has coordinates in the old basis. Call those coordinates $P_{ij}$:

In words, the columns of $P$ are the new basis vectors, written in old coordinates.

We are going to need $P$ to be invertible, and we can see that it is without appealing to anything outside what we already have. Since $\{e'_{k}\}$ is also a basis, we may equally expand the old vectors in the new ones, $e_{i}=\sum_{k}Q_{ki}e'_{k}$. Now substitute each expansion into the other:

The implication is uniqueness of coordinates (0.4.5) again. The left side has to be the expansion of $e'_{j}$ in the basis $\{e'_{k}\}$, which is $1$ in slot $j$ and $0$ elsewhere, and that is what the Kronecker delta $\delta_{kj}$ means.

So $QP=I$. Running the same argument with the roles exchanged gives $PQ=I$, and hence $Q=P^{-1}$.

4.1 · Components go the other way

The vector itself is not changing, so the two coordinate lists must be related somehow, and we want that relation. Take a vector $v$ and expand it both ways, $v=\sum_{j}x_{j}e_{j}=\sum_{j}x'_{j}e'_{j}$, then substitute (0.4.14) into the second expansion:

Let's stop on that, because it is the source of a lifetime of index confusion. The basis vectors transform with $P$. The components transform with $P^{-1}$.

They have to go in opposite directions, because their product is the actual vector, and the vector does not change at all. Make your ruler twice as long and every measurement expressed in rulers is halved.

In Chapter 2.4 this opposition is given names, covariant and contravariant, and it is why some indices are written upstairs and some downstairs.

4.2 · The similarity transformation

Finally we come to the case that matters most, which is a linear map from a space to itself. Such a map is called an operator. Let $T:V\to V$ be one, with matrix $A$ in the old basis and $A'$ in the new one:

The trick is to compute $T(e'_{j})$ in two different ways and then set the answers side by side. First, push $T$ through the definition of the new basis (0.4.14):

Second, use the new-basis matrix instead, and then express the new basis vectors back in the old basis:

Both lines are expansions of the same vector in the same basis $\{e_{i}\}$. So by uniqueness of coordinates the coefficients have to agree, which gives $AP=PA'$, and therefore

Two matrices related this way are called similar, and the content of (0.4.20) is short. Similar matrices are the same map, seen twice.

Here is a sanity check in coordinates. If $\vv{y}=A\vv{x}$ then $\vv{y}'=P^{-1}\vv{y}=P^{-1}AP\vv{x}'$. The sandwich reads right to left as "translate the new coordinates into old, apply the map, translate back".

Now let's look at what that line is worth, because it is more than it appears. Everything that is genuinely about the map must be left unchanged by (0.4.20), since $P$ encodes only our change of mind. Sections 5 and 6 will prove that the determinant and the trace are two such invariants, and Chapter 0.5 adds the eigenvalues and the characteristic polynomial.

The reverse reading is just as useful. Anything that changes under (0.4.20), such as any individual entry, is a fact about your description rather than about the physics.

That box tells you the failure exists and shows it at full size. It does not tell you how large the failure is when both operations are made small. That smaller version is the form every one of the three cash-outs takes, and the figure below measures it.

pair rotate + squeeze two shears rotate + dilate rotate + reflect (full size)

step ε

ε = 0.1000

A = rotation generator, B = squeeze generator · [A,B] = [ 0.000 2.000 ; 2.000 0.000 ]

x = (1.000, 0.300) [A,B]x = (0.600, 2.000) |[A,B]x| = 2.088061

|r| = 1.9269e-2 |r|/ε = 0.192694 |r|/ε² = 1.926940

∠(r, [A,B]x) = 2.274° · loop drawn ×12.9 about x (its true size is 9.68e-2)

5 · The determinant, as signed volume

The determinant is usually introduced as a formula involving alternating signs and minors, and from that formula nothing at all is visible. We will do it the other way round. First state the three properties that pin the determinant down, then notice that those are the properties of volume, and get the formulas afterwards as a consequence.

Think of an $n\times n$ matrix $A$ as an ordered list of its columns $a_{1},\dots,a_{n}$, each of them a vector in $\R^{n}$. The three properties are:

(D1) says linear in each column separately, with the others held fixed.

One consequence of (D2) is worth extracting straight away, because we will lean on it repeatedly. Suppose two columns are equal. Swapping them changes nothing, yet by (D2) it must flip the sign. So $D=-D$, and therefore $D=0$. Repeated column $\Rightarrow$ zero.

5.1 · These are the properties of volume

Let $\mathrm{Vol}(a_{1},\dots,a_{n})$ be the signed volume of the parallelepiped spanned by the columns from a common corner. In two dimensions that is the area of the parallelogram with edges $a_{1},a_{2}$. In three it is the volume of the skewed box.

We take exactly one fact from geometry, which is that volume equals base times perpendicular height. That is the fact which makes a leaning stack of paper occupy the same volume as a straight one. With it, all three properties follow.

(D3) is the choice of unit. The unit cube, spanned by $e_{1},\dots,e_{n}$, has volume $1$. That is what "unit" means.

(D2) is orientation. The parallelepiped spanned by $(a_{1},a_{2})$ and the one spanned by $(a_{2},a_{1})$ are the same set of points. But the two ordered lists have opposite handedness, since going from the first edge to the second is counterclockwise in one case and clockwise in the other. Signed volume records that with a sign. And if two edges coincide the solid is flat and its volume is genuinely zero, which matches the consequence we just derived from (D2).

(D1) is base times height. Hold all columns but $a_{j}$ fixed. They span a "base", meaning an $(n-1)$-dimensional parallelepiped of some fixed content $B$ lying in a hyperplane through the origin. The volume of the whole solid is then $B\times h$, where $h$ is the signed perpendicular height of $a_{j}$ above that hyperplane.

Now look at what $h$ is. It is the component of $a_{j}$ along the fixed unit normal $\hat n$ of the base, and taking a component is a linear operation, so $h(\alpha a_{j} + \beta a'_{j}) = \alpha h(a_{j}) + \beta h(a'_{j})$. Multiply through by the constant $B$ and you have exactly (D1). (If the base is itself degenerate then $B=0$ and both sides vanish.)

So signed volume satisfies (D1)–(D3). Now for the structural fact that makes this worth anything, proved in the grind box below. Those three properties determine the function completely, in the sense that there is exactly one $D$ satisfying them. Since $\det$ and signed volume both satisfy them, they have to be the same thing:

One more step converts that from a statement about an array into a statement about a map. (0.4.9) makes the columns of $A$ the images $A e_{j}$ of the standard basis vectors, so the parallelepiped in question is the image of the unit cube.

So here is the sentence to carry away. $\det A$ is the factor by which the map $A$ multiplies volumes, with a sign recording whether it preserves or reverses orientation. The interactive below lets you drive that with your hands.

Grind box — uniqueness, the explicit formula, and the $2\times2$ and $3\times3$ cases

Uniqueness. Suppose $D$ satisfies (D1)–(D3). Expand every column in the standard basis, $a_{j}=\sum_{i}A_{ij}e_{i}$, and use multilinearity in each of the $n$ slots in turn. Every slot contributes a sum over which basis vector was chosen there, so

$$D(a_{1},\dots,a_{n}) = \sum_{i_{1}=1}^{n}\cdots\sum_{i_{n}=1}^{n} A_{i_{1}1}A_{i_{2}2}\cdots A_{i_{n}n}\; D(e_{i_{1}},\dots,e_{i_{n}}).$$

Any term in which two of the $i$'s coincide has a repeated column and dies. So only the terms where $(i_{1},\dots,i_{n})$ is a permutation $\sigma$ of $(1,\dots,n)$ survive. Each such $D(e_{\sigma(1)},\dots,e_{\sigma(n)})$ can be sorted back into $D(e_{1},\dots,e_{n})=1$ by a sequence of swaps, each costing a factor $-1$ by (D2). Writing $\mathrm{sgn}(\sigma)=(-1)^{\#\text{swaps}}$,

$$\det A \;=\; \sum_{\sigma} \mathrm{sgn}(\sigma)\, A_{\sigma(1)1}A_{\sigma(2)2}\cdots A_{\sigma(n)n},$$

which is determined, with no freedom left anywhere. This is the Leibniz formula, and the derivation is simultaneously the proof that at most one such $D$ exists.

Is $\mathrm{sgn}$ well defined? It had better not depend on which sequence of swaps you used, so here is the check. Count the inversions of $\sigma$, meaning the pairs $i\lt j$ with $\sigma(i)\gt\sigma(j)$. Swapping two adjacent entries changes that count by exactly $\pm1$. A swap of two entries $d$ apart can be built from $2d-1$ adjacent swaps, so any single swap changes the inversion count by an odd number. Hence the parity of the number of swaps needed is fixed by $\sigma$ alone, and $\mathrm{sgn}(\sigma)=(-1)^{\#\text{inversions}}$.

Existence. Uniqueness is worthless if nothing satisfies the axioms, so check that the Leibniz formula does. Each term contains exactly one factor from each column, so it is linear in each column: (D1). ✓ Swapping columns $p,q$ replaces each $\sigma$ by $\sigma$ composed with a transposition, which is a relabelling of the sum that flips every sign: (D2). ✓ For $A=I$ every factor $\delta_{\sigma(j)j}$ forces $\sigma=\mathrm{id}$, leaving the single term $1$: (D3). ✓

Rows or columns? Substituting $i=\sigma(j)$ in the product shows that $\det A^{\top} = \sum_{\sigma}\mathrm{sgn}(\sigma)\prod_{i}A_{\sigma^{-1}(i)\,i}$. Now $\sigma^{-1}$ has the same parity as $\sigma$, since you reverse the order of the same swaps, and $\sigma\mapsto\sigma^{-1}$ merely reshuffles which terms are being summed. So $\det A^{\top}=\det A$, and every statement about columns is equally a statement about rows.

The $2\times2$ case, straight from the axioms. No formula is needed here. Expand bilinearly and kill the repeated columns:

$$\begin{aligned} D\big(ae_{1}+ce_{2},\; be_{1}+de_{2}\big) &= ab\underbrace{D(e_{1},e_{1})}_{0} + ad\underbrace{D(e_{1},e_{2})}_{1} + cb\underbrace{D(e_{2},e_{1})}_{-1} + cd\underbrace{D(e_{2},e_{2})}_{0}\\[4pt] &= ad-bc. \end{aligned}$$

The $3\times3$ case. Six permutations of $(1,2,3)$: three even ($123,231,312$) and three odd ($213,132,321$). The Leibniz formula gives

$$\det\begin{pmatrix}a&b&c\\ d&e&f\\ g&h&i\end{pmatrix} = aei + bfg + cdh - ceg - bdi - afh,$$

which regroups into the cofactor expansion you may have memorised, $a(ei-fh)-b(di-fg)+c(dh-eg)$. Both are the same six products. The alternating signs of the cofactor rule are (D2) in disguise, and the reason the formula has $n!$ terms is that it is a sum over permutations. That is also why nobody computes a large determinant this way.

Triangular matrices. If $A_{ij}=0$ whenever $i\lt j$ (lower triangular), then any $\sigma\neq\mathrm{id}$ must somewhere send a column $j$ to a row $i\lt j$, contributing a zero factor. Only the identity permutation survives, leaving $\det A = A_{11}A_{22}\cdots A_{nn}$. That is the fact the Familiar Ground box used to read off the eigenvalues of the transition matrix.

5.2 · The product rule, in one line

Now the payoff for having done it geometrically. We want $\det(AB)$, so think about what the composite does to a volume. The map $B$ multiplies every volume by $\det B$. The map $A$ then multiplies whatever it is handed by $\det A$. Doing both multiplies volumes by the product of the two. Hence

That is the entire proof, and it is worth pausing on how much better it is than the algebraic version. Nothing is being manipulated at all. The identity says that scaling factors multiply, which is a fact about the word "factor". (If you want the geometric hand-waving cashed out into axioms, the airtight version is in the next grind box, and it runs to four lines.)

Two consequences follow immediately. First, take $B=A^{-1}$, which gives $\det(A)\det(A^{-1})=\det I = 1$ and therefore $\det(A^{-1}) = 1/\det A$. Undoing a map undoes its volume scaling, as it must.

Second, and this is the one we will lean on for the rest of the chapter, the determinant is basis independent:

So the determinant is not a property of the array. It is a property of the map, which is why it will still make sense in Chapter 3.3, where there is no preferred basis anywhere.

Grind box — the product rule without the hand-waving

Fix $A$ and define $D(B) \equiv \det(AB)$, regarded as a function of the columns $b_{1},\dots,b_{n}$ of $B$. Column $j$ of $AB$ is $A b_{j}$, by (0.4.10). Now check the three axioms:

(D1) $b_{j}\mapsto Ab_{j}$ is linear, and $\det$ is linear in each column, so the composite is linear in each $b_{j}$. ✓

(D2) Swapping $b_{p}$ and $b_{q}$ swaps columns $Ab_{p}$ and $Ab_{q}$ of $AB$, which flips the sign of $\det(AB)$. ✓

(D3) Not normalised: $D(I) = \det(A)$, not $1$. Fix that by dividing. If $\det A\neq0$, the function $\tilde D(B) = D(B)/\det A$ satisfies all three axioms, so by the uniqueness proved above $\tilde D = \det$, i.e. $\det(AB) = \det A\,\det B$. ✓

If $\det A = 0$ the division is illegal. But in that case the columns of $A$ are dependent, as proved just below, and the columns of $AB$ are combinations of the columns of $A$, so they are dependent too. Both sides are then $0$. The identity holds in every case. $\blacksquare$

5.3 · Zero determinant means information destroyed

Here is the single most-used fact about determinants, and both directions of it are now cheap:

($\Leftarrow$) If $A$ is invertible then $\det A\det(A^{-1})=1$, so $\det A\neq0$.

($\Rightarrow$) Take the contrapositive, and suppose $A$ is not invertible. Then its columns cannot be independent. To see why, suppose they were. The corollary in the §2 grind box says that $n$ independent vectors in $\R^{n}$ form a basis, so every vector in $\R^{n}$ is $A\vv{x}$ for exactly one $\vv{x}$, and the map $\vv{y}\mapsto\vv{x}$ that undoes it is linear and is an inverse. That contradicts what we assumed.

So some column is a combination of the others, say $a_{1}=\sum_{j\ge2}c_{j}a_{j}$. Expanding by multilinearity in the first slot gives a sum of determinants, each with a repeated column, and each therefore zero. Hence $\det A=0$. $\blacksquare$

The geometry says the same thing more vividly. $\det A = 0$ means the image of the unit cube has no volume. The map has flattened $n$-dimensional space onto something lower-dimensional, such as a plane, a line or a point.

A whole direction's worth of information has been thrown away, and no map can retrieve it, because many different inputs now share an output. Non-invertible is not a technicality about matrices. It is destruction of information, visible as the collapse of a volume. That is exactly what the "singular" preset in the figure below does, and you can watch it happen by hand.

a

1.20

b

0.60

c

-0.30

d

1.00

det = 1.380 (area = 1.380)

tr = 2.20

presets rotation shear reflection singular

6 · Trace, and the identity that matters

The trace of a square matrix is the sum of its diagonal entries:

This looks like the least interesting definition in the chapter. It is the most useful one in the book, and everything follows from a single identity.

6.1 · The cyclic property

The identity worth having is that $\mathrm{tr}(AB)$ and $\mathrm{tr}(BA)$ agree. To get it, write out both sides using (0.4.13) and then swap the order of two finite sums:

The only thing used was that scalars commute, meaning $A_{ij}B_{ji}=B_{ji}A_{ij}$, even though the matrices emphatically do not. That is why the identity is startling. In general $AB\neq BA$, and yet their traces always agree. (Worked example 1 shows a pair whose two products differ in three of their four entries and whose traces are equal anyway.)

Here is the immediate consequence, and it is the reason the trace is worth anything at all:

That is (0.4.27) with $X=P^{-1}$ and $Y=AP$. The trace is basis independent. Like the determinant, it is a property of the map rather than of the array. That deserves a second look, because the definition mentions only diagonal entries, and "diagonal" has no meaning until a basis has been chosen.

6.2 · Trace is the infinitesimal determinant

Now the identity this section exists for. Take any $A$ and any small number $\epsilon$, and ask for the determinant of the matrix that is almost the identity:

Here is the honest derivation for general $n$, which needs nothing but the Leibniz formula from the §5 grind box. Write $M = I+\epsilon A$, so $M_{ij} = \delta_{ij}+\epsilon A_{ij}$, and

The plan is to split that sum into two pieces, the identity permutation and everything else, and then to show that everything else is too small to matter.

Every non-identity permutation contributes only at order $\epsilon^{2}$. If $\sigma\neq\mathrm{id}$ then it moves at least one index, and therefore at least two of them. A permutation cannot displace exactly one thing and leave everything else fixed, because if $\sigma(i)\ne i$ then whatever was at $\sigma(i)$ has been displaced as well.

Now count powers of $\epsilon$. For each moved index $i$ the Kronecker delta $\delta_{\sigma(i)i}$ vanishes, so the factor is $\epsilon A_{\sigma(i)i}$ and carries one power of $\epsilon$. Two or more moved indices means two or more powers of $\epsilon$. So all of these terms are $O(\epsilon^{2})$, and none of them can affect the first-order answer.

The identity permutation gives the trace. With $\sigma=\mathrm{id}$ every delta is $1$ and $\mathrm{sgn}=+1$, leaving

Let's see where that came from. Expanding the product means choosing, from each of the $n$ factors, either the $1$ or the $\epsilon A_{ii}$. Taking all the $1$s gives the leading $1$. Taking $\epsilon A_{ii}$ from exactly one factor and $1$ from the rest gives $\epsilon A_{ii}$, and summing over which factor was chosen gives $\epsilon\,\mathrm{tr}A$. Any choice of two or more is $O(\epsilon^{2})$.

Adding the two contributions gives (0.4.29). $\blacksquare$

6.3 · Where this gets spent — four places, immediately

1. The divergence of a vector field is the trace of its Jacobian. Let a fluid have velocity field $v(x)$. In a short time $\epsilon$ each particle moves from $x$ to $x+\epsilon\,v(x)$. That map has derivative matrix $I + \epsilon J$, where $J_{ij}=\pdv{v_{i}}{x_{j}}$, and Chapter 0.6 will call that matrix the Jacobian.

Chapter 0.6 will also show that the determinant of a map's derivative matrix is precisely its local volume factor, which is (0.4.22) applied to the linear approximation. So the volume of a small blob is multiplied, in time $\epsilon$, by

The divergence arrives in Chapter 0.7 looking like an arbitrary sum of partial derivatives. What that line says is that it is the fractional rate of change of volume, and it is that because the trace is the infinitesimal determinant. "Incompressible" and "divergence-free" are the same sentence.

2. Liouville's theorem is this identity. In Chapter 1.3 the state of a mechanical system is a point in phase space $(q,p)$, moving with velocity $\dot q = \pdv{H}{p}$, $\dot p = -\pdv{H}{q}$. That is a vector field on phase space, and its divergence is

The two mixed partial derivatives cancel because they commute (Chapter 0.6). The trace of the Jacobian therefore vanishes identically, so by (0.4.32) phase-space volume is exactly conserved by Hamiltonian flow. That is Liouville's theorem, one of the deepest facts in classical mechanics, and it is three lines of §6.

3. $\det \ee^{A} = \ee^{\mathrm{tr}A}$. Define the matrix exponential by the series $\ee^{tA}=\sum_{k}t^{k}A^{k}/k!$. It converges for every $t$, because the terms are bounded by those of a convergent scalar exponential series, which is Chapter 0.3. Differentiating term by term is legitimate for a power series in $t$, and gives $\dv{}{t}\ee^{tA} = A\,\ee^{tA}$. So to first order in $\epsilon$:

What we want next is a differential equation for the determinant. So let $f(t)\equiv\det \ee^{tA}$, take determinants of (0.4.34), and use the product rule (0.4.23) together with (0.4.29). That last step is legitimate because $\det$ is a polynomial in the entries, so an $O(\epsilon^{2})$ change in the entries makes an $O(\epsilon^{2})$ change in the determinant:

By Chapter 0.1, the function equal to a constant times its own derivative, starting at $1$, is the exponential. Hence $f(t)=\ee^{t\,\mathrm{tr}A}$, and setting $t=1$ gives the identity we wanted:

4. Why the generators of SU($N$) are traceless. A continuous symmetry is built by exponentiating a generator, $U(\theta) = \ee^{\theta X}$. The "S" in SU($N$) stands for special, which means $\det U = 1$, so the transformations preserve volume. By (0.4.36) that requires $\ee^{\theta\,\mathrm{tr}X}=1$ for every $\theta$, which forces

This is not a bookkeeping convention. In Chapter 6.1 you will count the anti-Hermitian $3\times3$ matrices and find a $9$-dimensional space. Tracelessness is one real linear condition, which removes exactly one dimension, leaving $3^{2}-1=8$.

There are eight gluons because of (0.4.29). The most consequential appearance of a sum of diagonal entries in physics is a particle count.

7 · Why complex vector spaces

Everything so far worked equally over $\R$ and over $\C$. Here is where they part, and the difference is not aesthetic.

An eigenvector of an operator $T$ is a nonzero $v$ with $Tv=\lambda v$. It is a direction the map leaves alone, stretching it by the factor $\lambda$. These are the directions in which a linear map is simple, and Chapter 0.5 is devoted to finding them.

Over $\R$, they need not exist. Let $R$ be rotation of the plane by $90^{\circ}$. Suppose $Rv=\lambda v$ for real $\lambda$ and nonzero real $v$, and apply $R$ again to get $R^{2}v=\lambda^{2}v$. But rotating twice by $90^{\circ}$ is $R^{2}=-I$, so $\lambda^{2}v = -v$ and $\lambda^{2}=-1$. There is no real solution.

The geometric statement is plain once said. A rotation leaves no direction pointing along itself. That is what rotating means.

Over $\C$, they always exist. Let $A$ be any $n\times n$ complex matrix, $n\ge1$. Consider

By the Leibniz formula this is a polynomial in $\lambda$ of degree exactly $n$. The identity permutation contributes $\prod_{i}(A_{ii}-\lambda)$, whose leading term is $(-\lambda)^{n}$. Every other permutation misses at least two diagonal factors and so has degree at most $n-2$, which means nothing can cancel that leading term.

Now bring in the fundamental theorem of algebra: every non-constant polynomial with complex coefficients has a complex root. (⚑ Quoted, not proved: this is the one result in this chapter we import rather than prove; it is a theorem of complex analysis, and Chapter 0.3's discussion of the complex plane is the right place to have made peace with it.)

So there is a $\lambda_{0}\in\C$ with $\det(A-\lambda_{0}I)=0$. By (0.4.25) the matrix $A-\lambda_{0}I$ is then not invertible, so some nonzero $v$ has $(A-\lambda_{0}I)v=0$, which says

Every operator on a complex vector space has at least one eigenvalue. Over $\R$ that statement is flatly false, and the $90^{\circ}$ rotation is the counterexample.

Watch what the complex numbers do to that rotation. Over $\C$ the rotation by angle $\theta$, $R_{\theta}=\begin{pmatrix}\cos\theta&-\sin\theta\\ \sin\theta&\cos\theta\end{pmatrix}$, acting on the complex vector $(1,-\ii)$, gives

The eigenvalue of a rotation is $\ee^{\ii\theta}$. That is Euler's formula from Chapter 0.3, arriving here as the spectrum of a rotation matrix. The rotation that had no real eigenvector has two complex ones, with eigenvalues $\ee^{\pm\ii\theta}$ of unit modulus.

Rotation and complex phase are the same phenomenon written twice, which is why Chapter 0.3 could say that $\mathrm{U}(1)$ and $\mathrm{SO}(2)$ are the same group.

This is why quantum mechanics is built on complex vector spaces rather than real ones, and the argument comes in two halves.

Physical quantities in Chapter 4.2 are represented by operators, and their measurable values are eigenvalues. A theory in which operators can fail to have any is not a theory of measurement.

And time evolution is a rotation in state space, which was Chapter 0.1's closing observation and is now precise. The operator $\ee^{-\ii\hat Ht/\hbar}$ has unit-modulus eigenvalues, so it preserves lengths, so probability is conserved. Take away the complex numbers and you lose the eigenvalues, the phases and the conservation of probability together.

8 · Worked examples

9 · Your turn