Part 0 · The Toolkit — Chapter 0.5

Inner Products, Eigenvectors, and the Spectral Theorem

Quantum mechanics, four parts early, with the physics stripped off.

Tools you'll need  — Chapter 0.1: nothing beyond fluency. Chapter 0.3: Euler's formula $\ee^{\ii\theta}=\cos\theta+ \ii\sin\theta$, and power series. Chapter 0.4: vector spaces, basis and dimension, linear maps and their matrices, change of basis, determinant, trace, and the fact that over $\C$ every square matrix has at least one eigenvalue. Everything here is built on those.

1 · Inner products

A vector space as Chapter 0.4 left it is a bleak place. You can add vectors and scale them, and that is all. There is no length, no angle, no notion of two vectors pointing in different directions. Every geometric word you know is missing.

An inner product is the single extra structure that puts all of them back at once. Let's see what it has to look like, and why it has no freedom in the matter.

1.1 · Why the axioms look the way they do

You already have an inner product in $\R^{n}$, namely the dot product $\vv{u}\cdot\vv{v}=\sum_i u_i v_i$. Its essential property is that $\vv{v}\cdot\vv{v}=\sum_i v_i^{2}$ is the squared length: a non-negative number that vanishes only for the zero vector.

Now try to carry that same formula over to $\C^{n}$ unchanged. It breaks immediately. Take $v=(1,\ii)$ in $\C^{2}$ and add up the squares of the components:

and yet $v\neq0$. A non-zero vector with zero length is useless to us. We would lose the ability to say that two states are different, and telling states apart is the one job a length has to do.

So the formula needs repair, and there is exactly one repair available. Replace $v_i^{2}$ by $\abs{v_i}^{2}=\bar v_i v_i$, which is non-negative for every complex number and zero only when $v_i=0$. That forces the complex inner product to conjugate one of its two arguments. Everything below is bookkeeping around that single fact.

An inner product on a complex vector space $V$ is a map $V\times V\to\C$, written $\avg{u,v}$, satisfying three axioms.

Three remarks follow, in increasing order of importance.

Axiom (iii) is not quite well posed until (i) is used. Writing "$\avg{v,v}\gt0$" presumes that $\avg{v,v}$ is real, and that is a consequence rather than an assumption. Set $u=v$ in (i) and you get $\avg{v,v}=\overline{\avg{v,v}}$. A complex number equal to its own conjugate is real. So the axioms are consistent, and we derived the fact rather than assuming it.

Linearity in one slot forces conjugate-linearity in the other. We only demanded linearity in the second argument. The first is then determined, and conjugate symmetry shows it in one line:

Pull a scalar out of the second slot and it comes out unchanged. Pull it out of the first and it comes out conjugated. This asymmetry is real and you have to keep track of it. It is also the reason the notation of §2 exists.

The convention. Which slot is the linear one is pure convention, and the two communities chose differently. Mathematicians make the first slot linear. Physicists make the second. We have chosen the physics convention above and will keep it for the whole book. The reason is Chapter 4.2, where $\avg{\phi,\psi}$ becomes a probability amplitude, read right-to-left as "amplitude to find $\psi$ in state $\phi$". The linear slot had better be the one holding the state that is actually evolving. If you open a mathematics text and every formula appears to have its conjugates on the wrong side, this is why.

1.2 · The standard examples

Space	Inner product	Where it turns up
$\R^{n}$	$\sum_i u_i v_i$	Ordinary geometry; least squares (§3)
$\C^{n}$	$\sum_i \bar u_i v_i$	Spin, qubits, any finite quantum system (4.12)
$\R^{n}$, weighted	$\sum_i w_i u_i v_i$, $w_i\gt0$	Weighted regression; $\chi^{2}$ fitting
Functions on $[a,b]$	$\int_a^b \overline{f(x)}\,g(x)\,\dd x$	Fourier series (0.9); wavefunctions (4.3)
$n\times n$ matrices	$\operatorname{tr}(A^{\dagger}B)$	Density matrices (4.19)

The fourth row is the one to test, because it is the one that will matter most later. Check it against the axioms and you will find nothing new is needed. Conjugate symmetry holds because $\overline{\int \bar g f} = \int \bar f g$. Linearity in the second slot holds because integration is linear. Positive definiteness holds because $\int\abs{f}^{2}\ge0$, with equality only if $f\equiv0$.

That row is the entire reason Chapter 4.3 is possible. A space of functions is an inner-product space, and every theorem of this chapter will try to move there. Some of them survive the trip. Some need repair, and the warning at the end of §6 says which ones and why.

1.3 · Length, distance, and the angle we cannot yet define

An inner product is all we need in order to say how long a vector is and how far apart two vectors are. So define the norm and the distance:

Both are well defined. We showed above that $\avg{v,v}$ is real and non-negative, so the square root is a real number. Call $u$ and $v$ orthogonal when $\avg{u,v}=0$. That relation is symmetric, since $\avg{v,u}=\overline{\avg{u,v}}=0$ as well, so it does not matter which vector you name first.

Now let's see what the three axioms hand us for free. Expand the squared norm of a sum, using nothing but the axioms, and the generalised Pythagoras drops out:

The reason is that $\avg{u+v,u+v}=\avg{u,u}+\avg{u,v}+\avg{v,u}+\avg{v,v}$, and the middle two terms are conjugates of each other, so they add to twice the real part. When $u\perp v$ the cross term dies and $\left\lVert u+v\right\rVert^{2}=\left\lVert u\right\rVert^{2}+\left\lVert v\right\rVert^{2}$. That is Pythagoras, in any dimension, over $\C$, derived in one line from three axioms.

Now the angle. In $\R^{n}$ you were told that $\cos\theta = \vv{u}\cdot\vv{v}/(\left\lVert u\right\rVert\left\lVert v\right\rVert)$, and we would like to define the angle that way in general. A definition is only legal if the right-hand side lands in $[-1,1]$, and nothing we have written down so far guarantees that it does. Proving it is the next item, and it is much more than a technicality.

1.4 · The Cauchy–Schwarz inequality

The claim is that for all $u,v$ in an inner-product space,

with equality exactly when $u$ and $v$ are parallel.

The proof rests on one idea. Positivity of a norm is an infinite supply of inequalities, and we get to choose which one to use. Whatever $\lambda\in\C$ we pick, axiom (iii) guarantees that

That statement is true for every $\lambda$, which is a great deal of information. To get at it we have to see the $\lambda$ explicitly, so expand the right-hand side, remembering that the first slot conjugates by (0.5.3) and the second does not:

where the two middle terms combined because $\bar\lambda\avg{v,u}=\overline{\lambda\avg{u,v}}$.

Now let's spend the freedom we have been saving. This quantity is non-negative for every $\lambda$, so it is non-negative for the $\lambda$ that makes it smallest, and that is the choice worth making. Assume first that $v\neq0$. (If $v=0$ then both sides of (0.5.6) are zero and there is nothing to prove.) Then take

With that choice, $\lambda\avg{u,v} = \abs{\avg{u,v}}^{2}/\avg{v,v}$, which is real and non-negative, and $\abs{\lambda}^{2}\avg{v,v}$ comes out to the same thing. Substituting both into (0.5.8), the two copies partially cancel:

Multiply through by $\avg{v,v}\gt0$ and you have (0.5.6). Equality holds precisely when the quantity in (0.5.7) is zero, which is to say when $u=\lambda v$. That is exactly the statement that the two vectors are parallel.

So the angle is now legal. Divide (0.5.6) by $\left\lVert u\right\rVert^{2}\left\lVert v\right\rVert^{2}$ and it says $\abs{\avg{u,v}}/(\left\lVert u\right\rVert\left\lVert v\right\rVert)\le1$, so we may define

Grind box — the triangle inequality, and what $\lambda$ was

The triangle inequality falls out. Start from (0.5.5) and bound the cross term. Two facts do it, applied in order: $\mathrm{Re}\,z\le\abs{z}$, and then Cauchy–Schwarz.

$$\begin{aligned} \left\lVert u+v\right\rVert^{2} &= \left\lVert u\right\rVert^{2}+\left\lVert v\right\rVert^{2}+2\,\mathrm{Re}\avg{u,v}\\[3pt] &\le \left\lVert u\right\rVert^{2}+\left\lVert v\right\rVert^{2}+2\left\lVert u\right\rVert\left\lVert v\right\rVert = \big(\left\lVert u\right\rVert+\left\lVert v\right\rVert\big)^{2}. \end{aligned}$$

Taking square roots gives $\left\lVert u+v\right\rVert\le\left\lVert u\right\rVert+\left\lVert v\right\rVert$. Every metric-space fact you have ever used about these spaces descends from this line, and this line descends from (0.5.6).

What the magic $\lambda$ was. It looks pulled from a hat. It is not. Minimise (0.5.8) honestly and it appears on its own. Write $\lambda = t\,\overline{\avg{u,v}}$ with $t$ real, so that $\lambda\avg{u,v}=t\abs{\avg{u,v}}^{2}$ is real. Then the expression is a real quadratic in $t$:

$$q(t) = \avg{u,u} - 2t\abs{\avg{u,v}}^{2} + t^{2}\abs{\avg{u,v}}^{2}\avg{v,v},$$

and a real quadratic is minimised where its derivative vanishes, so set that derivative to zero:

$$q'(t) = 0 \;\Longrightarrow\; t = \frac{1}{\avg{v,v}},$$

which reproduces (0.5.9) exactly. And the minimising vector $\lambda v = \dfrac{\avg{v,u}}{\avg{v,v}}\,v$ is precisely the orthogonal projection of $u$ onto the line through $v$, which is the subject of §3. So Cauchy–Schwarz is the statement that a projection is never longer than the thing being projected. Everything in this chapter is one geometric picture seen from different angles.

An alternative proof, for the file. If $\left\lVert u\right\rVert=\left\lVert v\right\rVert=1$ then $0\le\left\lVert u-\ee^{\ii\varphi}v\right\rVert^{2}=2-2\,\mathrm{Re}(\ee^{\ii\varphi}\avg{u,v})$. Choose $\varphi$ to be minus the argument of $\avg{u,v}$, which rotates that product onto the positive real axis, and the real part becomes $\abs{\avg{u,v}}$, which gives $\abs{\avg{u,v}}\le1$ directly. The general case follows by rescaling. Same idea, with phase doing the work instead of magnitude.

2 · Orthonormal bases

Chapter 0.4 showed that any basis lets you write every vector uniquely in coordinates. It also showed that finding those coordinates means solving a linear system, which is work. With an inner product in hand, one class of basis makes that work vanish altogether.

A set $\{e_1,\dots,e_n\}$ is orthonormal if

Orthonormal sets are automatically linearly independent, so in dimension $n$ any $n$ of them form a basis. Here is the one-line reason. Suppose $\sum_i c_i e_i=0$, and take $\avg{e_j,\cdot}$ of both sides. Linearity in the second slot gives $\sum_i c_i\avg{e_j,e_i}=c_j=0$ for every $j$, so every coefficient vanishes.

That same computation is the payoff, so let's run it again on a vector we actually care about. Write any $v=\sum_i c_i e_i$ and hit it with $\avg{e_j,\cdot}$:

The double sum has collapsed to a single term, and the $j$-th coefficient has fallen out on its own with nothing solved. Renaming the index, that gives the expansion we will use for the rest of the book:

Coordinates have become inner products. There is no system to solve and no matrix to invert. To find the $i$-th coordinate you take one inner product. This is the single practical reason orthonormal bases dominate physics.

Notice also that the physics convention is already paying for itself. Because the second slot is the linear one, $\avg{e_i,v}$ is the coefficient exactly, with no stray conjugate attached to it.

Two more consequences follow, and both come from the same move. Substitute (0.5.14) into an inner product, then use (0.5.12) to collapse the resulting double sum:

The second of those is Parseval's identity: the squared length of a vector is the sum of the squared magnitudes of its coordinates. The words "in an orthonormal basis" are load-bearing here, and the identity is false without them.

It is worth knowing now where Parseval gets spent, because it gets spent twice. In Chapter 0.9 the $e_i$ become the Fourier modes, and Parseval becomes the statement that total energy equals the sum of the energies in each mode. In Chapter 4.2 the $\abs{\avg{e_i,v}}^{2}$ become probabilities, and Parseval becomes the statement that they add to one.

2.1 · Gram–Schmidt: orthonormal bases always exist

The formulae above are worthless if orthonormal bases are rare. They are not rare, and what proves it is a recipe rather than an existence argument. Given any basis $\{v_1,\dots,v_n\}$, the following procedure manufactures an orthonormal one. Set

running $k=1,2,\dots,n$ in order. For $k=1$ the sum is empty, so $e_1=v_1/\left\lVert v_1\right\rVert$. In words: take the next vector, subtract off everything it has in common with the directions already fixed, and normalise what is left.

Grind box — why Gram–Schmidt works, line by line

Three things must be checked: that we never divide by zero, that the output is orthonormal, and that it spans the same space. All three come from one induction hypothesis:

$H_k$:   $\{e_1,\dots,e_k\}$ is orthonormal and $\mathrm{span}\{e_1,\dots,e_k\}=\mathrm{span}\{v_1,\dots,v_k\}$.

Base case. $v_1\neq0$ because it belongs to a basis, so $\left\lVert v_1\right\rVert\neq0$ and $e_1=v_1/\left\lVert v_1\right\rVert$ is a unit vector spanning the same line. $H_1$ holds.

No division by zero. Assume $H_{k-1}$. If $w_k$ were $0$, then by (0.5.16) $v_k=\sum_{i\lt k}\avg{e_i,v_k}e_i$ would lie in $\mathrm{span}\{e_1,\dots,e_{k-1}\}=\mathrm{span}\{v_1,\dots,v_{k-1}\}$. That contradicts the linear independence of the $v$'s. So $w_k\neq0$ and $e_k$ is defined.

Orthogonality. For any $j\lt k$, take the inner product of (0.5.16) with $e_j$ and use $\avg{e_j,e_i}=\delta_{ji}$:

$$\avg{e_j,w_k} = \avg{e_j,v_k} - \sum_{i\lt k}\avg{e_i,v_k}\,\delta_{ji} = \avg{e_j,v_k}-\avg{e_j,v_k} = 0.$$

Exactly one term of the sum survives, and it is exactly the term that cancels. Dividing by $\left\lVert w_k\right\rVert$ preserves this, so $e_k\perp e_j$ for all $j\lt k$, and $\left\lVert e_k\right\rVert=1$ by construction.

Same span. From (0.5.16), $e_k$ is a combination of $v_k$ and earlier $e$'s, hence of $v_1,\dots,v_k$. Rearranged, the same equation says $v_k$ is a combination of $e_1,\dots,e_k$. So each set spans the other's span, $H_k$ holds, and induction completes the proof at $k=n$. $\blacksquare$

Two consequences worth naming. First, every finite-dimensional inner-product space has an orthonormal basis, so the theorems below never have to assume one exists. Second, Gram–Schmidt is triangular: $e_k$ involves only $v_1,\dots,v_k$. Write that relationship as a matrix statement and you get $V=QR$, with $Q$ having orthonormal columns and $R$ upper triangular. That is the QR decomposition, and it is how a computer actually solves the least-squares problem of §3.

2.2 · Dirac notation, introduced where it is harmless

We now install the notation that Chapter 4.2 will use for everything. It is standard to meet it in the middle of learning quantum mechanics, where it acquires an entirely undeserved mystique. It is a piece of pure linear algebra and there is nothing in it. Here it is.

Write a vector $v\in V$ as $\ket{v}$, a ket. Now fix a vector $u$ and consider the map that sends any $w$ to the number $\avg{u,w}$. By axiom (ii) that map is linear in $w$, so it is a linear functional, an element of the space $V^{*}$ of linear maps $V\to\C$. (That space is itself a vector space, since sums and scalar multiples of linear maps are linear.) Call the functional $\bra{u}$, a bra. Then:

The bracket closes, and a closed bracket is a number.

Now leave the bracket open instead, and see what kind of object comes back. Define $\ket{u}\bra{v}$ to be the map that acts on $\ket{w}$ by applying $\bra{v}$ first and using the resulting number to scale $\ket{u}$:

Is that a linear map? Check it: $w\mapsto \avg{v,w}u$ is linear in $w$ because $\avg{v,\cdot}$ is, so yes. So the outer product $\ket{u}\bra{v}$ is an operator, while the inner product $\avg{u|v}$ is a number. In coordinates with respect to an orthonormal basis, the distinction is one you already know from Chapter 0.4:

Row times column is a number. Column times row is a matrix. That is the whole content.

Dirac notation is a device for making the type of an object visible in its shape. You can tell at a glance whether an expression is a scalar or an operator, without tracking any dimensions. Note also, from (0.5.3), that the map $u\mapsto\bra{u}$ is conjugate-linear: $\bra{\alpha u}=\bar\alpha\bra{u}$. The conjugates in (0.5.19) are that fact written out in coordinates.

2.3 · Completeness: the resolution of the identity

Now rewrite the expansion (0.5.14) in the new notation. Its coefficient $\avg{e_i,v}$ is $\avg{e_i|v}$, so

The regrouping in the second step is legal by (0.5.18). Since this holds for every $\ket{v}$, the operator in brackets acts as the identity on every vector. And two linear maps that agree on every vector are equal, so we may drop the $\ket{v}$ from both sides:

This is the completeness relation, also called the resolution of the identity. It says the basis is not missing any directions. Reconstruct any vector from its components and you get the vector back, with nothing left over.

It is used constantly, and always in the same way. You insert $\hat I$ into the middle of an expression in the form (0.5.21), and a hard object turns into a sum of easy ones. Chapter 4.2 does this on nearly every page. So does Chapter 0.9, where the sum becomes an integral over a continuum of modes.

3 · Projection, and why least squares is a right angle

Let $W\subseteq V$ be a subspace, meaning a subset closed under addition and scalar multiplication. Such a set is a vector space in its own right, and it inherits $V$'s inner product unchanged. By Gram–Schmidt it has an orthonormal basis $\{f_1,\dots,f_m\}$, and that basis is all we need in order to define the orthogonal projection onto $W$:

Compare that with (0.5.21). It is the same construction, truncated. We keep only the directions inside $W$ and throw the rest away.

Three properties follow, and all three are derived rather than assumed.

$P$ lands in $W$ and fixes $W$. $Pv$ is visibly a combination of the $f_j$, so $Pv\in W$. And if $w\in W$ then (0.5.14) applied inside $W$ says $w=\sum_j\avg{f_j,w}f_j=Pw$. Put those two together. For any $v$ at all, the vector $Pv$ lies in $W$, so applying $P$ to it again changes nothing, $P(Pv)=Pv$. In symbols,

Projecting something that has already been projected does nothing. That is what "projection" means, and (0.5.23) is the algebraic form of it.

$P$ is symmetric under the inner product. To see it, expand both sides with (0.5.22) and watch them meet in the middle:

The first equality uses conjugate-linearity in slot 1, in the form $\overline{\avg{f_j,u}}=\avg{u,f_j}$. In §4 this property will be written $P^{\dagger}=P$.

The residual is orthogonal to $W$. The residual is what the projection threw away, so take its inner product with each basis vector of $W$ in turn. For each $j$,

so $v-Pv\perp f_j$ for every $j$. That is enough to give $v-Pv\perp W$ entirely, because an inner product with any combination of the $f_j$ is the corresponding combination of zeros.

3.1 · Projection is the closest point — proved

Now the theorem that makes projection useful rather than merely tidy.

Claim: for every $w\in W$, $\left\lVert v-w\right\rVert\ge\left\lVert v-Pv\right\rVert$, with equality only for $w=Pv$.

We want to compare an arbitrary $w$ against $Pv$, so the move is to write the vector $v-w$ in terms of $Pv$. Add and subtract it:

The first piece is orthogonal to $W$ by (0.5.25). The second lies in $W$, because both $Pv$ and $w$ do. So the two pieces are orthogonal to each other, which means Pythagoras (0.5.5) applies with no cross term at all:

Let's look at what that last line is actually saying. The first term on the right does not depend on $w$ in any way. The second is non-negative, and it vanishes only when $w=Pv$. So the minimum is attained there and nowhere else. $\blacksquare$

Read (0.5.27) once more. It says "nearest point" and "perpendicular residual" are the same condition. Not similar, not related: the same. Every optimisation problem that turns out to have a linear-algebra answer is this theorem in disguise.

4 · The adjoint

Every operator on an inner-product space has a partner. What defines the partner is not a formula in coordinates but a piece of behaviour: how it interacts with the inner product.

Definition. The adjoint $A^{\dagger}$ of a linear map $A:V\to V$ is the linear map satisfying

Read it as a rule for moving an operator across the comma, and note that doing so costs you a dagger. No basis appears anywhere in (0.5.28), and that is the point. The adjoint is a property of $A$ and the inner product, not of any coordinate system.

A definition by behaviour owes us two checks: that such a map exists at all, and that there is only one of them. Both are short.

It is unique. Suppose $B$ and $C$ both satisfy (0.5.28). Then $\avg{u,(B-C)v}=0$ for every $u$ and $v$. We get to choose $u$, so choose $u=(B-C)v$. This gives $\left\lVert (B-C)v\right\rVert^{2}=0$, so $(B-C)v=0$ for every $v$, so $B=C$.

It exists. Fix an orthonormal basis and define $A^{\dagger}v \equiv \sum_i\avg{Ae_i,v}\,e_i$. This is linear in $v$, because the inner product is linear in slot 2. Now expand $u$ via (0.5.14) and compute the left-hand side of the definition:

The right-hand side of that chain is $\avg{u,A^{\dagger}v}$, which is what (0.5.28) asked for. Existence and uniqueness settled, in five lines.

4.1 · In an orthonormal basis it is the conjugate transpose

Chapter 0.4 defined the matrix of $A$ by $A e_j = \sum_i A_{ij}e_i$. Take $\avg{e_i,\cdot}$ of that and use (0.5.12), and you get a formula worth remembering: $A_{ij}=\avg{e_i,Ae_j}$. A matrix element is an inner product.

Now compute the matrix of $A^{\dagger}$ the same way. Use (0.5.28) to move the operator across the comma, then conjugate symmetry to turn the result around:

So $A^{\dagger}$ is the conjugate transpose: transpose the matrix and conjugate the entries. That familiar recipe is not the definition. It is what the definition (0.5.28) looks like once you commit to an orthonormal basis. Note the hypothesis, because it is an easy one to lose. In a non-orthonormal basis (0.5.30) is flatly false.

Three algebraic rules follow immediately from (0.5.28), each by moving operators across the comma one at a time:

The last one is worth doing out loud, since it is the one people misremember: $\avg{ABu,v}=\avg{Bu,A^{\dagger}v}=\avg{u,B^{\dagger}A^{\dagger}v}$, and uniqueness does the rest. The order reverses, exactly as it does for the transpose and the inverse. One more rule gets used constantly below and is derived the same way: $\big(\ket{u}\bra{v}\big)^{\dagger}=\ket{v}\bra{u}$.

4.2 · The three species of operator

Name	Condition	What it is for
Hermitian (self-adjoint)	$A^{\dagger}=A$	Observables. §6 is about these
Unitary	$U^{\dagger}U=\hat I$	Symmetries, time evolution, change of orthonormal basis
Normal	$AA^{\dagger}=A^{\dagger}A$	The largest class that can be diagonalised orthonormally

Both Hermitian and unitary operators are normal. For a Hermitian operator the reason is that $AA^{\dagger}=A^{2}=A^{\dagger}A$. For a unitary one it takes an extra step: $U^{\dagger}U=\hat I$ implies, in finite dimensions, that $U$ is invertible with $U^{-1}=U^{\dagger}$, and therefore $UU^{\dagger}=\hat I$ as well.

For a Hermitian matrix, (0.5.30) says $A_{ij}=\overline{A_{ji}}$. Read that entry by entry. The diagonal entries are real, and entries across the diagonal are conjugates of each other. A real Hermitian matrix is nothing but a symmetric matrix.

4.3 · Unitary maps preserve all geometry

This is a two-line calculation with large consequences. We want to know what $U$ does to an inner product, so move $U$ across the comma using its defining property and watch it cancel. If $U^{\dagger}U=\hat I$ then for any $u,v$:

The inner product is untouched, and everything geometric follows from that one line. Setting $v=u$ gives $\left\lVert Uu\right\rVert=\left\lVert u\right\rVert$, so lengths are preserved. Feeding both facts into (0.5.11) leaves $\cos\theta$ unchanged, so angles are preserved too. And taking $u,v$ from an orthonormal basis shows $\{Ue_i\}$ is again orthonormal, so unitary maps are exactly the maps that carry orthonormal bases to orthonormal bases.

A real unitary matrix satisfies $U^{\!\top}U=I$, which is precisely an orthogonal matrix, meaning a rotation or a reflection. Unitary is the complex analogue of rotation. Hold that next to the last section of Chapter 0.3, where $\ee^{\ii\theta}$ turned out to be a rotation of the plane. The connection is not an analogy, and §7 will make it an identity.

Here is where this goes. In Chapter 4.6 the state of a quantum system evolves by $\ket{\psi(t)}=U(t)\ket{\psi(0)}$, and $U(t)$ must be unitary because $\left\lVert \psi\right\rVert^{2}=1$ is a total probability and probabilities have to keep summing to one. Equation (0.5.32) is that requirement written in linear algebra, and it is the reason time evolution in quantum mechanics is a rotation rather than a flow that stretches.

5 · Eigenvalues and eigenvectors

A linear map generally does two things at once to a vector. It turns the vector, and it rescales it. Some directions are special, and they are the ones where only the second thing happens.

Definition. A non-zero $v$ is an eigenvector of $A$ with eigenvalue $\lambda$ if

Geometrically, $v$ spans a direction the map does not turn. It only stretches it, by the factor $\lambda$. A negative $\lambda$ flips it, and a complex $\lambda$ means, as we will see in a moment, that no such real direction exists at all. The requirement $v\neq0$ is essential, since $A0=\lambda 0$ holds for every $\lambda$ and would make the definition vacuous.

To find these directions we need something we can solve, so rewrite (0.5.33) as $(A-\lambda\hat I)v=0$. A non-zero solution exists exactly when $A-\lambda\hat I$ fails to be invertible, which by Chapter 0.4 happens exactly when its determinant vanishes. So the eigenvalues are the roots of the characteristic polynomial:

Expanding the determinant of an $n\times n$ matrix produces a polynomial of degree $n$ in $\lambda$, with leading term $(-\lambda)^{n}$. Which numbers you are allowed to use now matters enormously. Over $\R$ a polynomial need not have any roots at all, since $\lambda^{2}+1$ has none, and correspondingly a real matrix can have no real eigenvectors whatsoever. A rotation of the plane turns every direction, so none is preserved. Over $\C$ this cannot happen. As recorded in Chapter 0.4, every non-constant polynomial has a complex root, so

That single sentence is why quantum mechanics is built over $\C$ and not $\R$, and it is the hypothesis that makes §6 work.

Two pieces of vocabulary before we go on. The set of eigenvectors for a given $\lambda$, together with $0$, forms the eigenspace $E_\lambda=\ker(A-\lambda\hat I)$, a subspace of dimension at least one. An eigenvalue whose eigenspace has dimension greater than one is called degenerate.

5.1 · Seeing it

Before proving anything, look at the phenomenon. The figure below applies a $2\times2$ real matrix to every vector on the unit circle. The image is an ellipse. Take that as observed for the moment, since §6 will prove it, and the proof turns out to be the spectral theorem itself. A rotating input vector is drawn together with its image, and the eigen-directions are marked, meaning the angles at which input and image become parallel.

preset symmetric non-symmetric collapse rotation

input angle θ

θ = 62.00°

skew c

c = 0.000

snap θ to eigen-direction

A = [ 2.000 0.866 ; 0.866 1.000 ]

λ = 2.500, 0.500 eigen-directions 30.0°, 120.0° → 90.0° apart

∠(v, Av) = 24.88°

The figure's whole behaviour comes out of one small computation, and it is worth doing so that the sliders stop being magic. Write the family it uses as a symmetric matrix plus an antisymmetric one:

The eigenvalues are the roots $\lambda = \tfrac12\big(\operatorname{tr}A \pm\sqrt{\mathcal D}\big)$ of the quadratic (0.5.34), so what decides whether they are real is the discriminant. Form it:

At $c=0$ the discriminant is a sum of squares. It is therefore non-negative, and the eigenvalues of a real symmetric $2\times2$ matrix are always real. That is a preview of §6(a).

Turning on $c$ eats into the discriminant, and the eigenvalues stay real only until $c^{2}=\big(\tfrac{a-d}{2}\big)^{2}+b^{2}$, which is exactly the squared half-gap between the eigenvalues of the symmetric part. In the figure $a=2$, $b=\sqrt3/2$, $d=1$, so the symmetric matrix has eigenvalues $2.5$ and $0.5$ and a half-gap of $1$. The collapse therefore happens at $c=1$, which is what the button does.

6 · The spectral theorem

This is the centre of the chapter and of Part 0. Everything before it was preparation, and a great deal after it is consequence. The theorem comes in three parts, and we take them in order.

6.1 · (a) Hermitian ⇒ every eigenvalue is real

Let $A^{\dagger}=A$ and $Av=\lambda v$ with $v\neq0$. Since $A$ is Hermitian, the defining property (0.5.28) reads $\avg{Au,w}=\avg{u,Aw}$. We want a statement about $\lambda$ on its own, so apply that with $u=w=v$ and evaluate each side using (0.5.33):

The first step used conjugate-linearity in slot 1, and the last used linearity in slot 2. Since $v\neq0$, axiom (iii) gives $\avg{v,v}\gt0$, so we may divide by it:

Three lines. The entire asymmetry between the two slots of the inner product has been converted into a statement that a number is real.

6.2 · (b) Hermitian ⇒ eigenvectors with distinct eigenvalues are orthogonal

Let $Av=\lambda v$ and $Aw=\mu w$ with $\lambda\neq\mu$, both real by (a). We want to show that $\avg{v,w}=0$, and the route there is to compute one quantity in two different ways. Take $\avg{v,Aw}$:

The last step is where reality is spent, since it needs $\bar\lambda=\lambda$ from (a). The two expressions have to agree, so subtract one from the other:

since $\mu-\lambda\neq0$. Different eigenvalues force exactly perpendicular eigenvectors, not merely independent ones. Nothing was assumed about $A$ except $A^{\dagger}=A$.

6.3 · (c) The theorem

Before the proof, let's look at what that last equation says. Compare it with completeness (0.5.21). The identity is $\sum_i\ket{e_i}\bra{e_i}$, and $A$ is the same sum with the terms weighted by the eigenvalues. An operator that would be complicated in a random basis is, in its own eigenbasis, nothing but a list of numbers attached to a list of perpendicular directions.

Grant the orthonormal eigenbasis for a moment, and the formula is immediate. Both sides are linear maps, so it is enough to check that they agree on every basis vector, and they do:

By Chapter 0.4, two linear maps agreeing on a basis are equal. So the work is entirely in producing the basis, and producing it is what the grind box does.

Grind box — the induction proof of the spectral theorem

We induct on $n=\dim V$. The claim to be proved at each stage is: every Hermitian operator on an $n$-dimensional complex inner-product space has an orthonormal basis of eigenvectors.

Base case $n=1$. Pick any unit vector $e_1$. It spans $V$, so $Ae_1$ is a multiple of it, $Ae_1=\lambda_1 e_1$. Done, and $\lambda_1$ is real by part (a).

Step 1, get one eigenvector. Let $\dim V = n\ge2$ and assume the claim in dimension $n-1$. Because $V$ is complex, (0.5.35) supplies an eigenvalue $\lambda_1$ and an eigenvector, which we normalise to a unit vector $e_1$. By (a), $\lambda_1\in\R$.

Step 2, pass to the orthogonal complement. The plan is to peel off the line through $e_1$ and apply the induction hypothesis to whatever is left, so define

$$W \;=\; e_1^{\perp} \;=\; \{\,w\in V : \avg{e_1,w}=0\,\}.$$

It is a subspace, because $\avg{e_1,\cdot}$ is linear. It inherits the inner product of $V$ and every axiom still holds on a subset, so $W$ is a complex inner-product space in its own right. We claim $\dim W = n-1$. To see it, split any $v\in V$ using the projection of §3 onto the line through $e_1$:

$$v \;=\; \underbrace{\avg{e_1,v}\,e_1}_{\in\,\mathrm{span}\{e_1\}} \;+\; \underbrace{\big(v-\avg{e_1,v}e_1\big)}_{\in\,W},$$

and the second bracket lies in $W$ by (0.5.25). So if $\{f_2,\dots,f_m\}$ is any basis of $W$, then $\{e_1,f_2,\dots,f_m\}$ spans $V$. It is also independent: applying $\avg{e_1,\cdot}$ to a vanishing combination $\alpha e_1+\sum_j\beta_jf_j=0$ kills every $f_j$ term and leaves $\alpha=0$, after which the independence of the $f_j$ forces every $\beta_j=0$. So it is a basis, and since dimension is well defined (Chapter 0.4), $1+\dim W = n$.

Step 3, $A$ maps $W$ into $W$. This is the step that uses Hermiticity, and it is the hinge of the whole proof. For $w\in W$,

$$\avg{e_1,Aw} = \avg{Ae_1,w} = \avg{\lambda_1 e_1,w} = \lambda_1\avg{e_1,w} = 0,$$

using $A^{\dagger}=A$ in the first step and $\bar\lambda_1=\lambda_1$ in the third. So $Aw\in W$, which is to say the restriction $A|_{W}$ is a genuine operator on $W$. Without Hermiticity this fails. A general operator does not preserve the orthogonal complement of one of its eigenvectors, and the induction dies right here. That is precisely why the figure in §5 loses its perpendicular eigen-directions the moment you break the symmetry.

Step 4, the restriction is still Hermitian. For $u,w\in W$, the defining property is inherited verbatim, because it is a statement about inner products of vectors that all lie in $W$:

$$\avg{A|_{W}u,\,w} = \avg{Au,w} = \avg{u,Aw} = \avg{u,\,A|_{W}w}.$$

Step 5, induct. By the induction hypothesis $W$ has an orthonormal basis $e_2,\dots,e_n$ of eigenvectors of $A|_{W}$. But $A|_{W}$ acts as $A$ does, so these are eigenvectors of $A$. Each lies in $W$, hence is orthogonal to $e_1$, and they are orthonormal among themselves. So $\{e_1,e_2,\dots,e_n\}$ is an orthonormal set of $n$ eigenvectors in an $n$-dimensional space, which makes it a basis. $\blacksquare$

What was used. Exactly three things: that $\C$ guarantees an eigenvalue (0.5.35), that Hermiticity makes the orthogonal complement invariant, and that the property survives restriction. Remove any one of the three and the theorem is false.

6.4 · Degeneracy, and the honest statement

Nothing above assumed the eigenvalues were distinct, so let's ask what happens when they are not. If $\lambda$ is degenerate, the induction hands you several basis vectors with the same eigenvalue, and any orthonormal basis of the eigenspace $E_\lambda$ will do. The choice is not unique, and no theorem prefers one. What is unique is the eigenspace itself.

That suggests writing the decomposition in terms of eigenspaces rather than eigenvectors, since only the eigenspaces are forced on us. Group the terms by distinct eigenvalue $\lambda_k$ and write $P_k$ for the projection onto $E_{\lambda_k}$:

The middle identity holds because eigenvectors from different eigenspaces are orthogonal by (b). The right-hand one is completeness (0.5.21), regrouped. This is the basis-independent form, and it is the one that survives to infinite dimensions in Chapter 4.5.

Finally, the matrix version. Assemble the orthonormal eigenvectors as the columns of a matrix $U$. Orthonormality of the columns is exactly the statement $U^{\dagger}U=\hat I$, so $U$ is unitary, and $Ae_i=\lambda_ie_i$ for every column reads $AU=UD$. We want $A$ by itself, so multiply on the right by $U^{-1}=U^{\dagger}$:

In the language of Chapter 0.4 this is a change of basis, and the change of basis is unitary, which is to say a rotation. Every Hermitian operator is a diagonal one, seen from a rotated angle.

Two dividends fall out at once, using cyclicity of the trace from Chapter 0.4:

6.5 · The real case, and the ellipse we borrowed

A real symmetric matrix is Hermitian, so (a) gives it real eigenvalues. For a real $\lambda$ the matrix $A-\lambda I$ is real and singular, so it has a real null vector. And Gram–Schmidt inside each eigenspace uses only real arithmetic. Put those together and a real symmetric matrix has an orthonormal basis of real eigenvectors, so that (0.5.44) becomes $A=ODO^{\!\top}$ with $O$ real orthogonal. That is the classical principal axes theorem.

That closes the loan taken out in §5. Let $A$ be real, invertible, acting on the unit circle. A point $x$ of the image is $x=Au$ with $\left\lVert u\right\rVert=1$, which is the same as saying $\left\lVert A^{-1}x\right\rVert=1$. Squaring that condition turns it into a quadratic form set equal to one:

Now we are in a position to use the theorem, because $M$ is exactly the kind of matrix it governs. Note that $M^{\!\top}=M$, and that $x^{\!\top}\!Mx=\left\lVert A^{-1}x\right\rVert^{2}\gt0$ for $x\neq0$. So $M$ is real symmetric with strictly positive eigenvalues $\mu_i$, each $\mu_i=u_i^{\!\top}Mu_i\gt0$. Substituting $x=Oy$ turns (0.5.46) into $\sum_i\mu_iy_i^{2}=1$, an ellipse with semi-axes $1/\sqrt{\mu_i}$ along the eigenvectors of $M$. The image of a circle under any invertible linear map is an ellipse, and the reason is the theorem you just proved.

One detail is worth extracting, because the figure in §5 shows it. The ellipse's axes are the eigenvectors of $M$, not of $A$. If $A=ODO^{\!\top}$ is symmetric then $M=OD^{-2}O^{\!\top}$ has the same eigenvectors and the two coincide, which is why the eigen-lines lie exactly along the ellipse in the symmetric preset. Break the symmetry and they part company, visibly.

7 · Functions of operators

You know what $\ee^{x}$ means for a number. What could $\ee^{A}$ mean for an operator? The spectral theorem answers it at once. If $A$ acts on each $e_i$ by multiplying by $\lambda_i$, then whatever $f(A)$ is, it ought to act on $e_i$ by multiplying by $f(\lambda_i)$. So define

for $A$ Hermitian with spectral decomposition $A=\sum_i\lambda_i\ket{e_i}\bra{e_i}$, and any function $f$ defined on the eigenvalues.

There is one question to settle before we trust that definition. Does the answer depend on which orthonormal basis we happened to pick inside a degenerate eigenspace? In the projection form (0.5.43) the definition reads $f(A)=\sum_k f(\lambda_k)P_k$, and the $P_k$ are unique, so the answer is no. Equation (0.5.47) is well posed.

It had better also agree with the answer you would get by substituting $A$ into a power series, and it does. The key is that orthonormality makes the cross terms vanish, which is easiest to see by squaring $A$ and watching the double sum collapse:

By the same collapse, inductively, $A^{m}=\sum_i\lambda_i^{m}\ket{e_i}\bra{e_i}$ for every $m\ge1$. It holds for $m=0$ as well, since $A^{0}=\hat I=\sum_i\ket{e_i}\bra{e_i}$ by completeness. So every power behaves, and that is enough to push a whole series through. For any $f(z)=\sum_m c_m z^{m}$ whose series converges at every eigenvalue,

Swapping the two sums is legitimate because the sum over $i$ is finite. So (0.5.47) and "plug the matrix into the series" agree wherever both make sense. And (0.5.47) keeps working for functions with no power series at all, such as $\sqrt{\cdot}$ on a positive operator.

7.1 · The exponential of $\ii$ times a Hermitian operator

Claim: if $A$ is Hermitian then $U=\ee^{\ii A}$ is unitary.

Unitarity is a statement about the adjoint, so the first thing we need is the adjoint. Take it of (0.5.47) with $f(\lambda)=\ee^{\ii\lambda}$, using $(\alpha M)^{\dagger}=\bar\alpha M^{\dagger}$ and $\big(\ket{e}\bra{e}\big)^{\dagger}=\ket{e}\bra{e}$ from (0.5.31):

The middle step is where reality is spent. It uses $\overline{\ee^{\ii\lambda}}=\ee^{-\ii\bar\lambda}$, and $\bar\lambda=\lambda$ holds only because $A$ is Hermitian. Now we have both factors of $U^{\dagger}U$, so multiply them together, collapsing with orthonormality exactly as in (0.5.48):

So $\ee^{\ii A}$ is unitary. $\blacksquare$

The converse holds as well: every unitary operator is $\ee^{\ii A}$ for some Hermitian $A$, and the grind box proves it. Put the two halves together and Hermitian operators and unitary operators are the same information, related by an exponential. That is exactly how real numbers and points on the unit circle are related by $\theta\mapsto\ee^{\ii\theta}$ in Chapter 0.3. This is that statement with the number replaced by an operator.

Grind box — the converse: every unitary is $\ee^{\ii A}$

We need a spectral theorem for unitary operators. The induction of §6 goes through with two substitutions, and here they are.

Eigenvalues have modulus one. If $U\ket{v}=\mu\ket{v}$ with $v\neq0$ then by (0.5.32), $\avg{v,v}=\avg{Uv,Uv}=\abs{\mu}^{2}\avg{v,v}$, so $\abs{\mu}=1$ and we may write $\mu=\ee^{\ii\theta}$ with $\theta$ real.

The orthogonal complement is invariant. In finite dimensions $U^{\dagger}=U^{-1}$, and $Uv=\mu v$ gives $U^{\dagger}v=\mu^{-1}v=\bar\mu v$ since $\abs{\mu}=1$. So for $w\perp v$,

$$\avg{v,Uw} = \avg{U^{\dagger}v,w} = \avg{\bar\mu v,w} = \mu\avg{v,w} = 0,$$

which says $Uw\perp v$. The restriction $U|_{W}$ preserves inner products on $W$ and maps $W$ into $W$. Being injective on a finite-dimensional space it is onto, hence unitary on $W$.

Induct exactly as before, using (0.5.35) to start. The result is an orthonormal basis $\{f_i\}$ with $Uf_i=\ee^{\ii\theta_i}f_i$, so

$$U = \sum_i \ee^{\ii\theta_i}\ket{f_i}\bra{f_i} = \ee^{\ii A}, \qquad A \equiv \sum_i\theta_i\ket{f_i}\bra{f_i}.$$

That $A$ is Hermitian because the $\theta_i$ are real and each $\ket{f_i}\bra{f_i}$ is self-adjoint. $\blacksquare$ One caveat is worth recording. The $\theta_i$ are only defined modulo $2\pi$, so $A$ is not unique, and the exponential map is many-to-one, exactly as $\ee^{\ii\theta}$ is on $\R$.

8 · Commuting operators and simultaneous diagonalisation

Define the commutator $[A,B]\equiv AB-BA$. It measures the failure of two operators to be interchangeable, and it is zero exactly when order does not matter. The theorem below is the reason that quantity is worth a name.

Theorem. Two Hermitian operators $A$ and $B$ on a finite-dimensional complex inner-product space admit a common orthonormal basis of eigenvectors if and only if $[A,B]=0$.

8.1 · Forward direction (easy)

Suppose $\{e_i\}$ is an orthonormal basis with $Ae_i=a_ie_i$ and $Be_i=b_ie_i$. We want to compare $AB$ with $BA$, so apply each of them to a basis vector and see what comes out:

and $a_ib_i=b_ia_i$ because they are numbers. So $AB$ and $BA$ agree on a basis, hence are equal (Chapter 0.4), so $[A,B]=0$. $\blacksquare$

8.2 · Reverse direction (the real content)

Here the difficulty is degeneracy. If $A$ had $n$ distinct eigenvalues the argument would be three lines. It generally does not, and the honest proof must build the common basis inside each eigenspace.

Grind box — the reverse direction, degeneracy handled

Assume $A^{\dagger}=A$, $B^{\dagger}=B$, and $AB=BA$.

Step 1, decompose $V$ by $A$. By the spectral theorem, $V$ is the orthogonal direct sum of the eigenspaces of $A$ for its distinct eigenvalues $\lambda_1,\dots,\lambda_r$:

$$V = E_{\lambda_1}\oplus\cdots\oplus E_{\lambda_r}, \qquad E_{\lambda_k}\perp E_{\lambda_l}\ (k\neq l),$$

with the orthogonality being part (b) of §6 and the spanning being part (c).

Step 2, $B$ preserves each eigenspace. This is the only place the hypothesis $[A,B]=0$ is used. Let $v\in E_{\lambda_k}$, so $Av=\lambda_k v$. We want to know where $Bv$ sits, so feed it to $A$ and use the commutation to move $A$ past $B$:

$$A(Bv) = (AB)v = (BA)v = B(Av) = B(\lambda_k v) = \lambda_k (Bv),$$

so $Bv$ is again an eigenvector of $A$ with eigenvalue $\lambda_k$, or else zero, which lies in $E_{\lambda_k}$ too. Hence $B\big(E_{\lambda_k}\big)\subseteq E_{\lambda_k}$.

Step 3, the restriction is Hermitian. This runs exactly as in Step 4 of the spectral-theorem proof. For $u,w\in E_{\lambda_k}$, $\avg{Bu,w}=\avg{u,Bw}$ holds because it holds in all of $V$, and both vectors lie in the subspace. So $B|_{E_{\lambda_k}}$ is a Hermitian operator on the finite-dimensional inner-product space $E_{\lambda_k}$.

Step 4, apply the spectral theorem inside the eigenspace. Therefore $E_{\lambda_k}$ has an orthonormal basis of eigenvectors of $B$. And every vector in $E_{\lambda_k}$ is automatically an eigenvector of $A$, with eigenvalue $\lambda_k$, because that is what the eigenspace is. So this basis consists of simultaneous eigenvectors of both operators. This is exactly where degeneracy is handled. Within a degenerate eigenspace of $A$, the operator $B$ chooses the basis that $A$ could not.

Step 5, assemble. Take the union of these bases over $k=1,\dots,r$. Vectors from the same eigenspace are orthonormal by construction, and vectors from different eigenspaces are orthogonal by Step 1. The total count is $\sum_k\dim E_{\lambda_k}=\dim V$, so the union is an orthonormal basis of $V$ consisting of simultaneous eigenvectors of $A$ and $B$. $\blacksquare$

Remark. Run the same argument with three or more mutually commuting Hermitian operators and it produces a basis of simultaneous eigenvectors for all of them. Apply Step 2 to each in turn, refining the decomposition as far as the operators allow. Whether that refinement reaches eigenspaces of dimension one depends on the set: it does exactly when the operators are enough to tell every basis vector apart by its eigenvalues, and a set that is not enough still yields a common eigenbasis while leaving some joint eigenspace of dimension two or more. Chapter 4.9 §3 turns that distinction into a criterion.

The theorem promises that a common eigenbasis exists. It does not promise that the one your solver just handed you is it, and the figure below is that distinction, driven to the point where it bites.

gap δ in A

δ = 0.600

off-diagonal c in B

c = 0.600

let B choose inside the eigenspace

add a third operator C

reset

eigenvalues of A: -1.000000, 1.000000, 1.600000 · gap = 6.000e-1

‖[A,B]‖_F = 0.50911688

v1 = ( 0.0000, 0.0000, 1.0000) a = -1.000000 ⟨B⟩ = 2.000000 ΔB = 0.0e+0

v2 = ( 0.0000, 1.0000, 0.0000) a = 1.000000 ⟨B⟩ = 0.000000 ΔB = 0.600000

v3 = ( 1.0000, 0.0000, 0.0000) a = 1.600000 ⟨B⟩ = 0.000000 ΔB = 0.600000

not a common eigenbasis: largest spread 0.600000

9 · Worked examples

10 · Your turn