Chapter 13 · Introduction

Chapter 13 — Inner Product Spaces & Orthonormality

Lengths, angles and perpendicularity on abstract vector spaces: inner products, orthonormal bases, projections and least squares, the adjoint, and the self-adjoint, normal and unitary operators of the spectral theorem.

On New Year's Day 1801 Giuseppe Piazzi in Palermo found a faint moving point of light — the dwarf planet Ceres — and followed it for six weeks before it vanished into the glare of the Sun. Astronomers had too few, too noisy observations to say where it would reappear. The 24-year-old Carl Friedrich Gauss predicted the position, and Ceres was found again almost exactly where he said. His method, published in 1809 (Legendre had published the same idea in 1805), was least squares: among all orbits, choose the one that makes the sum of the squared errors smallest. In the language of this chapter, Gauss projected the vector of observations orthogonally onto a subspace. A century later David Hilbert and Erhard Schmidt carried the same geometry — lengths, angles, perpendicular projections — into spaces of functions, where Fourier series become expansions in an orthonormal basis and Parseval's identity is Pythagoras' theorem with infinitely many terms. Schmidt's orthogonalisation procedure, already used by Laplace and by Gram, is the Gram–Schmidt process of §13.1.

An inner product does more than measure: it pairs each linear map $T$ with an adjoint $T^*$, and the operators that interact well with their adjoints — self-adjoint ($T^*=T$), unitary ($T^*T=I$) and normal ($T^*T=TT^*$) — are exactly the ones that can be diagonalised by an orthonormal basis. That is the spectral theorem of §13.2, the summit of finite-dimensional linear algebra: a real symmetric matrix has real eigenvalues and perpendicular eigenvectors, a unitary matrix has eigenvalues on the unit circle, and a normal matrix is a diagonal matrix seen in a rotated orthonormal frame. From it come positive operators and their square roots, the singular value decomposition that every matrix — normal or not — admits, and quick estimates of where eigenvalues lie. Chapter 14 then turns the real spectral theorem into the principal axes of a quadratic form.

Inner products→Cauchy–Schwarz→Gram–Schmidt & $QR$→Projections & best approximation→Bessel & Parseval→Adjoint $T^*$→Least squares→Self-adjoint, normal, unitary→Spectral theorem→Positive operators & SVD→Gershgorin discs
🎯 By the end of this chapter you can…
  • decide whether a given formula is an inner product on $\mathbb{R}^n$, $\mathbb{C}^n$, a matrix or polynomial space, and use the Cauchy–Schwarz inequality, the parallelogram law and polarisation;
  • run the Gram–Schmidt process, write $A=QR$, use Gram determinants, and compute orthogonal complements, orthogonal projections, distances and best approximations (including orthogonal polynomials);
  • state and use Bessel's inequality and Parseval's identity, and solve least-squares problems through the normal equations $A^*A\hat x=A^*b$;
  • compute adjoints, use $\ker T^*=(\operatorname{im}T)^\perp$, and decide which idempotents are orthogonal projections;
  • recognise self-adjoint, unitary and normal matrices, read off their eigenvalue properties, and apply the complex and real spectral theorems to diagonalise, take powers and square roots;
  • test positive (semi)definiteness, compute a singular value decomposition and a polar decomposition, and locate eigenvalues with Rayleigh quotients and Gershgorin discs.
Teaser — What a Matrix Does to the Unit Circle

A unit vector $x$ (red) runs round the circle and $Ax$ (blue) traces the image curve (purple). The dashed green lines are the real eigen-directions, where $Ax$ is parallel to $x$. For a symmetric matrix they are perpendicular, they are the axes of the image ellipse, and the value $\langle Ax,x\rangle$ runs exactly between the two eigenvalues. Compare with a matrix that is not normal and with a rotation, which is normal but has no real eigen-direction.

x—
Ax—
⟨Ax, x⟩—
range of ⟨Ax, x⟩—
eigenvalues—
class—

How this is examined

CSIR NET draws steadily on this chapter, in both parts of the paper. Part B questions are usually short computations: the projection of a vector onto a line or plane, the distance from a point to a subspace, a polynomial orthogonal to $P_1$ for a given integral inner product, the value of a least-squares minimum, the eigenvalues of a Hermitian, unitary or orthogonal matrix built from a vector ($I-2uu^T$, $uu^*$), the singular values of a small matrix, or a bound on the eigenvalues from Gershgorin discs. Part C, where several options can be correct, tests the structure theory: "every normal matrix is diagonalisable by a unitary matrix", "a matrix with real eigenvalues is Hermitian", "if $P^2=P$ then $P$ is an orthogonal projection", "$A^*A$ and $AA^*$ have the same eigenvalues", "a positive definite matrix has positive diagonal entries", "a unitary matrix with real eigenvalues is Hermitian", "Gershgorin discs prove invertibility". Each false option is one of the counterexamples in the ⚠️ boxes of the two sections. Questions on the rank, signature and classification of quadratic forms, which use the spectral theorem, belong to Chapter 14. GATE MA asks very similar questions, often on Gram–Schmidt, projections and positive definiteness.

Sections in this chapter

✅ Before you start

This chapter uses the linear algebra of Chapters 9–12. You need bases, dimension and direct sums (§9.1); linear maps, their matrices, change of basis and the dual space (§9.2); rank, determinants and the special matrices — symmetric, Hermitian, orthogonal, unitary, idempotent — together with the conjugate transpose $A^*$ (§10.1). For §13.2 you also need eigenvalues, characteristic and minimal polynomials (§11.1), and diagonalisability, invariant subspaces and Schur's triangularisation (§12.1). Norms and the $\ell^2$ and $C[a,b]$ examples (§8.4), series of non-negative terms (§2.3) and the Weierstrass approximation theorem (§4.2) appear in examples. Complex numbers are used only through conjugation, $|z|^2=z\bar z$, and the fact that every complex polynomial has a root.