10.2 Self-Adjoint, Normal and Unitary Operators
Normal operators — those that commute with their adjoint — are exactly the operators that an orthonormal basis diagonalises. Self-adjoint, unitary and positive operators are the normal operators whose spectra are real, unimodular and non-negative; and every matrix, normal or not, becomes diagonal under two unitary changes of basis.
Chapter 9 asked when an operator can be diagonalised, and answered: exactly when its minimal polynomial splits into distinct linear factors. But a diagonalising basis found that way can be badly skewed — two eigenvectors may be almost parallel, and then the change of basis $P$ and its inverse have huge entries, and errors are magnified in every computation. An inner product lets us ask for much more: a basis of eigenvectors that is orthonormal. Then $P$ is unitary, $P^{-1}=P^*$ costs nothing to compute, lengths and angles are preserved, and the operator is literally a list of independent stretchings along perpendicular axes. The question of this section is: which operators admit such a basis? The answer — the spectral theorem — is strikingly clean: over $\mathbb{C}$ exactly the normal operators ($T^*T=TT^*$), over $\mathbb{R}$ exactly the self-adjoint ones.
The spectral theorem is the workhorse behind quantum mechanics (observables are self-adjoint, so they have real eigenvalues and orthogonal eigenstates), vibration analysis (normal modes), principal component analysis in statistics, and the classification of quadratic forms in §10.3. From it we get positive operators and their square roots, the polar decomposition, and the singular value decomposition, which diagonalises every matrix with two unitary matrices. CSIR NET examines this material relentlessly: "which of the following matrices are normal / unitarily diagonalisable / positive definite", eigenvalues of Hermitian and unitary matrices, $\|A\|$ versus the largest eigenvalue, square roots, and Gershgorin-type eigenvalue bounds.
Conventions. Throughout, $V$ is a finite-dimensional inner product space over $\mathbb{F}=\mathbb{R}$ or $\mathbb{C}$ (§10.1), with the inner product linear in the first variable and conjugate-linear in the second; on $\mathbb{F}^n$, $\langle x,y\rangle=y^*x=\sum_ix_i\overline{y_i}$. For $T\in \mathcal{L}(V)$ the adjoint $T^*$ is the unique operator with $\langle Tx,y\rangle=\langle x,T^*y\rangle$ for all $x,y$ (§10.1). We use freely: $(S+T)^*=S^*+T^*$, $(cT)^*=\bar cT^*$, $(ST)^*=T^*S^*$, $T^{**}=T$, $\ker T^*=(\operatorname{im}T)^\perp$, and, in an orthonormal basis, $[T^*]=[T]^*=\overline{[T]}^{\,T}$. For a matrix $A\in M_n(\mathbb{F})$ we use the operator $x\mapsto Ax$ on $\mathbb{F}^n$, whose adjoint is $x\mapsto A^*x$.
Self-Adjoint, Unitary and Normal Operators
An operator $T\in \mathcal{L}(V)$ is
- self-adjoint (or Hermitian) if $T^*=T$; for matrices: $A^*=A$ ($a_{ji}=\overline{a_{ij}}$), and for real matrices $A^T=A$ (symmetric);
- skew-adjoint (skew-Hermitian) if $T^*=-T$; for real matrices $A^T=-A$ (skew-symmetric);
- unitary if $T^*T=TT^*=I$; for matrices $U^*U=I$; a real unitary matrix ($Q^TQ=I$) is called orthogonal;
- normal if $T^*T=TT^*$, i.e. $T$ commutes with its adjoint.
Self-adjoint, skew-adjoint and unitary operators are all normal (in each case $T^*$ is $\pm T$ or $T^{-1}$, which commute with $T$). Diagonal matrices are normal, and so is every matrix $aI+bK$ with $K$ skew-symmetric and $a,b$ scalars, e.g. $\begin{pmatrix}a&-b\\b&a\end{pmatrix}$.
Think of the analogy with complex numbers: $A\mapsto A^*$ plays the role of $z\mapsto\bar z$. Hermitian matrices are the "real numbers", skew-Hermitian ones the "imaginary numbers", unitary ones the "numbers of modulus one", and every matrix splits as $A=H+iK$ with $H=\tfrac12(A+A^*)$ and $K=\tfrac1{2i}(A-A^*)$ both Hermitian (Exercise I.2). The analogy is exact for normal matrices and breaks down for the others — that is the content of the whole section.
- $T^*T=I$;
- $\langle Tx,Ty\rangle=\langle x,y\rangle$ for all $x,y\in V$ ($T$ preserves inner products);
- $\|Tx\|=\|x\|$ for all $x\in V$ ($T$ is an isometry);
- $T$ maps every orthonormal basis of $V$ to an orthonormal basis;
- $T$ maps some orthonormal basis of $V$ to an orthonormal basis.
Proof. (1)⇒(2): $\langle Tx,Ty\rangle=\langle x,T^*Ty\rangle=\langle x,y\rangle$. (2)⇒(3): take $y=x$. (3)⇒(2): the inner product is recovered from norms by the polarisation identities. Over $\mathbb{R}$, $\langle x,y\rangle=\tfrac14\left(\|x+y\|^2-\|x-y\|^2\right)$. Over $\mathbb{C}$, $$\langle x,y\rangle=\frac14\sum_{k=0}^{3}i^k\,\|x+i^ky\|^2 .$$ To check the complex one, expand $\|x+i^ky\|^2=\|x\|^2+\|y\|^2$ $+\overline{i^k}\langle x,y\rangle+i^k\langle y,x\rangle$ and multiply by $i^k$: the result is $i^k(\|x\|^2+\|y\|^2)+\langle x,y\rangle+i^{2k}\langle y,x\rangle$. Summing over $k=0,1,2,3$ kills the first and last terms (since $\sum i^k=\sum(-1)^k=0$) and leaves $4\langle x,y\rangle$. As $T$ preserves every norm on the right-hand side, it preserves $\langle x,y\rangle$. (2)⇒(4): $\langle Te_i,Te_j\rangle=\langle e_i,e_j\rangle=\delta_{ij}$, and $n$ orthonormal vectors in an $n$-dimensional space form a basis. (4)⇒(5) is immediate. (5)⇒(1): if $(e_i)$ and $(Te_i)$ are orthonormal, then for $x=\sum a_ie_i$ and $y=\sum b_je_j$, $$\langle Tx,Ty\rangle=\sum_{i,j}a_i\overline{b_j}\,\langle Te_i,Te_j\rangle=\sum_ia_i\overline{b_i}=\langle x,y\rangle,$$ so $\langle T^*Tx-x,y\rangle=0$ for all $y$, whence $T^*Tx=x$ for every $x$. Finally $T^*T=I$ makes $T$ injective, hence invertible ($V$ is finite-dimensional), and then $T^{-1}=T^*$ and $TT^*=I$. In an orthonormal basis $[T]^*[T]=[T^*T]=I$, and $|\det T|^2=\overline{\det T}\det T=\det(T^*T)=1$. ∎
- If $\mathbb{F}=\mathbb{C}$ and $\langle Tx,x\rangle=0$ for every $x\in V$, then $T=0$. Hence two operators on a complex space with $\langle Sx,x\rangle=\langle Tx,x\rangle$ for all $x$ are equal.
- If $\mathbb{F}=\mathbb{C}$, then $T$ is self-adjoint if and only if $\langle Tx,x\rangle\in\mathbb{R}$ for every $x$.
- If $\mathbb{F}=\mathbb{R}$, statement 1 fails (rotation of $\mathbb{R}^2$ through $\tfrac\pi2$ has $\langle Tx,x\rangle=0$ for all $x$), but it holds for self-adjoint $T$: if $T^*=T$ and $\langle Tx,x\rangle=0$ for all $x$, then $T=0$.
Proof. (1) The operator version of polarisation is $$4\langle Tx,y\rangle=\sum_{k=0}^{3}i^k\,\big\langle T(x+i^ky),\,x+i^ky\big\rangle,$$ verified exactly as above: the $k$-th term equals $i^k\big(\langle Tx,x\rangle+\langle Ty,y\rangle\big)$ plus $\langle Tx,y\rangle$ plus $i^{2k}\langle Ty,x\rangle$, and the sum over $k$ leaves $4\langle Tx,y\rangle$. If every $\langle Tz,z\rangle$ vanishes, then $\langle Tx,y\rangle=0$ for all $x,y$; with $y=Tx$ this gives $\|Tx\|^2=0$. Apply this to $S-T$ for the second sentence. (2) For every $x$, $$\langle Tx,x\rangle-\overline{\langle Tx,x\rangle}=\langle Tx,x\rangle-\langle x,Tx\rangle=\langle (T-T^*)x,x\rangle .$$ So all $\langle Tx,x\rangle$ are real iff $\langle(T-T^*)x,x\rangle=0$ for all $x$, iff $T=T^*$ by (1). (3) For the rotation $T(x_1,x_2)=(-x_2,x_1)$, $\langle Tx,x\rangle=-x_2x_1+x_1x_2=0$. If $T$ is self-adjoint on a real space, $\langle Ty,x\rangle=\langle y,Tx\rangle=\langle Tx,y\rangle$, so $$\langle T(x+y),x+y\rangle-\langle T(x-y),x-y\rangle=2\langle Tx,y\rangle+2\langle Ty,x\rangle=4\langle Tx,y\rangle,$$ and we finish as in (1). ∎
- If $T$ is self-adjoint, $\lambda\in\mathbb{R}$; and if $\lambda\ne\mu$ then $x\perp y$.
- If $T$ is skew-adjoint, $\lambda\in i\mathbb{R}$ (purely imaginary or $0$).
- If $T$ is unitary, $|\lambda|=1$; and if $\lambda\ne\mu$ then $x\perp y$.
- Every Hermitian matrix — in particular every real symmetric matrix — has a characteristic polynomial that splits over $\mathbb{R}$ into real linear factors. Hence every self-adjoint operator on a real inner product space of dimension $\ge1$ has an eigenvector.
Proof. (1) $\lambda\|x\|^2=\langle Tx,x\rangle=\langle x,Tx\rangle=\bar\lambda\|x\|^2$, so $\lambda=\bar\lambda$. Then $\mu$ is real too, and $\lambda\langle x,y\rangle=\langle Tx,y\rangle=\langle x,Ty\rangle=\bar\mu\langle x,y\rangle=\mu\langle x,y\rangle$, so $(\lambda-\mu)\langle x,y\rangle=0$. (2) $\lambda\|x\|^2=\langle Tx,x\rangle=-\langle x,Tx\rangle=-\bar\lambda\|x\|^2$, so $\lambda+\bar\lambda=2\operatorname{Re}\lambda=0$. (3) $\|x\|=\|Tx\|$ $=|\lambda|\,\|x\|$ by Theorem 10.19. Next $\langle x,y\rangle=\langle Tx,Ty\rangle=\lambda\bar\mu\langle x,y\rangle$; since $|\mu|=1$ we have $\bar\mu=1/\mu$, so $\lambda\bar\mu=\lambda/\mu\ne1$ when $\lambda\ne\mu$, forcing $\langle x,y\rangle=0$. (4) Regard a Hermitian $A\in M_n(\mathbb{C})$ (or a real symmetric one, for which $A^*=A^T=A$) as an operator on $\mathbb{C}^n$. By the fundamental theorem of algebra, $p_A(t)=\prod_{i=1}^n(t-\lambda_i)$ over $\mathbb{C}$, and each $\lambda_i$ is an eigenvalue of the Hermitian $A$, hence real by (1). So $p_A$ is a product of real linear factors. If $A$ is real and $\lambda$ is one of its (real) roots, then $\det(A-\lambda I)=0$ in $M_n(\mathbb{R})$, so a real eigenvector exists. For a self-adjoint operator on a real space apply this to its matrix in an orthonormal basis, which is symmetric because $[T]^T=[T^*]=[T]$. ∎
- $T$ is normal if and only if $\|Tx\|=\|T^*x\|$ for every $x\in V$.
- If $T$ is normal, $\ker T=\ker T^*$, and $T-\lambda I$ is normal for every scalar $\lambda$.
- If $T$ is normal, then $Tx=\lambda x\iff T^*x=\bar\lambda x$: $T$ and $T^*$ have the same eigenvectors, with conjugate eigenvalues.
- If $T$ is normal, eigenvectors for distinct eigenvalues are orthogonal.
- (Any $T$.) If a subspace $W$ is $T$-invariant, then $W^\perp$ is $T^*$-invariant. If $T$ is normal and $E_\lambda$ is an eigenspace, then $E_\lambda^\perp$ is invariant under both $T$ and $T^*$.
Proof. (1) Put $N=T^*T-TT^*$. Then $\|Tx\|^2-\|T^*x\|^2$ equals $\langle T^*Tx,x\rangle-\langle TT^*x,x\rangle=\langle Nx,x\rangle$. If $T$ is normal, $N=0$. Conversely, $N$ is self-adjoint, so $\langle Nx,x\rangle=0$ for all $x$ forces $N=0$ by Theorem 10.20 (parts 1 and 3 — this works over $\mathbb{R}$ and $\mathbb{C}$). (2) The first claim is (1). Next $(T-\lambda I)^*=T^*-\bar\lambda I$, and both products $(T-\lambda I)(T^*-\bar\lambda I)$ and $(T^*-\bar\lambda I)(T-\lambda I)$ expand to $TT^*-\bar\lambda T-\lambda T^*+|\lambda|^2I$ (using $TT^*=T^*T$). (3) Apply $\ker S=\ker S^*$ to the normal operator $S=T-\lambda I$. (4) If $Tx=\lambda x$ and $Ty=\mu y$, then by (3) $T^*y=\bar\mu y$ and $\lambda\langle x,y\rangle=\langle Tx,y\rangle=\langle x,T^*y\rangle=\langle x,\bar\mu y\rangle=\mu\langle x,y\rangle$. (5) For $u\in W^\perp$ and $w\in W$: $\langle T^*u,w\rangle=\langle u,Tw\rangle=0$ since $Tw\in W$. For normal $T$, $E_\lambda$ is invariant under $T$ and, by (3), under $T^*$ ($T^*x=\bar\lambda x$); applying the first statement to $T$ and to $T^*$ shows that $E_\lambda^\perp$ is invariant under $T^*$ and under $T^{**}=T$. ∎
The Spectral Theorem
- $T$ is normal;
- $V$ has an orthonormal basis consisting of eigenvectors of $T$;
- (matrix form) $A\in M_n(\mathbb{C})$ is unitarily diagonalisable: $A=UDU^*$ with $U$ unitary and $D$ diagonal.
Proof. (2)⇒(1): in an orthonormal eigenbasis $[T]=D$ is diagonal, so $[T^*]=D^*=\overline D$ is diagonal too, and diagonal matrices commute: $[T^*T]=\overline DD=D\overline D=[TT^*]$. (2)⇔(3): the columns of $U$ are the orthonormal eigenvectors, and $U^{-1}=U^*$.
(1)⇒(2). By Schur's unitary triangularisation (§9.2) there is an orthonormal basis $e_1,\dots,e_n$ in which $R=[T]=(r_{ij})$ is upper triangular. Because the basis is orthonormal, $[T^*]=R^*$ and $R^*R=RR^*$: the matrix $R$ is normal. We show that a normal upper triangular matrix is diagonal. Compare the $(1,1)$ entries of $R^*R$ and $RR^*$, i.e. $\|Re_1\|^2$ and $\|R^*e_1\|^2$ (Theorem 10.22(1)): $$\|Re_1\|^2=\sum_i|r_{i1}|^2=|r_{11}|^2,\qquad\|R^*e_1\|^2=\sum_j|r_{1j}|^2=|r_{11}|^2+\sum_{j\ge2}|r_{1j}|^2,$$ since the first column of $R$ has only one nonzero entry. Equality forces $r_{12}=\cdots=r_{1n}=0$. Then $R=\begin{pmatrix}r_{11}&0\\0&R'\end{pmatrix}$ with $R'$ upper triangular, and $R^*R=RR^*$ gives $R'^*R'=R'R'^*$; by induction on $n$, $R'$ is diagonal. So $R$ is diagonal and $e_1,\dots,e_n$ are eigenvectors. ∎
A second proof avoids Schur: take any eigenvector $e_1$ (one exists over $\mathbb{C}$), normalise it; by Theorem 10.22(5), $W=\{e_1\}^\perp$ is invariant under $T$ and $T^*$, the restriction $T|_W$ is normal (its adjoint is $T^*|_W$), and induction on $\dim V$ gives an orthonormal eigenbasis of $W$.
Proof. (⇐) In an orthonormal eigenbasis $[T]=D$ is diagonal, hence symmetric, so $[T^*]=D^T=D=[T]$. For matrices, $(QDQ^T)^T=QD^TQ^T=QDQ^T$.
(⇒) Induction on $n=\dim V$; $n=1$ is clear. By Theorem 10.21(4), the self-adjoint $T$ has a real eigenvalue $\lambda_1$ and a unit eigenvector $e_1$. The line $W=\operatorname{span}(e_1)$ is $T$-invariant, so $W^\perp$ is $T^*$-invariant (Theorem 10.22(5)), i.e. $T$-invariant. The restriction $S=T|_{W^\perp}$ is self-adjoint on the $(n-1)$-dimensional inner product space $W^\perp$, because $\langle Sx,y\rangle=\langle Tx,y\rangle=\langle x,Ty\rangle=\langle x,Sy\rangle$ for $x,y\in W^\perp$. By induction $W^\perp$ has an orthonormal basis $e_2,\dots,e_n$ of eigenvectors of $S$, hence of $T$, and $e_1,e_2,\dots,e_n$ is the required basis of $V=W\oplus W^\perp$. ∎
Over $\mathbb{R}$ "normal" is not enough: the rotation $\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ is orthogonal, hence normal, but has no real eigenvalue at all. The step that fails is the very first one — the existence of a real eigenvector — and Theorem 10.21(4) is exactly what supplies it for symmetric matrices.
- $V=E_1\oplus\cdots\oplus E_k$, the summands being mutually orthogonal;
- $P_j^2=P_j=P_j^*$, $P_iP_j=0$ for $i\ne j$, and $P_1+\cdots+P_k=I$;
- $T=\sum_j\lambda_jP_j$, $T^*=\sum_j\bar\lambda_jP_j$, and $f(T)=\sum_jf(\lambda_j)P_j$ for every polynomial $f$;
- $P_j=\ell_j(T)$, where $\ell_j(t)=\prod_{i\ne j}\dfrac{t-\lambda_i}{\lambda_j-\lambda_i}$ — each projection is a polynomial in $T$;
- conversely, if $T=\sum_j\mu_jQ_j$ with distinct scalars $\mu_j$ and nonzero orthogonal projections $Q_j$ satisfying $Q_iQ_j=0$ ($i\ne j$) and $\sum Q_j=I$, then the $\mu_j$ are exactly the eigenvalues of $T$ and $Q_j$ is the projection onto the eigenspace of $\mu_j$.
Proof. (1) By Theorems 10.23 and 10.24 the eigenvectors span $V$; eigenspaces for different eigenvalues are orthogonal (Theorem 10.22(4)). So each $x$ is uniquely $x=x_1+\cdots+x_k$ with $x_j\in E_j$, and these components are mutually orthogonal. (2) $P_jx=x_j$; the identities follow at once, and orthogonal projections are self-adjoint (§10.1). (3) $Tx=\sum_j\lambda_jx_j=\sum_j\lambda_jP_jx$; by induction $T^mx=\sum_j\lambda_j^mx_j$, hence $f(T)x=\sum_jf(\lambda_j)x_j$. For the adjoint, $(\sum\lambda_jP_j)^*=\sum\bar\lambda_jP_j^*=\sum\bar\lambda_jP_j$. (4) $\ell_j(\lambda_i)=\delta_{ij}$, so by (3) $\ell_j(T)=\sum_i\delta_{ij}P_i=P_j$. (5) For $x\in\operatorname{im}Q_j$, $Tx=\sum_i\mu_iQ_iQ_jx=\mu_jx$; so the $\operatorname{im}Q_j$ are eigenspaces, they are orthogonal and span $V$, and an eigenvector for any other value $\mu$ would be orthogonal to all of them, hence $0$. ∎
- $T$ is self-adjoint $\iff\sigma(T)\subseteq\mathbb{R}$;
- $T$ is skew-adjoint $\iff\sigma(T)\subseteq i\mathbb{R}$;
- $T$ is unitary $\iff\sigma(T)\subseteq\{z:|z|=1\}$;
- $T$ is an orthogonal projection $\iff\sigma(T)\subseteq\{0,1\}$;
- $T$ is nilpotent $\iff T=0$; more generally $\operatorname{rank}T^m=\operatorname{rank}T$ for all $m\ge1$;
- the operator norm $\|T\|=\max_{\|x\|=1}\|Tx\|$ equals the spectral radius $\rho(T)=\max_j|\lambda_j|$.
Proof. Write $T=\sum\lambda_jP_j$ and $T^*=\sum\bar\lambda_jP_j$ (Theorem 10.25). By the uniqueness in Theorem 10.25(5), $T^*=T$ iff $\bar\lambda_j=\lambda_j$ for every $j$, and $T^*=-T$ iff $\bar\lambda_j=-\lambda_j$. Next $T^*T=\sum|\lambda_j|^2P_j$, which equals $I=\sum P_j$ iff every $|\lambda_j|=1$. Also $T^2=\sum\lambda_j^2P_j$ equals $T$ iff $\lambda_j^2=\lambda_j$; such a $T$ is then self-adjoint by (1), and a self-adjoint idempotent is an orthogonal projection. For (5), $T^m=\sum\lambda_j^mP_j$ has the same nonzero eigenvalues (and the same eigenspaces) as $T$, so the same rank; if $T^m=0$ every $\lambda_j=0$ and $T=0$. For (6), by Pythagoras $\|Tx\|^2=\sum_j|\lambda_j|^2\|x_j\|^2$ $\le\rho^2\sum_j\|x_j\|^2=\rho^2\|x\|^2$, with equality for an eigenvector of an eigenvalue of largest modulus. ∎
Every implication "⇐" fails without normality: $C=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has $\sigma(C)=\{1\}$, which lies in $\mathbb{R}$, on the unit circle and in $\{0,1\}$, yet $C$ is not Hermitian, unitary or idempotent, and $\|C\|=\tfrac{1+\sqrt5}{2}\gt1=\rho(C)$ (Exercise III.4). Likewise $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ is nilpotent but not $0$.
- a real orthogonal matrix is orthogonally similar to $\operatorname{diag}\big(\pm1,\dots,\pm1,R_{\theta_1},\dots,R_{\theta_s}\big)$, where $R_\theta=\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}$;
- every $3\times3$ real orthogonal matrix with determinant $1$ is a rotation through some angle $\theta$ about an axis $\mathbb{R}u$ with $Qu=u$, and $\operatorname{tr}Q=1+2\cos\theta$;
- a real skew-symmetric matrix is orthogonally similar to $\operatorname{diag}\!\left(0,\dots,0,\begin{pmatrix}0&-b_1\\b_1&0\end{pmatrix},\dots\right)$; its rank is even and its nonzero eigenvalues are $\pm ib_j$.
Proof (sketch with all key steps). Real eigenvalues are handled as in Theorem 10.24: a unit eigenvector spans a line whose orthogonal complement is invariant under $A$ and $A^T$ (Theorem 10.22(5)). For a non-real eigenvalue $\lambda=a+ib$ ($b\ne0$) choose an eigenvector $z=x+iy\in\mathbb{C}^n$ with $x,y\in\mathbb{R}^n$. Since $A$ is real, $\bar z$ is an eigenvector for $\bar\lambda\ne\lambda$, so $\langle z,\bar z\rangle=0$ by Theorem 10.22(4). Expanding, $$\langle x+iy,\,x-iy\rangle=\|x\|^2-\|y\|^2+2i\langle x,y\rangle=0,$$ so $\|x\|=\|y\|\ne0$ and $x\perp y$. Comparing real and imaginary parts of $A(x+iy)=(a+ib)(x+iy)$ gives $Ax=ax-by$ and $Ay=bx+ay$; so in the orthonormal pair $\big(\tfrac{x}{\|x\|},-\tfrac{y}{\|y\|}\big)$ the restriction of $A$ has matrix $\begin{pmatrix}a&-b\\b&a\end{pmatrix}$. By Theorem 10.22(3), $A^Tz=\bar\lambda z$, so the plane $\operatorname{span}(x,y)$ is invariant under $A^T$ as well; its orthogonal complement is invariant under $A$ and $A^T$, and induction finishes. For (1), orthogonal blocks have $|\lambda|=1$, i.e. $\lambda=\pm1$ or $a^2+b^2=1$. For (2), with $n=3$ the normal form is either $\operatorname{diag}(\varepsilon,R_\theta)$ with $\varepsilon=\pm1$ — and $\det=\varepsilon\det R_\theta=\varepsilon$ forces $\varepsilon=1$ — or $\operatorname{diag}(\varepsilon_1,\varepsilon_2,\varepsilon_3)$ with product $1$, i.e. $I$ or (up to order) $\operatorname{diag}(1,-1,-1)=\operatorname{diag}(1,R_\pi)$. In every case it is $\operatorname{diag}(1,R_\theta)$, whose trace is $1+2\cos\theta$. For (3), skew-symmetric blocks have $a=0$ and real eigenvalues $0$. ∎
- Check the hypothesis. Real: is $A^T=A$? Complex: is $A^*A=AA^*$? (Hermitian, skew-Hermitian, unitary and circulant matrices are normal; for anything else compute both products.)
- Eigenvalues. Use $\operatorname{tr}$, $\det$, obvious eigenvectors (row sums, $(1,1,\dots,1)$), and the fact that the spectrum must be real / imaginary / unimodular for the special classes.
- Eigenvectors. Eigenvectors for different eigenvalues are automatically orthogonal — only normalise them. Inside an eigenspace of dimension $\ge2$, run Gram–Schmidt (§10.1); a non-orthogonal basis of one eigenspace is the only place where work is needed.
- Assemble and check. $U$ (or $Q$) has the orthonormal eigenvectors as columns; check $U^*AU=D$ on one column. For functions of $A$, use $f(A)=\sum f(\lambda_j)P_j$ with $P_j=\sum_{\text{basis of }E_j}u\,u^*$.
Positive Operators, Square Roots, Polar and Singular Value Decompositions
An operator $T\in \mathcal{L}(V)$ is positive semidefinite, written $T\succeq0$, if $T$ is self-adjoint and $\langle Tx,x\rangle\ge0$ for all $x\in V$; it is positive definite, $T\succ0$, if moreover $\langle Tx,x\rangle\gt0$ for every $x\ne0$. A matrix $A\in M_n(\mathbb{F})$ is positive (semi)definite if $A=A^*$ and $x^*Ax\gt0$ for all $x\ne0$ ($\ge0$ for all $x$). $T$ is negative (semi)definite if $-T$ is positive (semi)definite, and an indefinite self-adjoint operator is one that is neither positive nor negative semidefinite. We write $S\succeq T$ when $S-T\succeq0$.
Over $\mathbb{C}$ the self-adjointness requirement is automatic (Theorem 10.20(2)); over $\mathbb{R}$ it is part of the definition: $\begin{pmatrix}1&1\\-1&1\end{pmatrix}$ satisfies $x^TAx=x_1^2+x_2^2\gt0$ but is not symmetric, and is not called positive definite. Quadratic forms (§10.3) are the matrix side of the same notion.
- $T\succeq0$;
- every eigenvalue of $T$ is $\ge0$;
- $T=S^*S$ for some $S\in \mathcal{L}(V)$ (for matrices: $A=B^*B$, i.e. $A$ is the Gram matrix $a_{ij}=\langle b_j,b_i\rangle$ of the columns $b_j$ of $B$);
- $T=R^2$ for some self-adjoint $R\succeq0$.
Proof. (1)⇒(2): if $Tx=\lambda x$ with $x\ne0$, then $\lambda\|x\|^2=\langle Tx,x\rangle\ge0$. (2)⇒(4): write $T=\sum\lambda_jP_j$ (Theorem 10.25) with all $\lambda_j\ge0$ and put $R=\sum\sqrt{\lambda_j}\,P_j$. Then $R^*=R$, $R^2=\sum\lambda_jP_j=T$, and $\langle Rx,x\rangle=\sum_j\sqrt{\lambda_j}\,\|x_j\|^2\ge0$. (4)⇒(3): take $S=R$. (3)⇒(1): $(S^*S)^*=S^*S$ and $\langle S^*Sx,x\rangle=\langle Sx,Sx\rangle=\|Sx\|^2\ge0$. In the definite case each inequality becomes strict, and $\|Sx\|\gt0$ for all $x\ne0$ means exactly that $S$ is injective, i.e. invertible. The form $\langle Tx,y\rangle$ is linear in $x$, conjugate symmetric because $T=T^*$, and positive definite iff $T\succ0$. For the matrix facts: $a_{ii}=e_i^*Ae_i\gt0$; $\operatorname{tr}A$ and $\det A$ are the sum and product of the positive eigenvalues; for an index set $I$ and $x$ supported on $I$, $x^*Ax=x_I^*A_Ix_I$, so the principal submatrix $A_I$ is positive definite, and for $I=\{i,j\}$ its determinant $a_{ii}a_{jj}-|a_{ij}|^2$ is positive; finally $A^{-1}$ is Hermitian with eigenvalues $1/\lambda_j\gt0$. ∎
Proof. Existence is (2)⇒(4) above, and $\sqrt T=\sum\sqrt{\lambda_j}\,\ell_j(T)$ is a polynomial in $T$ by Theorem 10.25(4). Uniqueness: let $R\succeq0$ with $R^2=T$. As $R$ is self-adjoint, $R=\sum_i\mu_iQ_i$ with distinct eigenvalues $\mu_i\ge0$ and orthogonal eigenprojections $Q_i$, and then $T=R^2=\sum_i\mu_i^2Q_i$. Because $\mu\mapsto\mu^2$ is injective on $[0,\infty)$, the numbers $\mu_i^2$ are distinct, so by Theorem 10.25(5) they are the eigenvalues $\lambda_j$ of $T$ and the $Q_i$ are its eigenprojections $P_j$. Thus $\mu_i=\sqrt{\lambda_j}$ and $R=\sum\sqrt{\lambda_j}P_j$ — there was no choice. The last statement holds because $\sqrt{\lambda}\gt0\iff\lambda\gt0$. ∎
Without positivity, square roots are far from unique: every reflection $\begin{pmatrix}\cos\theta&\sin\theta\\\sin\theta&-\cos\theta\end{pmatrix}$ squares to $I$, so $I_2$ has infinitely many square roots, and $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has none at all (a square root $X$ would satisfy $X^4=0$, hence $X^2=0$ for a $2\times2$ matrix).
- $Av_i=\sigma_iu_i$ and $A^*u_i=\sigma_iv_i$; $v_1,\dots,v_r$ is an orthonormal basis of $\operatorname{im}A^*$, $v_{r+1},\dots,v_n$ of $\ker A$, $u_1,\dots,u_r$ of $\operatorname{im}A$, and $u_{r+1},\dots,u_m$ of $\ker A^*$;
- $\|A\|=\max_{\|x\|=1}\|Ax\|=\sigma_1$ and $\|A\|_F^2=\sum_{i,j}|a_{ij}|^2=\operatorname{tr}(A^*A)=\sum_i\sigma_i^2$;
- the nonzero eigenvalues of $AA^*$ are also $\sigma_1^2,\dots,\sigma_r^2$; if $m=n$, $|\det A|=\sigma_1\sigma_2\cdots\sigma_n$;
- if $A$ is normal with eigenvalues $\lambda_i$, its singular values are the $|\lambda_i|$;
- (Eckart–Young, statement) for $k\lt r$, $A_k=\sum_{i\le k}\sigma_iu_iv_i^*$ is a closest matrix of rank $\le k$ to $A$: $\min_{\operatorname{rank}B\le k}\|A-B\|=\|A-A_k\|=\sigma_{k+1}$.
Proof. By the spectral theorem applied to the positive semidefinite $A^*A$ there is an orthonormal basis $v_1,\dots,v_n$ of $\mathbb{F}^n$ with $A^*Av_i=\sigma_i^2v_i$, ordered so that $\sigma_1\ge\cdots\ge\sigma_n$; exactly $r$ of the $\sigma_i$ are positive (Definition 10.11). Then $$\langle Av_i,Av_j\rangle=\langle A^*Av_i,v_j\rangle=\sigma_i^2\langle v_i,v_j\rangle=\sigma_i^2\delta_{ij},$$ so $u_i=Av_i/\sigma_i$ ($i\le r$) are orthonormal, while $Av_i=0$ for $i\gt r$. Extend $u_1,\dots,u_r$ to an orthonormal basis $u_1,\dots,u_m$ of $\mathbb{F}^m$ (Gram–Schmidt, §10.1). Then $AV=U\Sigma$ column by column, i.e. $A=U\Sigma V^*$. (1) $A^*u_i=A^*Av_i/\sigma_i=\sigma_iv_i$; the subspace statements follow since $\operatorname{im}A=\operatorname{span}(Av_1,\dots,Av_n)$, which is $\operatorname{span}(u_1,\dots,u_r)$, while $\ker A=\operatorname{span}(v_{r+1},\dots,v_n)$, and the other two are orthogonal complements ($\ker A^*=(\operatorname{im}A)^\perp$, $\operatorname{im}A^*=(\ker A)^\perp$). (2) For $x=\sum c_iv_i$, $\|Ax\|^2=\sum_i\sigma_i^2|c_i|^2\le\sigma_1^2\|x\|^2$, with equality at $x=v_1$; and $\operatorname{tr}(A^*A)$ is the sum of its eigenvalues. (3) $AA^*=U\Sigma\Sigma^TU^*$ has eigenvalues $\sigma_i^2$ and zeros; $|\det A|=|\det\Sigma|=\prod\sigma_i$ because $|\det U|=|\det V|=1$. (4) If $A=WDW^*$ with $W$ unitary then $A^*A=W\overline DDW^*=W|D|^2W^*$. (5) is quoted without proof. ∎
Proof. From the SVD, $A=U\Sigma V^*=(UV^*)(V\Sigma V^*)$. Here $W=UV^*$ is unitary, and $P=V\Sigma V^*$ is self-adjoint with eigenvalues $\sigma_i\ge0$ and $P^2=V\Sigma^2V^*=A^*A$; by Theorem 10.29 it is the positive square root of $A^*A$. If $A=WP$ is any such factorisation then $A^*A=PW^*WP=P^2$, which forces $P=\sqrt{A^*A}$. If $A$ is invertible, so is $P$, and $W=AP^{-1}$. Likewise $A=(U\Sigma U^*)(UV^*)$ gives $A=P'W$. ∎
Locating Eigenvalues: Rayleigh Quotients and Gershgorin Discs
- $\lambda_n\le R(x)\le\lambda_1$ for all $x\ne0$; $R(x)=\lambda_1$ iff $x$ is an eigenvector for $\lambda_1$ (similarly for $\lambda_n$). Thus $\lambda_1=\max_{\|x\|=1}\langle Tx,x\rangle$ and $\lambda_n=\min_{\|x\|=1}\langle Tx,x\rangle$.
- The set $\{\langle Tx,x\rangle:\|x\|=1\}$ (the numerical range) is the whole interval $[\lambda_n,\lambda_1]$. In particular every diagonal entry $a_{ii}=e_i^*Ae_i$ of a Hermitian matrix lies in $[\lambda_{\min},\lambda_{\max}]$.
- (Courant–Fischer) $\displaystyle\lambda_k=\max_{\dim W=k}\ \min_{x\in W\setminus\{0\}}R(x)$ $\displaystyle{}=\min_{\dim W=n-k+1}\ \max_{x\in W\setminus\{0\}}R(x).$
Proof. (1) Write $x=\sum c_ie_i$. Then $\langle Tx,x\rangle=\sum_i\lambda_i|c_i|^2$ and $\langle x,x\rangle=\sum|c_i|^2$, so $R(x)$ is a weighted average of the $\lambda_i$ with weights $|c_i|^2/\|x\|^2$; it lies between the smallest and the largest, and equals $\lambda_1$ iff all the weight sits on indices with $\lambda_i=\lambda_1$, i.e. $x\in E_{\lambda_1}$. (2) For $x_t=\cos t\,e_1+\sin t\,e_n$ ($0\le t\le\frac\pi2$), $\|x_t\|=1$ and $\langle Tx_t,x_t\rangle=\lambda_1\cos^2t+\lambda_n\sin^2t$ runs continuously from $\lambda_1$ to $\lambda_n$. (3) For $W_0=\operatorname{span}(e_1,\dots,e_k)$, every $x\in W_0$ has weights only on $\lambda_1,\dots,\lambda_k$, so $R(x)\ge\lambda_k$, with equality at $e_k$. For any $W$ with $\dim W=k$, the subspace $Z=\operatorname{span}(e_k,\dots,e_n)$ has dimension $n-k+1$, so $\dim(W\cap Z)\ge k+(n-k+1)-n=1$; a nonzero $x\in W\cap Z$ has $R(x)\le\lambda_k$. Hence the maximum over $W$ of the minimum is exactly $\lambda_k$. The min–max form is the same argument with the roles exchanged. ∎
- Every eigenvalue of $A$ lies in $D_1\cup D_2\cup\cdots\cup D_n$.
- The same holds for the column discs $\{|z-a_{jj}|\le\sum_{i\ne j}|a_{ij}|\}$, since $A^T$ has the same eigenvalues; so the eigenvalues lie in the intersection of the two unions.
- If the union of $k$ of the discs is disjoint from the union of the remaining $n-k$, it contains exactly $k$ eigenvalues, counted with algebraic multiplicity.
Proof. (1) Let $Ax=\lambda x$, $x\ne0$, and choose $i$ with $|x_i|=\max_j|x_j|\gt0$. Row $i$ of $Ax=\lambda x$ reads $(\lambda-a_{ii})x_i=\sum_{j\ne i}a_{ij}x_j$, so $$|\lambda-a_{ii}|\le\sum_{j\ne i}|a_{ij}|\,\frac{|x_j|}{|x_i|}\le\sum_{j\ne i}|a_{ij}|=R_i ,$$ i.e. $\lambda\in D_i$. (2) Apply (1) to $A^T$. (3) (Sketch.) Let $D=\operatorname{diag}(a_{11},\dots,a_{nn})$ and $A(t)=D+t(A-D)$ for $0\le t\le1$. The discs of $A(t)$ have the same centres and radii $tR_i$, so they stay inside the $D_i$: if $\mathcal U$ is the union of the chosen $k$ discs and $\mathcal V$ the union of the others, every eigenvalue of every $A(t)$ lies in $\mathcal U\cup\mathcal V$, and $\mathcal U\cap\mathcal V=\varnothing$. The roots of $p_{A(t)}$ depend continuously on $t$ — e.g. the number of roots inside a closed curve $\Gamma$ around $\mathcal U$ that avoids $\mathcal U\cup\mathcal V$ is $\frac1{2\pi i}\oint_\Gamma p_{A(t)}'/p_{A(t)}\,dz$, a continuous integer-valued function of $t$, hence constant. At $t=0$ the eigenvalues are the $a_{ii}$, and exactly $k$ of them (the centres of the chosen discs) lie in $\mathcal U$. ∎
- $A$ is invertible (Lévy–Desplanques).
- If moreover $A$ is Hermitian with every $a_{ii}\gt0$, then $A\succ0$.
Proof. (1) $|0-a_{ii}|\gt R_i$ for each $i$, so $0$ lies in no disc and is not an eigenvalue. (2) The eigenvalues are real (Theorem 10.21) and lie in $\bigcup[a_{ii}-R_i,a_{ii}+R_i]\subseteq(0,\infty)$; apply Theorem 10.28. For the last claim, the isolated disc contains exactly one eigenvalue $\lambda$ by Theorem 10.33(3). Its centre $a_{ii}$ is real, so the disc is symmetric about $\mathbb{R}$, and $\bar\lambda$ — also an eigenvalue, since $p_A$ has real coefficients — lies in it too. Hence $\bar\lambda=\lambda$. ∎
Weak dominance is not enough: $\begin{pmatrix}1&1\\1&1\end{pmatrix}$ has $|a_{ii}|=R_i$ and is singular.
- Real eigenvalues do not make a matrix symmetric: $\begin{pmatrix}1&1\\0&2\end{pmatrix}$. Only for normal matrices does the spectrum determine the class (Theorem 10.26).
- $|\lambda|=1$ for all eigenvalues does not make a matrix unitary: $\begin{pmatrix}1&1\\0&1\end{pmatrix}$ — not normal, $\|A\|=\tfrac{1+\sqrt5}2$.
- Normal over $\mathbb{R}$ is not orthogonally diagonalisable over $\mathbb{R}$: the rotation $\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ has no real eigenvalue. Over $\mathbb{R}$ the spectral theorem needs symmetric.
- Diagonalisable is not unitarily diagonalisable: $\begin{pmatrix}1&1\\0&2\end{pmatrix}$ has eigenvectors $(1,0)$ and $(1,1)$, which are not orthogonal. Similar does not imply unitarily similar ($\operatorname{tr}A^*A=6$ vs $5$ for $\operatorname{diag}(1,2)$).
- Products and sums: the product of two Hermitian matrices is Hermitian only if they commute (Example 7(a)); the sum of two normal matrices need not be normal: $\begin{pmatrix}0&1\\1&0\end{pmatrix}+\begin{pmatrix}0&1\\-1&0\end{pmatrix}=\begin{pmatrix}0&2\\0&0\end{pmatrix}$.
- "$x^TAx\gt0$ for all $x\ne0$" is not positive definiteness for non-symmetric real $A$: $\begin{pmatrix}1&1\\-1&1\end{pmatrix}$ has eigenvalues $1\pm i$.
- Positive entries $\ne$ positive definite: $\begin{pmatrix}1&2\\2&1\end{pmatrix}$ has eigenvalue $-1$; and $\begin{pmatrix}2&-1\\-1&2\end{pmatrix}\succ0$ has negative entries. What a positive definite matrix must have is a positive diagonal and $|a_{ij}|^2\lt a_{ii}a_{jj}$.
- $\|A\|\ne\rho(A)$ unless $A$ is normal: $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has $\rho=0$ and $\|A\|=1$; Example 5 has $\rho=5\lt3\sqrt5$.
- Singular values are not $|\text{eigenvalues}|$ in general: $\{3\sqrt5,\sqrt5\}$ versus $\{5,3\}$ in Example 5. They agree exactly for normal matrices.
- Square roots: $I_2$ has infinitely many square roots (all reflections), only one of them positive; $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has none.
- A single Gershgorin disc may contain no eigenvalue: for $\begin{pmatrix}0.5&1\\4&-0.5\end{pmatrix}$ the disc $|z-0.5|\le1$ misses both eigenvalues $\pm\sqrt{4.25}$; only the union (or an isolated group) is guaranteed.
- Weak diagonal dominance does not give invertibility: $\begin{pmatrix}1&1\\1&1\end{pmatrix}$.
- Unitary matrices are diagonalisable over $\mathbb{C}$, but real orthogonal ones need not be over $\mathbb{R}$ (rotations); and a real orthogonal matrix need not have $\det=1$ (reflections).
Exercises
- 1 Show that $A=\begin{pmatrix}1&2i\\-2i&5\end{pmatrix}$ is Hermitian and find its eigenvalues.
- 2 Prove that every $A\in M_n(\mathbb{C})$ can be written uniquely as $A=H+iK$ with $H,K$ Hermitian, and that $A$ is normal if and only if $HK=KH$.
- 3 Find all $a,b\in\mathbb{C}$ for which $\begin{pmatrix}a&b\\-\bar b&\bar a\end{pmatrix}$ is unitary.
- 4 Prove that a normal nilpotent operator is $0$, and that a normal operator $T$ satisfies $\ker T^2=\ker T$.
- 5 Let $T$ be self-adjoint with $T^k=I$ for some $k\ge1$. Prove that $T^2=I$.
- 6 Let $T$ be normal. Prove that $\operatorname{im}T=\operatorname{im}T^*$ and $V=\ker T\oplus\operatorname{im}T$ with $\ker T\perp\operatorname{im}T$. Give a $2\times2$ matrix for which $\ker A\cap\operatorname{im}A\ne\{0\}$.
- 7 Prove that an upper triangular unitary matrix is diagonal, with diagonal entries of modulus $1$.
- 8 (Cayley transform.) Let $K$ be skew-Hermitian. Prove that $I+K$ is invertible and that $U=(I-K)(I+K)^{-1}$ is unitary.
- 1 Orthogonally diagonalise $\begin{pmatrix}3&1\\1&3\end{pmatrix}$.
- 2 Orthogonally diagonalise $\begin{pmatrix}1&2&2\\2&1&2\\2&2&1\end{pmatrix}$.
- 3 Unitarily diagonalise $\begin{pmatrix}2&i\\-i&2\end{pmatrix}$.
- 4 For $B=\begin{pmatrix}0&1\\1&0\end{pmatrix}$ find the spectral projections and compute $e^{tB}$ from $e^{tB}=\sum e^{t\lambda_j}P_j$.
- 5 Find the eigenvalues of the circulant $\begin{pmatrix}1&2&3&4\\4&1&2&3\\3&4&1&2\\2&3&4&1\end{pmatrix}$ and its norm $\|C\|$.
- 6 Prove that every $3\times3$ real orthogonal $Q$ with $\det Q=1$ has $1$ as an eigenvalue. Find the axis and angle of the rotation $Q=\begin{pmatrix}0&0&1\\1&0&0\\0&1&0\end{pmatrix}$.
- 7 Prove that a real normal $2\times2$ matrix is either symmetric or of the form $\begin{pmatrix}a&b\\-b&a\end{pmatrix}$ with $b\ne0$.
- 8 Prove that two commuting normal operators $S,T$ on a complex space have a common orthonormal eigenbasis.
- 1 Decide whether $\begin{pmatrix}2&-1&0\\-1&2&-1\\0&-1&2\end{pmatrix}$ is positive definite by finding its eigenvalues.
- 2 Find the positive square root of $\begin{pmatrix}10&6\\6&10\end{pmatrix}$.
- 3 Prove that $A^*A\succeq0$ for every $A\in M_{m\times n}(\mathbb{C})$, and that $A^*A\succ0$ iff the columns of $A$ are linearly independent.
- 4 Find the singular values and $\|A\|$ for $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$, and compare with $\rho(A)$.
- 5 Find the polar decomposition of $\begin{pmatrix}0&-2\\1&0\end{pmatrix}$.
- 6 Let $A\succ0$ and $B\succeq0$. Prove that $AB$ is diagonalisable with real non-negative eigenvalues, though $AB$ need not be Hermitian.
- 7 Prove that $\operatorname{tr}(AB)\ge0$ for $A,B\succeq0$, and that $A\succeq0$ with $\operatorname{tr}A=0$ implies $A=0$.
- 8 Prove that a positive semidefinite matrix with $a_{ii}=0$ has its whole $i$-th row and column equal to $0$.
- 1 Use Gershgorin discs to locate the eigenvalues of $\begin{pmatrix}5&1&0\\1&0&1\\0&1&-5\end{pmatrix}$. Can you decide from the discs whether the matrix is invertible?
- 2 Show without computing eigenvalues that $\begin{pmatrix}4&1&1\\1&5&2\\1&2&6\end{pmatrix}$ is positive definite.
- 3 For the matrix in 2, use the Rayleigh quotient with $x=(1,1,1)^T$ to give a lower bound for $\lambda_{\max}$, and compare with a Gershgorin upper bound.
- 4 Prove $\|A\|=\rho(A)$ for normal $A$, and give a non-normal $A$ with $\|A\|\gt\rho(A)$ and $\rho(A)=0$.
- 5 Find the numerical range $\{\langle Ax,x\rangle:\|x\|=1\}$ of $\begin{pmatrix}2&1\\1&2\end{pmatrix}$.
- 6 Applied (vibrations). Two carts of mass $1\ \text{kg}$ are joined to two walls and to each other by three springs of stiffness $1\ \text{N/m}$, so $x''=-Kx$ with $K=\begin{pmatrix}2&-1\\-1&2\end{pmatrix}$. Find the normal-mode frequencies and mode shapes, and explain why the mode shapes are orthogonal.
- 7 Applied (statistics). Two measurements have covariance matrix $\Sigma=\begin{pmatrix}5&2\\2&2\end{pmatrix}$ (in $\text{cm}^2$). Find the first principal direction and the fraction of the total variance it explains.
E1. CSIR Part B GATE MA A real symmetric $3\times3$ matrix $A$ has eigenvalues $2,2,5$, and $(1,1,1)^T$ is an eigenvector for the eigenvalue $5$. With $J$ the all-ones matrix, $A$ equals (A) $2I+J$; (B) $5I-J$; (C) $2I+3J$; (D) $3I+J$.
E2. CSIR Part B Which matrix is normal but neither Hermitian, skew-Hermitian nor unitary? (A) $\begin{pmatrix}1&1\\0&1\end{pmatrix}$; (B) $\begin{pmatrix}1&-2\\2&1\end{pmatrix}$; (C) $\begin{pmatrix}0&i\\i&0\end{pmatrix}$; (D) $\begin{pmatrix}2&i\\-i&2\end{pmatrix}$.
E3. CSIR Part B GATE MA For $A=\begin{pmatrix}1&2\\0&1\end{pmatrix}$, the operator norm $\|A\|=\max_{\|x\|_2=1}\|Ax\|_2$ is (A) $1$; (B) $\sqrt5$; (C) $1+\sqrt2$; (D) $3$.
E4. CSIR Part C Let $A\in M_n(\mathbb{C})$ be normal. Which of the following are true? (One or more options may be correct.) (A) If all eigenvalues of $A$ are real, then $A$ is Hermitian. (B) Every eigenvector of $A$ is an eigenvector of $A^*$. (C) $\operatorname{rank}A=\operatorname{rank}A^2$. (D) If $A$ has real entries, then $A=QDQ^T$ for some real orthogonal $Q$ and real diagonal $D$.
E5. CSIR Part C Let $A$ be a real $n\times n$ positive definite matrix, $n\ge2$. Which of the following are true? (A) If $B$ is also positive definite, then $AB$ is positive definite. (B) $A+A^{-1}-2I$ is positive semidefinite. (C) The entry of $A$ of largest absolute value lies on the diagonal. (D) There is exactly one real symmetric matrix $S$ with $S^2=A$.
E6. CSIR Part C Let $A=\begin{pmatrix}10&1&2\\1&4&1\\2&0&-6\end{pmatrix}$. Which of the following are true? (A) $A$ is invertible. (B) Exactly two eigenvalues of $A$ have positive real part. (C) $A$ has a pair of non-real eigenvalues. (D) $\det A\lt0$.
Suggested plan — 3 lectures × 60 min.
- The special classes. Definition 10.9 with the complex-number analogy; Theorem 10.19 (prove the polarisation identity on the board — it is used twice); Theorem 10.20 with the rotation counterexample; Theorem 10.21 in full; Theorem 10.22(1)–(4). Example 1 as a class activity (each row of students takes one matrix).
- The spectral theorem. Theorem 10.23 via Schur (the "first row" argument is the heart — do it slowly), Theorem 10.24 by induction, then Theorem 10.25 and Theorem 10.26 with the warning examples. Example 2 on the board; Example 3 to show a normal matrix that is not Hermitian. Project Interactive 1 with the two symmetric presets: the green eigen-lines are the axes of the ellipse.
- Positivity, SVD and eigenvalue location. Theorems 10.28–10.29 with Example 4; the SVD (Theorem 10.30) and Example 5, then Interactive 1 again with "Example 5" and "shear" to contrast singular values with eigenvalues. Gershgorin (Theorem 10.33) with Example 6 and Interactive 2 — let students predict where the eigenvalues end before pressing Play. Finish with E4 or E6.
Discussion questions. (1) Over $\mathbb{R}$, why is "normal" not enough for orthogonal diagonalisation, and what replaces diagonal form (Theorem 10.27)? (2) If you know only the eigenvalues of a matrix, what can you say about $\|A\|$? What if you also know $A$ is normal? (3) Why does a positive definite matrix have a unique positive square root, while $I$ has infinitely many square roots?
Misconceptions to watch for. Deciding "Hermitian" or "unitary" from the eigenvalues of a non-normal matrix; forgetting to orthonormalise inside a repeated eigenspace (and orthonormalising across different eigenspaces, which is unnecessary); confusing singular values with $|\lambda|$; calling a non-symmetric real matrix positive definite because $x^TAx\gt0$; believing that every Gershgorin disc contains an eigenvalue; forgetting conjugates in $\langle x,y\rangle$ on $\mathbb{C}^n$.
Suggested homework. I(1, 2, 4, 8), II(2, 3, 5, 6, 8), III(2, 4, 5, 6), IV(1, 3, 6); Exam corner E2, E4, E5.
Research question. How few numbers does it take to store an image, a data table or a climate record "almost exactly" — and why is the truncated singular value decomposition the best possible answer?
Why it matters. A grey-scale image, a table of patients × gene expressions, or a grid of sea-surface temperatures × months is a matrix $A$. Truncating its SVD to $A_k=\sum_{i\le k}\sigma_iu_iv_i^*$ is the idea behind principal component analysis in statistics, "empirical orthogonal functions" in climate science, latent-factor models in recommender systems and noise reduction in medical imaging. The Eckart–Young theorem (Theorem 10.30(5)) says no rank-$k$ matrix does better.
What to do.
- By hand: find the SVD of $A=\begin{pmatrix}2&0&1\\0&2&1\end{pmatrix}$ (start from $AA^T$), the best rank-one approximation $A_1$, and $\|A-A_1\|$ and $\|A-A_1\|_F$. Check that they equal $\sigma_2$ and $\sqrt{\sum_{i\gt1}\sigma_i^2}$.
- In Python (
numpy.linalg.svd), load any grey-scale image as a matrix, plot $\log\sigma_i$ against $i$, and display $A_k$ for $k=5,20,50$. Record the relative error $\|A-A_k\|_F/\|A\|_F$ and the storage $k(m+n+1)$ compared with $mn$. - Test Eckart–Young numerically: generate 1000 random rank-$k$ matrices $B$ (as products of random $m\times k$ and $k\times n$ matrices) and check that $\|A-B\|$ never beats $\sigma_{k+1}$.
- Add Gaussian noise to the image and compare $A_k$ of the noisy image with the original. For which $k$ is the error smallest?
Tools: Python with NumPy and Matplotlib (or any language with an SVD routine).
Hand in: A two-page report with your hand computation, the singular-value plot, the reconstructions, a table of errors and storage, and one paragraph explaining why the singular values of a "natural" image decay quickly.
Going further. Read about randomized SVD algorithms for huge matrices, or about the nuclear-norm relaxation used in matrix completion (filling in the missing entries of a ratings table).
🧪 Self-check quiz
Q1. Every eigenvalue of a Hermitian matrix is
Q2. Which matrix is not normal?
Q3. A real square matrix $A$ can be written $QDQ^T$ with $Q$ real orthogonal and $D$ real diagonal if and only if
Q4. The positive semidefinite square root of $\begin{pmatrix}5&4\\4&5\end{pmatrix}$ is
Q5. For $A=\begin{pmatrix}3&0\\4&5\end{pmatrix}$, the operator norm $\|A\|=\max_{\|x\|=1}\|Ax\|$ is