8.4 Compact Self-Adjoint Operators and the Spectral Theorem
Compact operators behave like matrices: they are limits of finite-rank operators, their nonzero spectrum consists of eigenvalues of finite multiplicity accumulating only at $0$, and — when they are self-adjoint — they are diagonal in an orthonormal basis of eigenvectors. Applications: integral equations and Sturm–Liouville problems.
Every real symmetric matrix can be diagonalised by an orthogonal matrix: there is an orthonormal basis of eigenvectors, the eigenvalues are real, and $\|A\|$ is the largest $|\lambda|$ (§3.3). Section 8.3 gave us orthonormal bases in infinite dimensions; the natural question is which operators on a Hilbert space can be diagonalised in the same way. Not all bounded self-adjoint operators can: multiplication by $t$ on $L^2[0,1]$ is self-adjoint but has no eigenvectors at all. What rescues the finite-dimensional picture is compactness — an operator that maps bounded sets to sets with compact closure. Integral operators with square-integrable kernels are compact, and for them Hilbert and Schmidt proved, in the first decade of the twentieth century, that a symmetric kernel is diagonalised by its eigenfunctions: $$Tx=\sum_n\lambda_n\langle x,e_n\rangle e_n,\qquad\lambda_n\to0.$$
This spectral theorem for compact self-adjoint operators is the engine behind the classical theory of integral equations (the Fredholm alternative), the completeness of the eigenfunctions of Sturm–Liouville problems (and therefore of Fourier sine series, Legendre and Bessel expansions), principal component analysis of random curves, and the normal modes of vibrating systems. GATE tests it through computations — the eigenvalues and norm of an integral operator with a "degenerate" kernel such as $x+t$, the spectrum of a diagonal or multiplication operator, the solvability of $u-\lambda Tu=f$ — and through MSQs on which operators are compact and what the spectrum of a compact operator can look like.
Compact Operators
- Every compact operator is bounded. Every bounded finite-rank operator is compact.
- $K(X,Y)$ is a subspace of $B(X,Y)$, and if $Y$ is a Banach space it is closed: a norm limit of compact operators is compact.
- Ideal property: if $T$ is compact and $S$ bounded, then $ST$ and $TS$ (when defined) are compact.
- Riesz's lemma: if $M$ is a closed proper subspace of $X$ and $0\lt\theta\lt1$, there is a unit vector $u$ with $\operatorname{dist}(u,M)\ge\theta$.
- The identity of $X$ is compact if and only if $\dim X\lt\infty$. Consequently a compact operator on an infinite-dimensional space has no bounded inverse.
Proof. (1) If $T$ were unbounded we could pick $\|x_n\|\le1$ with $\|Tx_n\|\ge n$; then $(Tx_n)$ has no convergent (indeed no bounded) subsequence. If $T$ is bounded with finite-dimensional range, $T(B)$ is a bounded subset of a finite-dimensional normed space, whose closure is compact (Heine–Borel, all norms being equivalent there — §8.1).
(2) Sums and scalar multiples: extract subsequences one after the other. For closedness let $\|T_n-T\|\to0$ with $T_n$ compact and let $(x_k)$ satisfy $\|x_k\|\le1$. Choose a subsequence on which $T_1x_k$ converges, a further subsequence on which $T_2x_k$ converges, and so on; the diagonal sequence $(z_k)$ makes every $(T_nz_k)_k$ convergent. Then $$\begin{aligned}\|Tz_k-Tz_j\|&\le\|(T-T_n)z_k\|+\|T_nz_k-T_nz_j\|\\&\quad+\|(T_n-T)z_j\|\\&\le2\|T-T_n\|+\|T_nz_k-T_nz_j\|.\end{aligned}$$ Given $\varepsilon$, fix $n$ with $2\|T-T_n\|\lt\varepsilon/2$, then $k,j$ large: $(Tz_k)$ is Cauchy and converges because $Y$ is complete.
(3) If $(x_n)$ is bounded, $(Tx_n)$ has a convergent subsequence and $S$ preserves convergence; and $(Sx_n)$ is bounded, so $(TSx_n)$ has a convergent subsequence.
(4) Pick $x\notin M$; $d=\operatorname{dist}(x,M)\gt0$ since $M$ is closed. Choose $m_0\in M$ with $\|x-m_0\|\le d/\theta$ and put $u=(x-m_0)/\|x-m_0\|$. For $m\in M$, $m_0+\|x-m_0\|m\in M$, so $$\|u-m\|=\frac{\big\|x-(m_0+\|x-m_0\|m)\big\|}{\|x-m_0\|}\ge\frac{d}{d/\theta}=\theta.$$
(5) If $\dim X\lt\infty$ the identity has finite rank. If $\dim X=\infty$, choose a unit vector $u_1$ and inductively, by (4) applied to the finite-dimensional (hence closed, §8.1) proper subspace $\operatorname{span}\{u_1,\dots,u_{n-1}\}$, a unit vector $u_n$ at distance $\ge\frac12$ from it. Then $\|u_n-u_m\|\ge\frac12$ for $m\ne n$: the unit ball is not compact. If $T$ were compact with $T^{-1}$ bounded, $I=T^{-1}T$ would be compact by (3). ∎
- If $T$ is compact, then $\|Te_n\|\to0$ for every orthonormal sequence $(e_n)$ in $H$.
- $T$ is compact if and only if it is a norm limit of finite-rank operators.
- $T$ is compact if and only if $T^*$ is compact.
- Every Hilbert–Schmidt operator is compact, and $\|T\|\le\|T\|_{HS}$. In particular an integral operator $(T_Ku)(x)=\int_a^bK(x,t)u(t)\,dt$ with $K\in L^2([a,b]^2)$ is compact on $L^2[a,b]$, with $$\begin{aligned}\|T_K\|&\le\|T_K\|_{HS}\\&=\Big(\int_a^b\!\!\int_a^b|K(x,t)|^2\,dt\,dx\Big)^{1/2}.\end{aligned}$$
- A diagonal operator $D_\lambda x=(\lambda_nx_n)$ on $\ell^2$ ($(\lambda_n)$ bounded) is compact if and only if $\lambda_n\to0$.
Proof. (1) If not, there are $\varepsilon\gt0$ and a subsequence with $\|Te_{n_k}\|\ge\varepsilon$; by compactness a further subsequence has $Te_{n_k}\to y$, and $\|y\|\ge\varepsilon$. But $\langle Te_{n_k},y\rangle=\langle e_{n_k},T^*y\rangle\to0$ by Bessel's inequality (§8.3), while it also tends to $\langle y,y\rangle\ne0$ — a contradiction.
(2) "If" is Theorem 8.29(1), (2). Conversely let $T$ be compact. The set $\overline{T(B)}$ is compact, hence separable, so $\overline{\operatorname{ran}T}$ is a separable Hilbert space with a countable orthonormal basis $(f_k)$ (§8.3). Let $P_n$ be the orthogonal projection onto $\operatorname{span}\{f_1,\dots,f_n\}$; $P_nT$ has finite rank. If $\|T-P_nT\|\not\to0$, there are $\varepsilon\gt0$ and unit vectors $x_n$ (along a subsequence) with $\|(I-P_n)Tx_n\|\ge\varepsilon$; passing to a further subsequence, $Tx_n\to y\in\overline{\operatorname{ran}T}$. Then $$\begin{aligned}\|(I-P_n)Tx_n\|&\le\|(I-P_n)(Tx_n-y)\|\\&\quad+\|(I-P_n)y\|\\&\le\|Tx_n-y\|+\|(I-P_n)y\|\to0,\end{aligned}$$ the last term by Parseval in $\overline{\operatorname{ran}T}$ — a contradiction.
(3) If $T_n=\sum_i\langle\cdot,a_i\rangle b_i$ has finite rank, so does $T_n^*=\sum_i\langle\cdot,b_i\rangle a_i$, and $\|T_n^*-T^*\|=\|T_n-T\|$ (§8.3). Apply (2) to $T$ and to $T^{**}=T$.
(4) Let $P_N$ project onto $\operatorname{span}\{e_1,\dots,e_N\}$. For $x=\sum c_ne_n$, by Cauchy–Schwarz in $\ell^2$, $$\begin{aligned}\|(T-TP_N)x\|&=\Big\|\sum_{n\gt N}c_nTe_n\Big\|\le\sum_{n\gt N}|c_n|\,\|Te_n\|\\&\le\|x\|\Big(\sum_{n\gt N}\|Te_n\|^2\Big)^{1/2}.\end{aligned}$$ So the finite-rank operators $TP_N$ converge to $T$ in norm, and $T$ is compact by (2); with $N=0$ the same estimate gives $\|T\|\le\|T\|_{HS}$. For the integral operator: for almost every $x$ the function $K_x=K(x,\cdot)$ is in $L^2$ (Fubini), and $(T_Ke_n)(x)=\langle K_x,\bar e_n\rangle$. Since $(\bar e_n)$ is again an orthonormal basis, Parseval gives $\sum_n|(T_Ke_n)(x)|^2=\|K_x\|^2$, and integrating in $x$ (monotone convergence) $\sum_n\|T_Ke_n\|^2=\int\|K_x\|^2dx=\|K\|_{L^2}^2$.
(5) If $\lambda_n\to0$, the truncations $D^{(N)}=D_{(\lambda_1,\dots,\lambda_N,0,\dots)}$ have finite rank and $\|D_\lambda-D^{(N)}\|=\sup_{n\gt N}|\lambda_n|\to0$. If $\lambda_n\not\to0$, then $\|D_\lambda e_n\|=|\lambda_n|\not\to0$, contradicting (1). ∎
Spectrum, Eigenvalues and Self-Adjoint Operators
In finite dimensions $\lambda$ is in the spectrum exactly when $A-\lambda I$ is not injective. In infinite dimensions an injective operator need not be onto, and the spectrum is defined through invertibility. Spectra are taken in complex Banach spaces; for self-adjoint operators everything below also works word for word over $\mathbb{R}$.
- Neumann series. If $\|A\|\lt1$, then $I-A$ is invertible, $(I-A)^{-1}=\sum_{n\ge0}A^n$ and $\|(I-A)^{-1}\|\le\frac1{1-\|A\|}$.
- $\sigma(T)$ is a closed subset of the disc $\{|\lambda|\le\|T\|\}$, hence compact.
- $\sigma(T)\ne\varnothing$, and $r(T)=\lim_n\|T^n\|^{1/n}\le\|T\|$ (Gelfand's spectral radius formula).
- If $X$ is a Hilbert space, $\sigma(T^*)=\{\bar\lambda:\lambda\in\sigma(T)\}$.
Proof. (1) $\sum\|A^n\|\le\sum\|A\|^n\lt\infty$ and $B(X)$ is complete (§8.1), so $S=\sum A^n$ converges, with $\|S\|\le\frac1{1-\|A\|}$. The telescoping identities $(I-A)\sum_{n\le N}A^n=I-A^{N+1}=\sum_{n\le N}A^n(I-A)$ and $A^{N+1}\to0$ give $(I-A)S=S(I-A)=I$.
(2) If $|\lambda|\gt\|T\|$, then $T-\lambda I=-\lambda(I-T/\lambda)$ is invertible by (1). If $\lambda_0\in\rho(T)$ and $R=(T-\lambda_0)^{-1}$, then $$T-\lambda=(T-\lambda_0)\big(I-(\lambda-\lambda_0)R\big)$$ is invertible for $|\lambda-\lambda_0|\lt1/\|R\|$; so $\rho(T)$ is open and $\sigma(T)$ closed.
(3) (Sketch.) For $f\in B(X)^*$ the function $\lambda\mapsto f\big((T-\lambda)^{-1}\big)$ is holomorphic on $\rho(T)$ (expand $(T-\lambda)^{-1}$ in a Neumann series around each $\lambda_0$, as in (2)) and tends to $0$ as $|\lambda|\to\infty$, because $\|(T-\lambda)^{-1}\|\le\frac1{|\lambda|-\|T\|}$. If $\sigma(T)$ were empty it would be a bounded entire function, hence $0$ by Liouville's theorem (§5.1), for every $f$; by Hahn–Banach (§8.1) $(T-\lambda)^{-1}=0$, which is absurd. The formula for $r(T)$ comes from the Laurent expansion $(T-\lambda)^{-1}=-\sum_n T^n/\lambda^{n+1}$, whose radius of convergence is governed by $\limsup\|T^n\|^{1/n}$; we use only the statement. (4) $(T-\lambda)^*=T^*-\bar\lambda$, and an operator is invertible iff its adjoint is ($(S^{-1})^*=(S^*)^{-1}$). ∎
- Every eigenvalue of $T$ is real, and eigenvectors for different eigenvalues are orthogonal.
- $\|T\|=\sup_{\|x\|=1}|\langle Tx,x\rangle|=\max(|m|,|M|)$.
- $\sigma(T)\subseteq[m,M]$, and $m,M\in\sigma(T)$ (as approximate eigenvalues). Hence $\sigma(T)$ is real and $r(T)=\|T\|$.
- If a subspace $N$ is invariant under $T$ ($TN\subseteq N$), so is $N^\perp$.
Proof. (1) If $Tx=\lambda x$ with $\|x\|=1$, then $\lambda=\langle Tx,x\rangle=\langle x,Tx\rangle=\bar\lambda$. If also $Ty=\mu y$, $\mu\ne\lambda$, then $\lambda\langle x,y\rangle=\langle Tx,y\rangle=\langle x,Ty\rangle=\mu\langle x,y\rangle$, so $\langle x,y\rangle=0$.
(2) Let $\alpha=\sup_{\|x\|=1}|\langle Tx,x\rangle|$; clearly $\alpha\le\|T\|$. For unit vectors $x,y$, self-adjointness gives $\langle Ty,x\rangle=\overline{\langle Tx,y\rangle}$, so $$\begin{aligned}4\operatorname{Re}\langle Tx,y\rangle&=\langle T(x+y),x+y\rangle-\langle T(x-y),x-y\rangle\\&\le\alpha\big(\|x+y\|^2+\|x-y\|^2\big)=4\alpha\end{aligned}$$ by the parallelogram law. If $Tx\ne0$, take $y=Tx/\|Tx\|$: $\|Tx\|\le\alpha$. Hence $\|T\|\le\alpha$.
(3) Let $\lambda=a+ib$. For $b\ne0$: $|\operatorname{Im}\langle(T-\lambda)x,x\rangle|=|b|\,\|x\|^2$, so $\|(T-\lambda)x\|\ge|b|\,\|x\|$. For real $\lambda\gt M$: $\langle(\lambda-T)x,x\rangle\ge(\lambda-M)\|x\|^2$, so $\|(T-\lambda)x\|\ge(\lambda-M)\|x\|$; similarly for $\lambda\lt m$. In each case $T-\lambda$ is bounded below, hence injective with closed range (§8.2), and its range is dense because $\big(\operatorname{ran}(T-\lambda)\big)^\perp=\ker(T-\bar\lambda)=\{0\}$ (§8.3; $\bar\lambda$ satisfies the same bound). So $\lambda\in\rho(T)$. To see $M\in\sigma(T)$, put $A=M-T\ge0$. The form $[x,y]=\langle Ax,y\rangle$ is positive semidefinite, and the proof of Cauchy–Schwarz (§8.3) still gives $|\langle Ax,y\rangle|^2\le\langle Ax,x\rangle\langle Ay,y\rangle$. With $y=Ax$: $$\|Ax\|^4\le\langle Ax,x\rangle\langle A^2x,Ax\rangle\le\langle Ax,x\rangle\|A\|\,\|Ax\|^2,$$ so $\|Ax\|^2\le\|A\|\langle Ax,x\rangle$. Choosing unit $x_n$ with $\langle Tx_n,x_n\rangle\to M$ gives $\langle Ax_n,x_n\rangle\to0$, hence $\|(T-M)x_n\|\to0$: $M$ is an approximate eigenvalue. The same argument with $T-m\ge0$ handles $m$. Finally $r(T)=\max(|m|,|M|)=\|T\|$ by (2).
(4) If $x\in N^\perp$ and $n\in N$, then $\langle Tx,n\rangle=\langle x,Tn\rangle=0$. ∎
- For $\lambda\ne0$, the eigenspace $\ker(T-\lambda I)$ is finite-dimensional.
- For $\lambda\ne0$, $T-\lambda I$ is injective if and only if it is surjective (Fredholm alternative). Hence every nonzero point of $\sigma(T)$ is an eigenvalue.
- For every $\varepsilon\gt0$ only finitely many eigenvalues satisfy $|\lambda|\ge\varepsilon$. So $\sigma(T)$ is countable, and $0$ is its only possible accumulation point.
- If $\dim X=\infty$, then $0\in\sigma(T)$.
Proof. (1) On $N=\ker(T-\lambda)$, $T=\lambda I$; so the closed unit ball of $N$ is $T(\lambda^{-1}B_N)$, which has compact closure. By Theorem 8.29(5), $\dim N\lt\infty$. (4) is Theorem 8.29(5).
(3) Suppose there were infinitely many distinct eigenvalues $\lambda_1,\lambda_2,\dots$ with $|\lambda_n|\ge\varepsilon$ and eigenvectors $x_n$. Eigenvectors of distinct eigenvalues are linearly independent, so $M_n=\operatorname{span}\{x_1,\dots,x_n\}$ increase strictly. By Riesz's lemma choose unit $y_n\in M_n$ with $\operatorname{dist}(y_n,M_{n-1})\ge\frac12$. Now $(T-\lambda_n)M_n\subseteq M_{n-1}$ (it kills the $x_n$-component) and $TM_m\subseteq M_m$. For $m\lt n$, $$Ty_n-Ty_m=\lambda_ny_n-\big[(\lambda_n-T)y_n+Ty_m\big]$$ with the bracket in $M_{n-1}$, so $\|Ty_n-Ty_m\|\ge|\lambda_n|\cdot\frac12\ge\frac\varepsilon2$. Then $(Ty_n)$ has no convergent subsequence, contradicting compactness.
(2) (Sketch.) One shows first that $\operatorname{ran}(T-\lambda)$ is closed (otherwise there are $x_n$ with $\operatorname{dist}(x_n,\ker(T-\lambda))=1$, bounded, and $(T-\lambda)x_n\to0$; compactness gives $Tx_{n_k}\to y$, then $x_{n_k}\to y/\lambda$, and $y/\lambda$ lies in the kernel, a contradiction). If $T-\lambda$ is injective but not onto, the closed subspaces $\operatorname{ran}(T-\lambda)^k$ decrease strictly, and Riesz's lemma produces a sequence as in (3) contradicting compactness; the converse direction uses the adjoint. For compact self-adjoint operators on a Hilbert space, (2) also follows directly from the spectral theorem (Theorem 8.36(2)). ∎
The Spectral Theorem for Compact Self-Adjoint Operators
The finite-dimensional proof of the spectral theorem finds one eigenvector by maximising $\langle Ax,x\rangle$ on the unit sphere and then restricts $A$ to the orthogonal complement. In infinite dimensions the maximum need not be attained ($M_t$ has none); compactness is exactly what makes it attained.
Proof. By Theorem 8.32(2) there are unit vectors $x_n$ with $|\langle Tx_n,x_n\rangle|\to\|T\|$; passing to a subsequence, $\langle Tx_n,x_n\rangle\to\lambda$ with $\lambda=\|T\|$ or $\lambda=-\|T\|$. Since $\lambda$ and $\langle Tx_n,x_n\rangle$ are real, $$\begin{aligned}\|Tx_n-\lambda x_n\|^2&=\|Tx_n\|^2-2\lambda\langle Tx_n,x_n\rangle+\lambda^2\\&\le2\lambda^2-2\lambda\langle Tx_n,x_n\rangle\to0.\end{aligned}$$ By compactness a subsequence has $Tx_{n_k}\to y$. Then $\lambda x_{n_k}=Tx_{n_k}-(Tx_{n_k}-\lambda x_{n_k})\to y$, so $\|y\|=|\lambda|\ne0$, and by continuity $Ty=\lim T(\lambda x_{n_k})=\lambda\lim Tx_{n_k}=\lambda y$. ∎
- $Tx=\sum_n\lambda_n\langle x,e_n\rangle e_n$ for every $x\in H$ (norm convergence);
- if the sequence is infinite, $\lambda_n\to0$; and $\big\|T-\sum_{n\le N}\lambda_n\langle\cdot,e_n\rangle e_n\big\|=|\lambda_{N+1}|$ (or $0$);
- $H=\ker T\oplus\overline{\operatorname{span}}\{e_n\}$; adjoining an orthonormal basis of $\ker T$ gives an orthonormal basis of $H$ consisting of eigenvectors of $T$;
- every nonzero eigenvalue of $T$ is one of the $\lambda_n$ and has finite multiplicity (the number of $n$ with $\lambda_n=\lambda$); $\|T\|=|\lambda_1|$; $\ \sigma(T)\setminus\{0\}=\{\lambda_n\}$, and $0\in\sigma(T)$ if $\dim H=\infty$.
Proof. Construction. Put $H_1=H$. Suppose $e_1,\dots,e_{n-1}$ are constructed and let $H_n=\{e_1,\dots,e_{n-1}\}^\perp$. Each $e_k$ spans a $T$-invariant line, so $H_n$ is $T$-invariant (Theorem 8.32(4)), and $T_n=T|_{H_n}$ is compact and self-adjoint on the Hilbert space $H_n$. If $T_n=0$, stop. Otherwise Theorem 8.34 gives a unit $e_n\in H_n$ with $Te_n=\lambda_ne_n$, $|\lambda_n|=\|T_n\|$. Since $H_{n+1}\subseteq H_n$, $|\lambda_{n+1}|=\|T_{n+1}\|\le\|T_n\|=|\lambda_n|$.
$\lambda_n\to0$. If $|\lambda_n|\ge\varepsilon\gt0$ for infinitely many (hence, by monotonicity, all) $n$, then $\|Te_n-Te_m\|^2=\lambda_n^2+\lambda_m^2\ge2\varepsilon^2$, and $(Te_n)$ has no convergent subsequence — impossible for compact $T$.
Expansion. For $x\in H$ let $y_N=x-\sum_{n\le N}\langle x,e_n\rangle e_n$. Then $y_N\in H_{N+1}$ and $\|y_N\|\le\|x\|$ (Bessel's inequality, §8.3), so $$\begin{aligned}\Big\|Tx-\sum_{n\le N}\lambda_n\langle x,e_n\rangle e_n\Big\|&=\|Ty_N\|\\&\le|\lambda_{N+1}|\,\|x\|\to0.\end{aligned}$$ This proves (1) and the upper bound in (2); equality holds at $x=e_{N+1}$. If the process stopped at step $N+1$, $Ty_N=0$ and the sum is finite.
Decomposition. If $x\perp e_n$ for all $n$, then $Tx=0$ by (1). Conversely, if $Tx=0$, then $\lambda_n\langle x,e_n\rangle=\langle x,Te_n\rangle=\langle Tx,e_n\rangle=0$, so $x\perp e_n$. Thus $\ker T=\{e_n\}^\perp$ and (3) follows from the projection theorem (§8.3).
Eigenvalues. If $Tx=\mu x$ with $\mu\ne0$, $x\ne0$, then $\mu\langle x,e_n\rangle=\langle Tx,e_n\rangle=\lambda_n\langle x,e_n\rangle$, so $\langle x,e_n\rangle=0$ unless $\lambda_n=\mu$; as $x=\mu^{-1}Tx\in\overline{\operatorname{span}}\{e_n\}$, $x$ is a combination of the finitely many $e_n$ with $\lambda_n=\mu$ (finitely many because $\lambda_n\to0$). So $\mu$ is some $\lambda_n$ and its eigenspace is $\operatorname{span}\{e_n:\lambda_n=\mu\}$. $\|T\|=\|T_1\|=|\lambda_1|$. A nonzero $\mu$ outside $\{\lambda_n\}$ is in $\rho(T)$ by Theorem 8.36(2) below, and $0\in\sigma(T)$ in infinite dimensions by Theorem 8.29(5). ∎
In words: a compact self-adjoint operator is a diagonal operator $D_\lambda$ with $\lambda_n\to0$, written in a suitable orthonormal basis. Interactive 1 runs the proof as an algorithm.
- Rayleigh–Ritz. If $\lambda_1^+\ge\lambda_2^+\ge\cdots\gt0$ are the positive eigenvalues (with multiplicity) and $e_1^+,e_2^+,\dots$ their eigenvectors, then $$\lambda_k^+=\max_{\substack{\|x\|=1\\ x\perp e_1^+,\dots,e_{k-1}^+}}\langle Tx,x\rangle.$$ In particular $\langle Tx,x\rangle\le\lambda_1^+\|x\|^2$, and any trial vector gives a lower bound for $\lambda_1^+$.
- Fredholm alternative. Let $\mu\ne0$. If $\mu$ is not an eigenvalue, then $(T-\mu)u=f$ has, for every $f$, exactly one solution $$u=-\frac f\mu+\sum_n\frac{\lambda_n\langle f,e_n\rangle}{\mu(\lambda_n-\mu)}e_n,$$ and $\|(T-\mu)^{-1}\|=1/\operatorname{dist}(\mu,\sigma(T))$. If $\mu$ is an eigenvalue, a solution exists if and only if $f\perp\ker(T-\mu)$, and then the solutions form a translate of $\ker(T-\mu)$.
- Hilbert–Schmidt. $T$ is Hilbert–Schmidt iff $\sum_n\lambda_n^2\lt\infty$, and $\|T\|_{HS}^2=\sum_n\lambda_n^2$ (with multiplicity). For an integral operator with Hermitian kernel $K\in L^2$: $\sum_n\lambda_n^2=\int\!\!\int|K|^2$.
- Mercer's theorem (statement). If $K$ is continuous on $[a,b]^2$, Hermitian, and $T_K\ge0$, then $K(x,t)=\sum_n\lambda_ne_n(x)\overline{e_n(t)}$ absolutely and uniformly, and $\sum_n\lambda_n=\int_a^bK(x,x)\,dx$.
Proof. (1) Write $x=x_0+\sum_nc_ne_n$ with $x_0\in\ker T$. Then $\langle Tx,x\rangle=\sum_n\lambda_n|c_n|^2$. If $x\perp e_1^+,\dots,e_{k-1}^+$, every term with $\lambda_n\gt0$ has $\lambda_n\le\lambda_k^+$ and the terms with $\lambda_n\lt0$ are $\le0$, so $\langle Tx,x\rangle\le\lambda_k^+\sum|c_n|^2\le\lambda_k^+\|x\|^2$; equality at $x=e_k^+$.
(2) Expand $f=P_0f+\sum\langle f,e_n\rangle e_n$ and look for $u=u_0+\sum c_ne_n$. Then $(T-\mu)u=-\mu u_0+\sum(\lambda_n-\mu)c_ne_n$, so the equation splits into $-\mu u_0=P_0f$ and $(\lambda_n-\mu)c_n=\langle f,e_n\rangle$. If $\mu\ne\lambda_n$ for all $n$, then $|\lambda_n-\mu|\ge\delta\gt0$ (because $\lambda_n\to0\ne\mu$), $c_n=\langle f,e_n\rangle/(\lambda_n-\mu)$ is square-summable, and $$u=-\frac{P_0f}\mu+\sum_n\frac{\langle f,e_n\rangle}{\lambda_n-\mu}e_n;$$ substituting $P_0f=f-\sum\langle f,e_n\rangle e_n$ and $\frac1{\lambda_n-\mu}+\frac1\mu=\frac{\lambda_n}{\mu(\lambda_n-\mu)}$ gives the formula. The inverse is "diagonal" with entries $-1/\mu$ and $1/(\lambda_n-\mu)$, whose supremum is $1/\operatorname{dist}(\mu,\sigma(T))$. If $\mu=\lambda_k$ for some $k$, the equations with $\lambda_n=\mu$ read $0=\langle f,e_n\rangle$ — the condition $f\perp\ker(T-\mu)$ — and leave those $c_n$ free.
(3) Compute $\|T\|_{HS}$ in the orthonormal basis of eigenvectors of Theorem 8.35(3): $\sum\|Te_n\|^2=\sum\lambda_n^2$; for integral operators combine with Theorem 8.30(4). (4) is quoted without proof. ∎
- Degenerate kernel $K(x,t)=\sum_{i=1}^ra_i(x)b_i(t)$: the range lies in $\operatorname{span}\{a_i\}$. Put $u=\sum c_ia_i$; then $Tu=\lambda u$ becomes the $r\times r$ eigenproblem $Ac=\lambda c$ with $A_{ij}=\int b_ia_j$. All other "eigenvalues" are $0$, with infinite-dimensional kernel.
- Self-adjoint? Real symmetric (or Hermitian) kernel ⇒ real eigenvalues, orthogonal eigenfunctions, $\|T\|=\max|\lambda|$. Without symmetry, $\|T\|$ can exceed every $|\lambda|$ (the Volterra operator).
- Green's-function kernels ($\min(x,t)$, $\min(x,t)-xt$, …): differentiate $v=Tu$ twice to get a boundary value problem; the eigenvalues of $T$ are the reciprocals of the Sturm–Liouville eigenvalues.
- Checks: $\sum\lambda_n=\int K(x,x)\,dx$ (finite rank, or Mercer for positive continuous kernels) and $\sum\lambda_n^2=\int\!\!\int|K|^2$. A trial function $u$ gives $\lambda_1^+\ge\langle Tu,u\rangle/\|u\|^2$.
- Solvability of $u-\lambda Tu=f$: unique iff $1/\lambda$ is not an eigenvalue of $T$; at a characteristic value, solvable iff $f$ is orthogonal to the eigenspace.
Applications: Integral Equations and Sturm–Liouville Problems
The most important application turns a differential operator, which is unbounded, into a compact self-adjoint integral operator by inverting it.
- there is a continuous, real, symmetric Green's function $G$ such that for $f\in C[a,b]$ the unique solution of $Lu=f$ with the boundary conditions is $u(x)=\int_a^bG(x,t)f(t)\,dt$; explicitly $G(x,t)=u_a(\min(x,t))\,u_b(\max(x,t))/C$, where $u_a$, $u_b$ solve $Lu=0$ with the condition at $a$, resp. $b$, and $C=p\,(u_a'u_b-u_au_b')$ is constant;
- $T_G$ is compact, self-adjoint and injective on $L^2[a,b]$;
- the eigenvalues of the Sturm–Liouville problem $Lu=\lambda u$ are real and simple, form a sequence $\lambda_1\lt\lambda_2\lt\cdots$ with $|\lambda_n|\to\infty$, and the normalised eigenfunctions form an orthonormal basis of $L^2[a,b]$.
Proof (sketch). (1) If $u_a,u_b$ were dependent, $u_a$ would satisfy both conditions and $0$ would be an eigenvalue; so they are independent and $C\ne0$; $C$ is constant because $(p(u_a'u_b-u_au_b'))'$ $=u_b(pu_a')'-u_a(pu_b')'$ $=u_bqu_a-u_aqu_b=0$. Direct differentiation of $u(x)=\frac1C\big(u_b(x)\int_a^xu_af+u_a(x)\int_x^bu_bf\big)$ — exactly as in Example 4 — gives $Lu=f$ and the boundary conditions; uniqueness holds because $0$ is not an eigenvalue. $G$ is visibly symmetric. (2) A continuous symmetric kernel gives a compact (Hilbert–Schmidt) self-adjoint operator. If $T_Gf=0$ for continuous $f$, then $f=L(T_Gf)=0$; for $f\in L^2$ the same identity holds a.e. (with $pu'$ absolutely continuous), so $T_G$ is injective. (3) For $\lambda\ne0$: $Lu=\lambda u$ with the boundary conditions iff $u=T_G(\lambda u)$ iff $T_Gu=\frac1\lambda u$. So the eigenvalues of $L$ are the reciprocals $1/\mu_n$ of the nonzero eigenvalues of $T_G$ (real), and since $\ker T_G=\{0\}$, Theorem 8.35(3) makes the eigenfunctions an orthonormal basis; $\mu_n\to0$ gives $|\lambda_n|\to\infty$. Simplicity: two eigenfunctions for the same $\lambda$ both satisfy the condition at $a$, so their Wronskian vanishes at $a$ and they are dependent. That the $\lambda_n$ are bounded below (so $\lambda_n\to+\infty$) follows from $\langle Lu,u\rangle=\int(p|u'|^2+q|u|^2)+$ boundary terms; we omit it. ∎
- Bounded is not compact. The identity and every isometry of an infinite-dimensional space, the shifts, $M_t$ on $L^2[0,1]$, and projections onto infinite-dimensional subspaces are bounded but not compact.
- Strong limits of finite-rank operators need not be compact: the truncations $P_N\to I$ pointwise on $\ell^2$, but $\|P_N-I\|=1$. Only norm limits preserve compactness.
- $0$ is always in the spectrum of a compact operator in infinite dimensions, but it need not be an eigenvalue (diagonal $(\frac1n)$), and when it is, its eigenspace may be infinite-dimensional (kernel $x+t$). Nonzero eigenvalues have finite multiplicity.
- A compact operator may have no eigenvalues (Volterra, $\sigma=\{0\}$); a self-adjoint operator may have no eigenvalues (multiplication by $t$, $\sigma=[0,1]$). The spectral theorem needs both compactness and self-adjointness (or normality).
- $\|T\|=r(T)$ only for self-adjoint (normal) operators. $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has $r=0$, $\|\cdot\|=1$; $V$ has $r=0$, $\|V\|=\frac2\pi$. For a non-symmetric kernel, do not compute $\|T\|$ as $\max|\lambda|$.
- Real spectrum does not imply self-adjoint (nilpotent matrices, $V$).
- The spectrum is more than the eigenvalues in infinite dimensions: the right shift has $\sigma(S)$ = closed unit disc but no eigenvalues; truncating it to $N\times N$ matrices gives nilpotent matrices with spectrum $\{0\}$ — finite sections can mislead badly for non-self-adjoint operators.
- Eigenfunction expansions converge in $L^2$, not necessarily pointwise; uniform convergence of $\sum\lambda_ne_n(x)\overline{e_n(t)}$ needs Mercer's hypotheses (continuous kernel, $T\ge0$).
- Characteristic values versus eigenvalues: $u=f+\lambda Tu$ fails to be uniquely solvable when $\lambda=1/\mu$ for an eigenvalue $\mu\ne0$ of $T$ — not when $\lambda$ itself is an eigenvalue. A classic exam slip.
Exercises
- 1 Which diagonal operators $D_\lambda$ on $\ell^2$ are compact? (a) $\lambda_n=n^{-1/2}$; (b) $\lambda_n=(-1)^n$; (c) $\lambda_n=2^{-n}$; (d) $\lambda_n=1$ if $n$ is prime and $0$ otherwise.
- 2 Prove that the weighted shift $Tx=(0,x_1,\tfrac{x_2}2,\tfrac{x_3}3,\dots)$ is compact on $\ell^2$.
- 3 Let $\varphi\in C[0,1]$. Prove that $M_\varphi$ is compact on $L^2[0,1]$ only if $\varphi\equiv0$.
- 4 Prove the ideal property: if $T$ is compact and $S$ bounded, $ST$ and $TS$ are compact. Deduce that no compact operator on $\ell^2$ is invertible in $B(\ell^2)$.
- 5 Let $T\in B(H)$. Prove that if $T^*T$ is compact then $T$ is compact.
- 6 Compute $\|V\|_{HS}$ for the Volterra operator and deduce an upper bound for $\|V\|$. Compare with $\|V\|=\frac2\pi$.
- 7 Give a sequence of finite-rank operators converging pointwise (but not in norm) to a non-compact operator.
- 8 Let $K\in C([0,1]^2)$. Prove that $(Tf)(x)=\int_0^1K(x,t)f(t)\,dt$ is compact on $(C[0,1],\|\cdot\|_\infty)$.
- 1 For $A=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ and $B=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ on $\mathbb{C}^2$ (Euclidean norm), find the spectrum, the norm and the spectral radius.
- 2 Find $\sigma(D)$ and the eigenvalues of the diagonal operator $D$ on $\ell^2$ with $\lambda_n=1+\frac{(-1)^n}n$.
- 3 Prove that $\sigma(T^*)=\{\bar\lambda:\lambda\in\sigma(T)\}$ for $T\in B(H)$.
- 4 Find the spectrum and eigenvalues of $M_\varphi$ on $L^2[0,1]$ for (a) $\varphi(t)=t^2$; (b) $\varphi=\mathbf 1_{[0,1/2]}$.
- 5 Show that $V^n$ has kernel $\frac{(x-t)^{n-1}}{(n-1)!}$ for $t\le x$ (and $0$ otherwise), that $\|V^n\|\le\frac1{n!}$, and hence that $\sigma(V)=\{0\}$.
- 6 Prove that the spectrum of a unitary operator lies on the unit circle.
- 7 Let $T\ge0$ be self-adjoint. Prove that $\sigma(T)\subseteq[0,\|T\|]$ and $\|T\|\in\sigma(T)$.
- 8 On $\ell^2$ let $Tx=(x_2,x_1,x_4,x_3,\dots)$. Show $T$ is self-adjoint and unitary, find $\sigma(T)$ with multiplicities, and decide whether $T$ is compact.
- 1 On $L^2[-1,1]$ let $K(x,t)=1+3xt$. Find the eigenvalues with multiplicities and show $T=2P$ for an orthogonal projection $P$.
- 2 On $L^2[0,2\pi]$ let $K(x,t)=\cos(x-t)$. Find the nonzero eigenvalues, their multiplicities and $\|T\|$.
- 3 On $L^2[-1,1]$ let $K(x,t)=xt+x^2t^2$. Find the nonzero eigenvalues and $\|T\|$.
- 4 For $K=\min(x,t)$ on $[0,1]$ (Example 4), find the smallest $N$ such that the $N$-term truncation $T_N$ satisfies $\|T-T_N\|\lt0.01$.
- 5 Prove that $\langle Tu,u\rangle=\int_0^1\big(\int_s^1u\big)^2ds$ for $K=\min(x,t)$ by writing $\min(x,t)$ as an integral.
- 6 For $K=\min(x,t)-xt$ on $[0,1]$, compute the Rayleigh quotients of $u=1$ and $u=x(1-x)$ and compare with $\lambda_1=\frac1{\pi^2}$.
- 7 Let $T$ be compact self-adjoint with nonzero eigenvalues $(\lambda_n)$ (with multiplicity). Prove that $T$ is Hilbert–Schmidt iff $\sum\lambda_n^2\lt\infty$, and compute $\sum\lambda_n^2$ for $K=\min(x,t)$.
- 8 Prove that a compact self-adjoint $T$ with $T^k=0$ for some $k\ge1$ is $0$, and that $\|T^2\|=\|T\|^2$.
- 1 Solve $u(x)=1+\lambda\int_0^1xt\,u(t)\,dt$ for every real $\lambda$, and identify the characteristic value.
- 2 Find the characteristic values of $u(x)=f(x)+\lambda\int_{-1}^1(1+xt)u(t)\,dt$, and the condition on $f$ for solvability at the smaller one.
- 3 Find the eigenvalues and eigenfunctions of $-u''=\lambda u$, $u(0)=0$, $u'(1)=0$, and show that its Green's function is $\min(x,t)$.
- 4 Use Mercer's theorem for $\min(x,t)$ to prove $\sum_{n\ge1}\frac1{(2n-1)^2}=\frac{\pi^2}8$.
- 5 Use $\int\!\!\int G^2=\sum\mu_n^2$ for $G=\min(x,t)-xt$ to prove $\sum\frac1{n^4}=\frac{\pi^4}{90}$.
- 6 Show that $0$ is an eigenvalue of $-u''=\lambda u$, $u'(0)=u'(1)=0$, so no Green's function exists. Instead find the Green's function of $-u''+u$ with these conditions and use it to prove that $\{1\}\cup\{\sqrt2\cos n\pi x\}_{n\ge1}$ is an orthonormal basis of $L^2[0,1]$.
- 7 (Modelling) A taut string of unit length and unit tension, fixed at both ends, carries a load $f(x)$; its small deflection solves $-u''=f$, $u(0)=u(1)=0$. For the uniform load $f\equiv1$, expand the deflection in the eigenfunctions $\sqrt2\sin n\pi x$ and deduce $\sum_{k\ge0}\frac{(-1)^k}{(2k+1)^3}=\frac{\pi^3}{32}$.
E1. GATE MA · MCQ Which one of the following operators on $\ell^2$ is compact? (A) $(x_1,x_2,x_3,\dots)\mapsto(x_2,x_3,x_4,\dots)$ (B) $(x_1,x_2,x_3,\dots)\mapsto(x_1,\tfrac{x_2}2,\tfrac{x_3}3,\dots)$ (C) $(x_1,x_2,x_3,\dots)\mapsto(x_1,0,x_3,0,x_5,\dots)$ (D) $(x_1,x_2,x_3,\dots)\mapsto\big((1+\tfrac1n)x_n\big)_n$
E2. GATE MA · MCQ Let $T:L^2[-1,1]\to L^2[-1,1]$, $(Tu)(x)=\int_{-1}^{1}(1+xt)\,u(t)\,dt$. Then $\sigma(T)$ is (A) $\{2,\tfrac23\}$ (B) $\{0,2,\tfrac23\}$ (C) $\{0,2\}$ (D) $[0,2]$
E3. GATE MA · MSQ CSIR Part C Let $T\ne0$ be a compact self-adjoint operator on an infinite-dimensional separable Hilbert space $H$. Which of the following are necessarily true? (One or more options may be correct.) (A) $0\in\sigma(T)$. (B) $\|T\|$ or $-\|T\|$ is an eigenvalue of $T$. (C) Every eigenspace of $T$ is finite-dimensional. (D) $H$ has an orthonormal basis consisting of eigenvectors of $T$.
E4. GATE MA · MSQ Let $(Vu)(x)=\int_0^xu(t)\,dt$ on $L^2[0,1]$. Which of the following are true? (One or more options may be correct.) (A) $V$ is compact. (B) $V$ is self-adjoint. (C) $V$ has no eigenvalues. (D) $\|V\|$ equals the spectral radius of $V$.
E5. GATE MA · NAT Let $(Tu)(x)=\int_0^1(1+2xt)\,u(t)\,dt$ on $L^2[0,1]$. The value of $\|T\|$, correct to three decimal places, is ______.
E6. GATE MA · NAT Let $u\in C[0,1]$ satisfy $u(x)=x^2+2\int_0^1xt\,u(t)\,dt$ for all $x\in[0,1]$. The value of $u(1)$, correct to two decimal places, is ______.
Suggested plan — 3 lectures × 60 min.
- Compact operators. Definition 8.14, Theorem 8.29 (prove (2) and Riesz's lemma with (5)); Definition 8.15 and Theorem 8.30 (prove (1), (4), (5); sketch (2)). Example 1 as a class vote, then GATE corner E1.
- Spectra. Definition 8.16, Theorem 8.31 (Neumann series in full), Example 2 — draw the four spectra side by side. Definition 8.17 and Theorem 8.32 (the polarization proof of (2) is short and worth doing). State Theorem 8.33 and prove (3). Close with Example 7 (true/false) and E4.
- The spectral theorem and applications. Theorem 8.34 and Theorem 8.35 in full; run Interactive 1 with $\min(x,t)$ and then with $\cos\pi(x-t)$ to show a double eigenvalue. Examples 3 and 4; Interactive 2 to "see" $\|T-T_N\|=|\lambda_{N+1}|$. Theorem 8.36(2) via Example 5; Theorem 8.37 with Example 6. Drill E2, E5, E6.
Discussion questions. (1) The proof of Theorem 8.34 maximises $|\langle Tx,x\rangle|$. Where exactly does compactness enter, and what happens for $M_t$? (2) Why does a self-adjoint operator have $r(T)=\|T\|$, and why is this false for the Volterra operator? (3) A differential operator is unbounded, yet we proved completeness of its eigenfunctions with a theorem about bounded operators. How?
Misconceptions to watch for. "Compact means bounded"; "the spectrum is the set of eigenvalues"; "$0$ is an eigenvalue of every compact operator"; "every compact operator has an eigenvalue"; "$\|T\|=\max|\lambda|$" for non-symmetric kernels; confusing the characteristic value $\lambda$ of $u=f+\lambda Tu$ with the eigenvalue $1/\lambda$ of $T$; forgetting that the kernel of a degenerate-kernel operator is infinite-dimensional.
Suggested homework. I(1, 2, 3, 5, 6), II(1, 2, 4, 5, 8), III(1, 3, 4, 6, 7), IV(1, 2, 3, 6, 7); GATE corner E3, E5, E6.
Research question. A random curve — a Brownian path $B(t)$ on $[0,1]$, whose covariance is $\operatorname{Cov}(B(s),B(t))=\min(s,t)$ — can be expanded in the eigenfunctions of its covariance operator (the Karhunen–Loève expansion). How many terms are needed to capture 95% and 99% of its total variance, and how good do the truncated random curves look?
Why it matters. Karhunen–Loève expansions are principal component analysis for functions. They are used to compress and denoise climate records and growth curves, to simulate random loads in structural engineering, to price path-dependent financial options, and to reduce the dimension of uncertain inputs in engineering simulations. The spectral theorem guarantees the expansion; the eigenvalues say how much each term is worth.
What to do.
- Using Example 4, write $B(t)=\sum_n\sqrt{\lambda_n}\,Z_n\,e_n(t)$ with independent standard normal $Z_n$, $\lambda_n=\frac4{(2n-1)^2\pi^2}$, $e_n=\sqrt2\sin((n-\frac12)\pi t)$. Show that the expected value of $\|B\|_2^2$ is $\sum\lambda_n=\frac12$ (Mercer).
- Compute the fraction $F_N=\sum_{n\le N}\lambda_n/\frac12$ and find the smallest $N$ with $F_N\ge0.95$ and with $F_N\ge0.99$.
- Discretise the kernel on a grid of 200 points (Nyström), compute the eigenvalues numerically and compare with the exact ones.
- Simulate a few Brownian paths with $N=5$ and $N=50$ terms (seeded random numbers) and compare them with a random-walk simulation. Repeat steps 2–3 for the Brownian bridge kernel $\min(s,t)-st$.
Tools: Python with NumPy (or MATLAB/Octave); a plotting tool.
Hand in: A table of $F_N$, the eigenvalue comparison, two plots of simulated paths, and a paragraph explaining to a climate scientist what the first eigenfunction "means".
Going further. Read about functional principal component analysis and about Mercer kernels in machine learning (kernel PCA, Gaussian processes), where the same spectral theorem is used with kernels such as $e^{-|s-t|^2}$.
🧪 Self-check quiz
Q1. Which operator on $\ell^2$ is compact?
Q2. The spectrum of the diagonal operator $x\mapsto(x_n/n)$ on $\ell^2$ is
Q3. For a compact self-adjoint operator $T$, $\|T\|$ equals
Q4. The Volterra operator $Vu(x)=\int_0^xu$ on $L^2[0,1]$ is
Q5. The eigenfunctions $\sqrt2\sin n\pi x$ of $-u''=\lambda u$, $u(0)=u(1)=0$, form an orthonormal basis of $L^2[0,1]$ because