8.3 Hilbert Spaces, Orthonormal Bases, Projection and Riesz Representation
Inner products bring the geometry of the plane back to infinite dimensions: the parallelogram law, orthogonal complements and the projection theorem, Bessel's inequality and Parseval's identity for orthonormal bases of $\ell^2$ and $L^2$, the Riesz representation of every bounded functional as an inner product, and the Hilbert-space adjoint.
A Banach space measures lengths but not angles. Sections 8.1 and 8.2 showed what can still be done with a norm alone, and also what goes missing: closest points need not exist or be unique, a functional can fail to attain its norm, and Hahn–Banach extensions are usually far from unique. Add an inner product and the geometry of Euclidean space returns almost completely. Every closed subspace has an orthogonal complement and an orthogonal projection; every vector is the sum of its "Fourier series" with respect to an orthonormal basis, and its length can be read off from its coefficients (Parseval); and every bounded linear functional is simply an inner product with a fixed vector (Riesz). These facts are the foundation of least-squares approximation, Fourier analysis, signal processing, the finite-element method and quantum mechanics, where states are unit vectors of a Hilbert space.
The model spaces are $\ell^2$ and $L^2[a,b]$; the main warning is that an inner-product space which is not complete — such as $C[0,1]$ with $\int_0^1 f\bar g$ — loses the projection theorem and the Riesz theorem. GATE asks for numbers that this geometry makes easy: the distance from a function to a subspace of polynomials, the norm of a functional on $L^2$ or $\ell^2$, the sum of a series via Parseval's identity. It also asks MSQs on which norms come from inner products, on orthogonal complements of non-closed subspaces, and on the properties of orthonormal sequences. Section 8.4 builds on everything here to diagonalise compact self-adjoint operators.
Inner Products and Hilbert Spaces
As in §8.1, $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. Inner products are linear in the first variable and conjugate-linear in the second (physicists usually use the opposite convention); over $\mathbb{R}$ every bar below can be ignored.
An inner product on a vector space $H$ over $\mathbb{F}$ is a map $\langle\cdot,\cdot\rangle:H\times H\to\mathbb{F}$ such that for all $x,y,z\in H$ and $\alpha\in\mathbb{F}$:
Then $\langle x,\alpha y+z\rangle=\bar\alpha\langle x,y\rangle+\langle x,z\rangle$ and $\langle x,x\rangle$ is real. The induced norm is $\|x\|=\sqrt{\langle x,x\rangle}$ (a norm by Theorem 8.20). A vector space with an inner product is an inner-product space (or pre-Hilbert space); if it is complete in the induced norm it is a Hilbert space.
Proof. If $y=0$ both sides vanish. Otherwise, for every scalar $\lambda$, $$\begin{aligned}0\le\|x-\lambda y\|^2&=\|x\|^2-\bar\lambda\langle x,y\rangle\\&\quad-\lambda\langle y,x\rangle+|\lambda|^2\|y\|^2.\end{aligned}$$ Choose $\lambda=\langle x,y\rangle/\|y\|^2$: the three last terms become $-2|\langle x,y\rangle|^2/\|y\|^2+|\langle x,y\rangle|^2/\|y\|^2$, so $0\le\|x\|^2-|\langle x,y\rangle|^2/\|y\|^2$, which is the inequality. Equality holds iff $\|x-\lambda y\|=0$, i.e. $x=\lambda y$; and if $x$, $y$ are dependent with $y\ne0$, then $x=\lambda y$ for some $\lambda$ and equality is immediate. For the triangle inequality, $$\begin{aligned}\|x+y\|^2&=\|x\|^2+2\operatorname{Re}\langle x,y\rangle+\|y\|^2\\&\le(\|x\|+\|y\|)^2.\end{aligned}$$ The other norm axioms are immediate. Continuity: $|\langle x_n,y_n\rangle-\langle x,y\rangle|$ is at most $\|x_n-x\|\,\|y_n\|+\|x\|\,\|y_n-y\|\to0$, since $(\|y_n\|)$ is bounded. ∎
- Parallelogram law. In an inner-product space, $\|x+y\|^2+\|x-y\|^2=2\|x\|^2+2\|y\|^2$.
- Polarization. Over $\mathbb{R}$, $\langle x,y\rangle=\tfrac14\big(\|x+y\|^2-\|x-y\|^2\big)$; over $\mathbb{C}$, $\langle x,y\rangle=\tfrac14\sum_{k=0}^{3}i^k\|x+i^ky\|^2$.
- Jordan–von Neumann. A norm on a vector space is induced by an inner product if and only if it satisfies the parallelogram law; the inner product is then given by (2).
Proof. (1), (2). Expanding as in the proof of Theorem 8.20, $\|x+cy\|^2=\|x\|^2+2\operatorname{Re}\big(\bar c\langle x,y\rangle\big)+\|y\|^2$ for $|c|=1$. With $c=\pm1$ and adding we get (1); subtracting gives $4\operatorname{Re}\langle x,y\rangle$, which is the real case of (2). Over $\mathbb{C}$ put $z=\langle x,y\rangle$; the terms $\|x\|^2+\|y\|^2$ cancel because $\sum_ki^k=0$, and $$\begin{aligned}\sum_{k=0}^3i^k\cdot2\operatorname{Re}(i^{-k}z)&=2\big(\operatorname{Re}z+i\operatorname{Im}z\\&\qquad+\operatorname{Re}z+i\operatorname{Im}z\big)=4z.\end{aligned}$$
(3) Necessity is (1). For sufficiency (real case) define $\langle x,y\rangle=\tfrac14(\|x+y\|^2-\|x-y\|^2)$. It is symmetric, $\langle x,x\rangle=\|x\|^2$ and $\langle 0,y\rangle=0$. The key step is additivity in $x$. Apply the parallelogram law to the pairs $x+y,\ x'+y$ and $x-y,\ x'-y$: $$\begin{aligned}\|x+y\|^2+\|x'+y\|^2&=\tfrac12\|x+x'+2y\|^2\\&\quad+\tfrac12\|x-x'\|^2,\\ \|x-y\|^2+\|x'-y\|^2&=\tfrac12\|x+x'-2y\|^2\\&\quad+\tfrac12\|x-x'\|^2.\end{aligned}$$ Subtracting and dividing by $4$: $\langle x,y\rangle+\langle x',y\rangle=\tfrac12\langle x+x',2y\rangle$. Taking $x'=0$ gives $\langle w,y\rangle=\tfrac12\langle w,2y\rangle$ for every $w$; with $w=x+x'$ we obtain $\langle x,y\rangle+\langle x',y\rangle=\langle x+x',y\rangle$. Additivity gives $\langle rx,y\rangle=r\langle x,y\rangle$ for rational $r$, and both sides are continuous in $r$ (the norm is continuous), so this holds for all real $r$. In the complex case one defines $\langle x,y\rangle$ by the complex polarization formula, applies the real result to the real and imaginary parts, and checks $\langle ix,y\rangle=i\langle x,y\rangle$ directly. ∎
Every inner-product space $X$ can be completed: there is a Hilbert space $H$ containing $X$ as a dense subspace, with the inner product extended by continuity. The completion of $C[a,b]$ (or of the polynomials) in $\|\cdot\|_2$ is $L^2[a,b]$, and that of $c_{00}$ is $\ell^2$.
Orthogonality and the Projection Theorem
Proof. Choose $k_n\in K$ with $\|x-k_n\|\to d$. The parallelogram law for the vectors $x-k_n$ and $x-k_m$ gives $$\begin{aligned}\|k_n-k_m\|^2&=2\|x-k_n\|^2+2\|x-k_m\|^2\\&\quad-4\big\|x-\tfrac12(k_n+k_m)\big\|^2.\end{aligned}$$ By convexity $\tfrac12(k_n+k_m)\in K$, so the last norm is at least $d$, and the right-hand side is at most $2\|x-k_n\|^2+2\|x-k_m\|^2-4d^2\to0$. Thus $(k_n)$ is Cauchy; $H$ is complete and $K$ closed, so $k_n\to k_0\in K$, and $\|x-k_0\|=d$ by continuity of the norm. If $k_1\in K$ also has $\|x-k_1\|=d$, the same identity with $k_0,k_1$ gives $\|k_0-k_1\|^2\le2d^2+2d^2-4d^2=0$.
For the characterisation write $x-k=(x-k_0)-(k-k_0)$, so that $$\begin{aligned}\|x-k\|^2&=\|x-k_0\|^2-2\operatorname{Re}\langle x-k_0,k-k_0\rangle\\&\quad+\|k-k_0\|^2.\end{aligned}$$ If the real part is $\le0$ for all $k$, then $\|x-k\|\ge\|x-k_0\|$. Conversely, if $k_0$ is the closest point, apply the identity to $k_0+s(k-k_0)\in K$, $0\lt s\le1$: $0\le-2s\operatorname{Re}\langle x-k_0,k-k_0\rangle+s^2\|k-k_0\|^2$; divide by $s$ and let $s\to0$. ∎
The proof uses exactly two things: the parallelogram law (for uniqueness and for the Cauchy estimate) and completeness of $K$. In a Banach space closest points can fail to exist or fail to be unique — Interactive 1 below shows the second failure in the plane.
- Every $x\in H$ can be written uniquely as $x=m+n$ with $m\in M$, $n\in M^\perp$: $\ H=M\oplus M^\perp$. Here $m=P_Mx$ is the closest point of $M$ to $x$, characterised by $m\in M$ and $x-m\perp M$.
- The orthogonal projection $P=P_M$ is linear and bounded, $P^2=P$, $\langle Px,y\rangle=\langle x,Py\rangle$, $\|Px\|^2+\|x-Px\|^2=\|x\|^2$, and $\|P\|=1$ if $M\ne\{0\}$. Its range is $M$, its kernel is $M^\perp$, and $I-P=P_{M^\perp}$.
- $M^{\perp\perp}=M$. For an arbitrary subset $S$, $S^{\perp\perp}=\overline{\operatorname{span}}\,S$; in particular a subspace $N$ is dense in $H$ if and only if $N^\perp=\{0\}$.
Proof. (1) A subspace is convex, so Theorem 8.22 gives a unique closest point $m\in M$. For a subspace, $k-m$ runs through all of $M$ as $k$ does, so the characterisation reads $\operatorname{Re}\langle x-m,w\rangle\le0$ for all $w\in M$; replacing $w$ by $-w$ and by $iw$ shows $\langle x-m,w\rangle=0$, i.e. $x-m\in M^\perp$. Conversely, if $m\in M$ and $x-m\perp M$, then for $k\in M$ Pythagoras gives $\|x-k\|^2=\|x-m\|^2+\|m-k\|^2\ge\|x-m\|^2$, so $m$ is the closest point. Uniqueness of the decomposition: if $m+n=m'+n'$, then $m-m'=n'-n\in M\cap M^\perp=\{0\}$.
(2) If $x=Px+(x-Px)$ and $y=Py+(y-Py)$, then $\alpha x+y=(\alpha Px+Py)+\big(\alpha(x-Px)+(y-Py)\big)$ is a decomposition of the required kind, so by uniqueness $P(\alpha x+y)=\alpha Px+Py$. Pythagoras gives $\|Px\|^2+\|x-Px\|^2=\|x\|^2$, hence $\|Px\|\le\|x\|$, with equality for $x\in M$, where $Px=x$; so $\|P\|=1$ and $P^2=P$. Next, $\langle Px,y\rangle=\langle Px,Py\rangle+\langle Px,y-Py\rangle=\langle Px,Py\rangle$, and symmetrically $\langle x,Py\rangle=\langle Px,Py\rangle$. The statements about range and kernel follow from (1). Finally $x=(x-Px)+Px$ with $x-Px\in M^\perp$ and $Px\in M\subseteq(M^\perp)^\perp$; this is the decomposition of $x$ with respect to the closed subspace $M^\perp$, so $P_{M^\perp}x=x-Px$.
(3) Always $M\subseteq M^{\perp\perp}$. If $x\in M^{\perp\perp}$, write $x=m+n$ as in (1). Then $n=x-m\in M^{\perp\perp}$ (a subspace containing $M$) and $n\in M^\perp$, so $\langle n,n\rangle=0$. Hence $x=m\in M$. For a subset $S$, $S^\perp=(\overline{\operatorname{span}}\,S)^\perp$, so $S^{\perp\perp}=(\overline{\operatorname{span}}\,S)^{\perp\perp}=\overline{\operatorname{span}}\,S$. Finally $N^\perp=\{0\}$ iff $N^{\perp\perp}=H$ iff $\overline N=H$. ∎
- Orthonormal basis of $M$ known ($e_1,\dots,e_n$): $P_Mx=\sum\langle x,e_k\rangle e_k$ and $\operatorname{dist}(x,M)^2=\|x\|^2-\sum|\langle x,e_k\rangle|^2$ (Theorem 8.24).
- Spanning set $x_1,\dots,x_n$ only: solve the normal equations $\sum_jc_j\langle x_j,x_i\rangle=\langle x,x_i\rangle$; then $\operatorname{dist}^2=\langle x-P_Mx,x\rangle$, or use the Gram determinants $\operatorname{dist}^2=\dfrac{G(x_1,\dots,x_n,x)}{G(x_1,\dots,x_n)}$ with $G=\det\big(\langle x_j,x_i\rangle\big)$. For Example 2(a): $\tfrac{1/2160}{1/12}=\tfrac1{180}$.
- Hyperplane $M=\{y\}^\perp$: $\operatorname{dist}(x,M)=|\langle x,y\rangle|/\|y\|$. Line $\operatorname{span}\{y\}$: $\operatorname{dist}^2=\|x\|^2-|\langle x,y\rangle|^2/\|y\|^2$.
- Complement trick: $\operatorname{dist}(x,M)=\|P_{M^\perp}x\|$ — if $M$ is cut out by finitely many conditions, project onto the small space $M^\perp$ instead.
- Symmetry on $[-a,a]$: even and odd functions are orthogonal; project the even and odd parts of $x$ separately. Legendre polynomials on $[-1,1]$ (and shifted Legendre on $[0,1]$) are ready-made orthogonal bases of polynomial subspaces.
Orthonormal Sets, Bessel, Parseval and Orthonormal Bases
- For finitely many $e_1,\dots,e_n$ and scalars $c_k$, $$\begin{aligned}\Big\|x-\sum_{k=1}^nc_ke_k\Big\|^2&=\|x\|^2-\sum_{k=1}^n|\langle x,e_k\rangle|^2\\&\quad+\sum_{k=1}^n|c_k-\langle x,e_k\rangle|^2.\end{aligned}$$ So the closest point of $\operatorname{span}\{e_1,\dots,e_n\}$ to $x$ is $\sum\langle x,e_k\rangle e_k$, uniquely.
- Bessel's inequality: $\sum_\alpha|\langle x,e_\alpha\rangle|^2\le\|x\|^2$. In particular only countably many Fourier coefficients of $x$ are nonzero, and $\langle x,e_n\rangle\to0$ for every orthonormal sequence $(e_n)$.
Proof. (1) Put $a_k=\langle x,e_k\rangle$. By orthonormality, $$\begin{aligned}\Big\|x-\sum c_ke_k\Big\|^2&=\|x\|^2-\sum\bar c_ka_k\\&\quad-\sum c_k\bar a_k+\sum|c_k|^2.\end{aligned}$$ Since $$|c_k-a_k|^2=|c_k|^2-c_k\bar a_k-\bar c_ka_k+|a_k|^2,$$ this equals $\|x\|^2-\sum|a_k|^2+\sum|c_k-a_k|^2$, which is smallest exactly when every $c_k=a_k$. (2) Taking $c_k=a_k$ in (1) gives $\sum_{k\le n}|a_k|^2\le\|x\|^2$ for every finite subfamily, so the sum over all $\alpha$ (the supremum of finite partial sums) is at most $\|x\|^2$. For each $m$, fewer than $m^2\|x\|^2+1$ indices have $|\langle x,e_\alpha\rangle|\gt\frac1m$; the union over $m$ of these finite sets is countable. For a sequence, the terms of a convergent series tend to $0$. ∎
Proof. For the partial sums $s_N=\sum_{n\le N}c_ne_n$ and $M\gt N$, Pythagoras gives $\|s_M-s_N\|^2=\sum_{N\lt n\le M}|c_n|^2$. So $(s_N)$ is Cauchy in $H$ iff the partial sums of $\sum|c_n|^2$ are Cauchy in $\mathbb{R}$; as $H$ is complete, Cauchy is the same as convergent. The formulas follow from $\|s_N\|^2=\sum_{n\le N}|c_n|^2$ and $\langle s_N,e_k\rangle=c_k$ ($N\ge k$) by continuity. For a rearrangement with partial sums $s'_N$: given $\varepsilon\gt0$ choose $K$ with $\sum_{n\gt K}|c_n|^2\lt\varepsilon$; once $N$ is so large that the first $N$ rearranged terms include $c_1e_1,\dots,c_Ke_K$, the vector $s'_N-s_K$ is a finite sum of terms with indices $\gt K$, so $\|s'_N-s_K\|^2\lt\varepsilon$, while $\|s-s_K\|^2\le\varepsilon$. Hence $\|s'_N-s\|\lt2\sqrt\varepsilon$, and the rearranged series also converges to $s$. Finally, by Bessel $\sum|\langle x,e_n\rangle|^2\lt\infty$, so $p=\sum\langle x,e_n\rangle e_n$ exists; it lies in $M=\overline{\operatorname{span}}\{e_n\}$ and $\langle x-p,e_k\rangle=0$ for all $k$, so $x-p\perp M$ and $p=P_Mx$ by Theorem 8.23(1). ∎
- $\{e_\alpha\}$ is maximal (an orthonormal basis);
- if $\langle x,e_\alpha\rangle=0$ for all $\alpha$, then $x=0$;
- the linear span of $\{e_\alpha\}$ is dense in $H$;
- $x=\sum_\alpha\langle x,e_\alpha\rangle e_\alpha$ for every $x\in H$;
- Parseval's identity: $\|x\|^2=\sum_\alpha|\langle x,e_\alpha\rangle|^2$ for every $x$;
- $\langle x,y\rangle=\sum_\alpha\langle x,e_\alpha\rangle\overline{\langle y,e_\alpha\rangle}$ for all $x,y$.
Proof. (1)⇔(2): if $x\ne0$ is orthogonal to every $e_\alpha$, then adding $x/\|x\|$ gives a larger orthonormal set; conversely a strictly larger orthonormal set contains a unit vector orthogonal to every $e_\alpha$. (2)⇔(3): by Theorem 8.23(3), the span is dense iff its orthogonal complement — which is $\{e_\alpha\}^\perp$ — is $\{0\}$. (2)⇒(4): only countably many coefficients of $x$ are nonzero (Bessel); list them as a sequence and let $p=\sum\langle x,e_\alpha\rangle e_\alpha$ (Theorem 8.25). Then $x-p$ is orthogonal to every $e_\alpha$, so $x=p$. (4)⇒(6): apply the continuity of the inner product to the partial sums of the expansions of $x$ and $y$. (6)⇒(5): take $y=x$. (5)⇒(2): if all coefficients vanish, $\|x\|^2=0$.
Existence. Order the orthonormal subsets of $H$ by inclusion. The union of a chain is orthonormal (two of its vectors lie in a common member), so by Zorn's lemma there is a maximal orthonormal set. Separability. If $\{x_k\}$ is dense, discard each $x_k$ that is a combination of the earlier ones and apply Gram–Schmidt (§3.3) to the rest: this gives a countable orthonormal set whose span equals $\operatorname{span}\{x_k\}$, which is dense; it is a basis by (3). Conversely, finite combinations of a countable basis with coefficients in $\mathbb{Q}$ (or $\mathbb{Q}+i\mathbb{Q}$) form a countable dense set, by (4). Isomorphism. $Ux=(\langle x,e_n\rangle)$ maps $H$ into $\ell^2$ by Bessel, preserves norms and inner products by (5) and (6), and is onto by Theorem 8.25: $(c_n)\in\ell^2$ is $U\big(\sum c_ne_n\big)$. ∎
For Fourier series it is convenient to write $\hat f(k)=\frac1{2\pi}\int_{-\pi}^{\pi}f(t)e^{-ikt}dt$, so that $\langle f,e^{ikt}/\sqrt{2\pi}\rangle=\sqrt{2\pi}\,\hat f(k)$ and Parseval reads $$\frac1{2\pi}\int_{-\pi}^{\pi}|f|^2=\sum_{k\in\mathbb{Z}}|\hat f(k)|^2.$$
The Riesz Representation Theorem and Adjoint Operators
In §8.1 we found $(\ell^2)^*=\ell^2$ by a computation special to sequences. In a Hilbert space the projection theorem gives the same conclusion for every space at once.
Proof. By Cauchy–Schwarz $|f_y(x)|\le\|x\|\,\|y\|$, with equality at $x=y$; so $\|f_y\|=\|y\|$. If $\langle x,y\rangle=\langle x,y'\rangle$ for all $x$, take $x=y-y'$ to get $y=y'$: the representing vector is unique. Existence. If $f=0$ take $y_f=0$. Otherwise $N=\ker f$ is a closed (because $f$ is continuous) proper subspace, so by the projection theorem $N^\perp\ne\{0\}$. Choose $z\in N^\perp$ with $f(z)=1$. For every $x$, $f\big(x-f(x)z\big)=0$, so $x-f(x)z\in N$ is orthogonal to $z$: $$0=\langle x-f(x)z,z\rangle=\langle x,z\rangle-f(x)\|z\|^2.$$ Hence $f(x)=\langle x,y_f\rangle$ with $y_f=z/\|z\|^2$. Conjugate-linearity: $\langle x,\alpha y\rangle=\bar\alpha\langle x,y\rangle$. With the stated inner product, $\langle\alpha f_y,f_z\rangle=\langle f_{\bar\alpha y},f_z\rangle=\langle z,\bar\alpha y\rangle=\alpha\langle f_y,f_z\rangle$, and $\langle f_y,f_y\rangle=\|y\|^2=\|f_y\|^2$, so $H^*$ is a Hilbert space. If $\varphi\in H^{**}$, Riesz in $H^*$ gives $f_z$ with $\varphi(f_y)=\langle f_y,f_z\rangle=\langle z,y\rangle=f_y(z)$ for all $y$: $\varphi$ is evaluation at $z$, so the canonical map $H\to H^{**}$ is onto. ∎
Two consequences answer questions left open in §8.1. In a Hilbert space a bounded functional $g$ on a subspace $M$ has a unique norm-preserving extension, namely $F=g\circ P_{\overline M}$ (Exercise IV.5) — the disc has a unique tangent at each boundary point. And every bounded functional on a Hilbert space attains its norm (at $y_f/\|y_f\|$), in contrast with $c_0$.
- $T^*$ exists, is unique, linear and bounded.
- $(S+T)^*=S^*+T^*$, $(\alpha T)^*=\bar\alpha T^*$, $(ST)^*=T^*S^*$, $T^{**}=T$, $I^*=I$.
- $\|T^*\|=\|T\|$ and $\|T^*T\|=\|T\|^2$.
- $\ker T^*=(\operatorname{ran}T)^\perp$ and $\overline{\operatorname{ran}T}=(\ker T^*)^\perp$; likewise with $T$ and $T^*$ exchanged.
- An operator $P\in B(H)$ is an orthogonal projection (onto a closed subspace) if and only if $P^2=P=P^*$.
Proof. (1) Fix $y\in K$. The map $x\mapsto\langle Tx,y\rangle$ is linear with $|\langle Tx,y\rangle|\le\|T\|\,\|x\|\,\|y\|$, so by Theorem 8.27 there is a unique vector $T^*y\in H$ with $\langle Tx,y\rangle=\langle x,T^*y\rangle$ for all $x$, and $\|T^*y\|\le\|T\|\,\|y\|$. Linearity: $\langle x,T^*(\alpha y+y')\rangle=\langle Tx,\alpha y+y'\rangle$, which is $\bar\alpha\langle Tx,y\rangle+\langle Tx,y'\rangle=\langle x,\alpha T^*y+T^*y'\rangle$ for all $x$. (2) Each identity follows from uniqueness, e.g. $\langle STx,y\rangle=\langle Tx,S^*y\rangle=\langle x,T^*S^*y\rangle$, and $\langle T^*y,x\rangle=\overline{\langle x,T^*y\rangle}=\overline{\langle Tx,y\rangle}=\langle y,Tx\rangle$ shows $T^{**}=T$. (3) From (1), $\|T^*\|\le\|T\|$; applied to $T^*$ it gives $\|T\|=\|T^{**}\|\le\|T^*\|$. Also $\|T^*T\|\le\|T^*\|\,\|T\|=\|T\|^2$, while $\|Tx\|^2=\langle T^*Tx,x\rangle\le\|T^*T\|\,\|x\|^2$ gives $\|T\|^2\le\|T^*T\|$. (4) $y\in\ker T^*$ iff $\langle x,T^*y\rangle=0$ for all $x$ iff $\langle Tx,y\rangle=0$ for all $x$ iff $y\perp\operatorname{ran}T$. Taking complements and using Theorem 8.23(3) gives the second identity. (5) Orthogonal projections satisfy $P^2=P=P^*$ by Theorem 8.23(2). Conversely let $P^2=P=P^*$ and $M=\operatorname{ran}P=\ker(I-P)$, a closed subspace. For every $x$, $Px\in M$, and for $m=Pz\in M$, $\langle x-Px,Pz\rangle=\langle P(x-Px),z\rangle$ $=\langle Px-P^2x,z\rangle=0$. So $x-Px\perp M$, and $Px=P_Mx$ by Theorem 8.23(1). ∎
- Most norms are not inner-product norms. $\ell^p$ and $L^p$ for $p\ne2$, $c_0$, $\ell^\infty$ and $(C[a,b],\|\cdot\|_\infty)$ all fail the parallelogram law (test $e_1,e_2$, or $1$ and $t$).
- Inner-product space ≠ Hilbert space. $C[0,1]$ with $\int f\bar g$, the polynomials, and $c_{00}\subseteq\ell^2$ are incomplete.
- The projection theorem needs a closed subspace of a complete space. If $M$ is dense but not closed (polynomials in $L^2$, $c_{00}$ in $\ell^2$), then $M^\perp=\{0\}$ and $M\oplus M^\perp=M\ne H$; always $M^{\perp\perp}=\overline M$, not $M$. In the incomplete space $c_{00}$, even a closed subspace can have $M^\perp=\{0\}$ (Example 3).
- Closest points in Banach spaces may be non-unique — $(\mathbb{R}^2,\|\cdot\|_\infty)$, the $x$-axis and $(0,1)$: every $(t,0)$ with $|t|\le1$ — or may not exist: in $c_0$ no point of $\ker f$, $f(x)=\sum x_n2^{-n}$, is closest to $e_1$, because $f$ does not attain its norm (§8.1).
- Orthonormal basis ≠ Hamel basis; a non-separable Hilbert space such as $\ell^2(\mathbb{R})$ has uncountable orthonormal bases, and "basis" never means finite combinations.
- Orthonormal sequences converge weakly to $0$ but are never norm-Cauchy ($\|e_n-e_m\|=\sqrt2$); the closed unit ball of an infinite-dimensional Hilbert space is not compact.
- Bessel is not Parseval. Equality for every $x$ characterises orthonormal bases; $\{\sin nt\}$ on $[-\pi,\pi]$ and $\{e_{2n}\}$ in $\ell^2$ are orthonormal but incomplete. And in an incomplete space a maximal orthonormal set can violate Parseval.
- Riesz needs completeness and boundedness. $f(x)=\sum x_n/n$ on $c_{00}$ is bounded with no representer in $c_{00}$; $u\mapsto u(0)$ on $(C[0,1],\|\cdot\|_2)$ is unbounded and represented by nothing.
- The Riesz map is conjugate-linear over $\mathbb{C}$: $y_{\alpha f}=\bar\alpha y_f$.
- Adjoints. $\|T^*\|=\|T\|$ but $T^*\ne T$ in general; $S^*S=I$ does not imply $SS^*=I$ (the shift); a product of self-adjoint operators is self-adjoint only if they commute.
Exercises
- 1 Which of the following define inner products on $\mathbb{R}^2$? (a) $x_1y_1+2x_2y_2-x_1y_2-x_2y_1$; (b) $x_1y_1-x_2y_2$; (c) $x_1y_2+x_2y_1$; (d) $3x_1y_1+x_1y_2+x_2y_1+x_2y_2$.
- 2 Prove that in a real inner-product space $x\perp y$ iff $\|x+y\|=\|x-y\|$ iff $\|x+y\|^2=\|x\|^2+\|y\|^2$. Show by an example in $\mathbb{C}$ that the last equivalence fails over $\mathbb{C}$.
- 3 Using $f=\mathbf 1_{[0,1/2]}$ and $g=\mathbf 1_{(1/2,1]}$, show that the norm of $L^p[0,1]$ is induced by an inner product only for $p=2$.
- 4 Show that the space $\mathcal{P}$ of all polynomials with $\langle p,q\rangle=\int_0^1p\bar q$ is an inner-product space that is not a Hilbert space.
- 5 Let $D$ be a dense subset of an inner-product space $X$. Prove that if $\langle x,z\rangle=\langle y,z\rangle$ for all $z\in D$, then $x=y$.
- 6 Prove that $\|x+y\|=\|x\|+\|y\|$ in an inner-product space iff one of $x,y$ is a non-negative multiple of the other. Show that this fails in $\ell^1$.
- 7 Prove Apollonius' identity: $\|z-x\|^2+\|z-y\|^2$ $=\tfrac12\|x-y\|^2+2\big\|z-\tfrac12(x+y)\big\|^2$.
- 8 On $C^1[0,1]$ let $\langle u,v\rangle=\int_0^1(uv+u'v')$. Show this is an inner product and that the space is not complete. (Hint: $u_n(x)=\sqrt{(x-\frac12)^2+\frac1{n^2}}$.)
- 1 In $\mathbb{R}^3$ find the orthogonal projection of $(1,2,3)$ onto the plane $x+y+z=0$, and its distance from the plane.
- 2 In real $L^2[-1,1]$ let $M$ be the subspace of even functions. Show that $M^\perp$ is the subspace of odd functions, find $P_Me^t$ and $\operatorname{dist}(e^t,M)$.
- 3 In real $\ell^2$, find $\operatorname{dist}(e_1,M)$ for $M=\{x:\sum_nx_n2^{-n}=0\}$.
- 4 Let $M=\{f\in L^2[0,1]:\int_0^1f=0\}$. Show $M$ is closed, find $M^\perp$, $P_M$, and $\operatorname{dist}(t,M)$.
- 5 Let $P,Q$ be the orthogonal projections onto closed subspaces $M,N$. Prove that $PQ$ is an orthogonal projection iff $PQ=QP$, and that it is then the projection onto $M\cap N$.
- 6 Let $M=\{f\in L^2[0,1]:f=0\text{ a.e. on }[0,\tfrac12]\}$. Show $M$ is closed and find $M^\perp$ and $P_M$.
- 7 Give a proper dense subspace $N$ of $L^2[0,1]$, and find $N^\perp$ and $N^{\perp\perp}$.
- 8 Find the polynomial $q$ of degree $\le2$ minimising $\int_{-1}^1\big(|t|-q(t)\big)^2dt$, and the minimum distance.
- 1 Verify that $\{e^{ikt}/\sqrt{2\pi}\}_{k\in\mathbb{Z}}$ is orthonormal in $L^2[-\pi,\pi]$ and $\{\sqrt2\sin n\pi t\}_{n\ge1}$ is orthonormal in $L^2[0,1]$.
- 2 Expand $f(t)=t$ in the basis $\{\sqrt2\sin n\pi t\}$ of $L^2[0,1]$ and use Parseval to show again that $\sum1/n^2=\pi^2/6$.
- 3 Use the square wave $\operatorname{sgn}t$ on $[-\pi,\pi]$ to prove $\sum_{k\ge1}\frac1{(2k-1)^2}=\frac{\pi^2}8$.
- 4 Apply Gram–Schmidt to $1,t,t^2$ in $L^2[0,1]$.
- 5 Prove that every orthonormal set in a separable Hilbert space is countable.
- 6 Prove that $\int_{-\pi}^{\pi}f(t)\cos nt\,dt\to0$ as $n\to\infty$ for every $f\in L^2[-\pi,\pi]$.
- 7 Let $(e_n)$ be an orthonormal basis and $(f_n)$ an orthonormal sequence with $\sum_n\|e_n-f_n\|^2\lt1$. Prove that $(f_n)$ is an orthonormal basis.
- 8 The vectors $e_n-e_{n+1}$ ($n\ge1$) are not orthogonal. Show that nevertheless their linear span is dense in $\ell^2$.
- 1 Find the representing vector and the norm of $f(u)=\int_0^1e^tu(t)\,dt$ on $L^2[0,1]$.
- 2 Find the norm of $f(x)=\sum_{n\ge1}x_n/n$ on $\ell^2$ and a unit vector at which it is attained.
- 3 On $\mathcal{P}_1$ with $\langle p,q\rangle=\int_{-1}^1pq$, find $y$ with $p(1)=\langle p,y\rangle$ for all $p$, and the norm of $p\mapsto p(1)$.
- 4 Show that $u\mapsto u(\tfrac12)$ is not bounded on $(C[0,1],\|\cdot\|_2)$.
- 5 Let $M$ be a subspace of a Hilbert space $H$ and $g\in M^*$. Prove that $g$ has exactly one extension $F\in H^*$ with $\|F\|=\|g\|$.
- 6 Find the adjoint of the weighted shift $Tx=(0,x_1,\tfrac{x_2}2,\tfrac{x_3}3,\dots)$ on $\ell^2$, and $\|T\|$.
- 7 For the Volterra operator $V$ on $L^2[0,1]$, show that $V+V^*$ is the rank-one operator $u\mapsto\langle u,1\rangle1$.
- 8 Let $U\in B(H)$ be an isometry. Prove that $U^*U=I$ and that $UU^*$ is the orthogonal projection onto $\operatorname{ran}U$. When is $U$ unitary?
E1. GATE MA · MCQ Which one of the following norms on $\mathbb{R}^2$ is induced by an inner product? (A) $\|(x,y)\|=|x|+2|y|$ (B) $\|(x,y)\|=\max(|x|,|y|)$ (C) $\|(x,y)\|=\sqrt{2x^2-2xy+y^2}$ (D) $\|(x,y)\|=(x^4+y^4)^{1/4}$
E2. GATE MA · MCQ Let $f:L^2[0,1]\to\mathbb{R}$, $f(u)=\int_0^1(1+t)\,u(t)\,dt$ (real scalars). Then $\|f\|$ equals (A) $\tfrac32$ (B) $2$ (C) $\sqrt{7/3}$ (D) $\tfrac73$
E3. GATE MA · MSQ CSIR Part C Let $M$ be a subspace (not necessarily closed) of a Hilbert space $H$. Which of the following statements are true? (One or more options may be correct.) (A) $M^\perp$ is a closed subspace of $H$. (B) $(M^\perp)^\perp=M$. (C) If $M$ is closed, then every $x\in H$ is uniquely $m+n$ with $m\in M$, $n\in M^\perp$. (D) If $M^\perp=\{0\}$, then $M=H$.
E4. GATE MA · MSQ Let $(e_n)_{n\ge1}$ be an orthonormal sequence in an infinite-dimensional Hilbert space $H$. Which of the following are true? (One or more options may be correct.) (A) $\langle x,e_n\rangle\to0$ for every $x\in H$. (B) $(e_n)$ has a norm-convergent subsequence. (C) $\sum_n\langle x,e_n\rangle e_n$ converges in $H$ for every $x\in H$. (D) If $\sum_n|\langle x,e_n\rangle|^2=\|x\|^2$ for every $x\in H$, then the closed linear span of $\{e_n\}$ is $H$.
E5. GATE MA · NAT In real $L^2[-1,1]$, the distance from $f(t)=t^3$ to the subspace spanned by $1$ and $t$, correct to two decimal places, is ______.
E6. GATE MA · NAT The Fourier series of $f(t)=|t|$ on $[-\pi,\pi]$ is $\dfrac\pi2-\dfrac4\pi\displaystyle\sum_{k\ge1}\dfrac{\cos(2k-1)t}{(2k-1)^2}$. Using Parseval's identity, the value of $\displaystyle\sum_{k=1}^{\infty}\frac1{(2k-1)^4}$, correct to two decimal places, is ______.
Suggested plan — 3 lectures × 60 min.
- Inner products and geometry. Definition 8.10 with the rule cards of examples; Theorem 8.20 (the "choose $\lambda$" proof on the board); Theorem 8.21 — prove (1), (2) and the additivity step of (3), which students rarely see. Example 1 as a class quiz ("which norms are Euclidean?"), then GATE corner E1.
- Projections. Definition 8.11, Theorem 8.22 with a picture of the parallelogram argument, Theorem 8.23. Project Interactive 1: start at $p=2$, then switch to $p=\infty$ and let the animation stop at $\theta=0^\circ$; ask why the segment appears. Example 2 and the strategy box (Gram determinants); Example 3 to show what completeness buys. Finish with E3 and E5.
- Bases, Riesz and adjoints. Theorems 8.24–8.26 (prove 8.24 and the equivalences of 8.26); Interactive 2 in square-wave mode — let the class predict the Bessel sum at $N=1$ ($8/\pi^2\approx81\%$), then move $\delta$. Example 4 (Parseval sums); Theorem 8.27 in full; Definition 8.13, Theorem 8.28 and Example 7. E2, E4 and E6 as a closing drill.
Discussion questions. (1) The proof of the closest-point theorem uses the parallelogram law twice. Where exactly, and what goes wrong for the sup norm? (2) Why is "maximal orthonormal set" a better definition of basis than "orthonormal set whose span is everything"? (3) Riesz identifies $H^*$ with $H$. Why does the identification fail to be linear over $\mathbb{C}$, and does that matter for real GATE problems?
Misconceptions to watch for. "Every norm has a parallelogram law"; "$M^{\perp\perp}=M$ for every subspace"; "an orthonormal basis is a basis in the linear-algebra sense"; "Bessel's inequality is always an equality"; "every functional on an inner-product space is an inner product"; forgetting the square root in $\|f\|=\|y_f\|$; confusing $\int|g|$ (norm on $C$) with $\|g\|_2$ (norm on $L^2$).
Suggested homework. I(1, 3, 6, 8), II(1, 2, 4, 5, 8), III(2, 3, 4, 7), IV(1, 3, 5, 6, 8); GATE corner E1, E5, E6.
Research question. How many Fourier coefficients does it take to capture 99% of the energy of a signal, and how does the answer depend on the smoothness of the signal? Parseval's identity turns this into a question about the decay of the coefficients.
Why it matters. Transform coding — the idea behind JPEG images, MP3 audio and the compression of ECG recordings in portable heart monitors — keeps only the largest coefficients in an orthonormal basis and discards the rest. By Bessel and Parseval, the squared $L^2$ error of keeping a set of coefficients is exactly the energy of the discarded ones. Signals with jumps compress badly in a Fourier basis; smooth signals compress extremely well.
What to do.
- For the square wave $\operatorname{sgn}t$, the triangle wave $|t|-\frac\pi2$ and the smooth pulse $e^{\cos t}-I_0(1)$ (mean removed; its coefficients are modified Bessel values $I_k(1)$), compute the fraction $F_N=\|S_Nf\|^2/\|f\|^2$ of energy captured by harmonics up to $N$, using Parseval rather than numerical integration.
- Find the smallest $N$ with $F_N\ge0.99$ for each signal.
- Plot $1-F_N$ against $N$ on log–log axes and relate the slopes to the decay rates $|\hat f(k)|\sim k^{-1}$, $k^{-2}$ and faster than any power.
- Check one value against Interactive 2, and explain in one paragraph why the location of a jump cannot be "seen" by a few Fourier coefficients.
Tools: Python with NumPy and SciPy (scipy.special.iv for $I_k$), or a spreadsheet.
Hand in: A table of $F_N$ for $N=1,3,5,\dots$, the log–log plot, and a short paragraph recommending a basis for compressing (i) a smooth temperature cycle and (ii) a signal with abrupt switches.
Going further. Repeat with the Haar wavelet basis of $L^2[0,1]$, keeping the $N$ largest coefficients rather than the first $N$ (nonlinear approximation): a single jump then costs only about one coefficient per scale.
🧪 Self-check quiz
Q1. Which of these is a Hilbert space?
Q2. If $P$ is the orthogonal projection onto a closed subspace $M\ne\{0\}$, then $\|P\|$ equals
Q3. Bessel's inequality $\sum|\langle x,e_\alpha\rangle|^2\le\|x\|^2$ is an equality for every $x$ exactly when the orthonormal set $\{e_\alpha\}$ is
Q4. The norm of $u\mapsto\int_0^1u(t)\,dt$ on $L^2[0,1]$ is
Q5. The adjoint of the right shift $S(x_1,x_2,\dots)=(0,x_1,x_2,\dots)$ on $\ell^2$ is