Chapter 8 · Functional Analysis

8.3 Hilbert Spaces, Orthonormal Bases, Projection and Riesz Representation

Inner products bring the geometry of the plane back to infinite dimensions: the parallelogram law, orthogonal complements and the projection theorem, Bessel's inequality and Parseval's identity for orthonormal bases of $\ell^2$ and $L^2$, the Riesz representation of every bounded functional as an inner product, and the Hilbert-space adjoint.

A Banach space measures lengths but not angles. Sections 8.1 and 8.2 showed what can still be done with a norm alone, and also what goes missing: closest points need not exist or be unique, a functional can fail to attain its norm, and Hahn–Banach extensions are usually far from unique. Add an inner product and the geometry of Euclidean space returns almost completely. Every closed subspace has an orthogonal complement and an orthogonal projection; every vector is the sum of its "Fourier series" with respect to an orthonormal basis, and its length can be read off from its coefficients (Parseval); and every bounded linear functional is simply an inner product with a fixed vector (Riesz). These facts are the foundation of least-squares approximation, Fourier analysis, signal processing, the finite-element method and quantum mechanics, where states are unit vectors of a Hilbert space.

The model spaces are $\ell^2$ and $L^2[a,b]$; the main warning is that an inner-product space which is not complete — such as $C[0,1]$ with $\int_0^1 f\bar g$ — loses the projection theorem and the Riesz theorem. GATE asks for numbers that this geometry makes easy: the distance from a function to a subspace of polynomials, the norm of a functional on $L^2$ or $\ell^2$, the sum of a series via Parseval's identity. It also asks MSQs on which norms come from inner products, on orthogonal complements of non-closed subspaces, and on the properties of orthonormal sequences. Section 8.4 builds on everything here to diagonalise compact self-adjoint operators.

Inner Products and Hilbert Spaces

As in §8.1, $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$. Inner products are linear in the first variable and conjugate-linear in the second (physicists usually use the opposite convention); over $\mathbb{R}$ every bar below can be ignored.

📘 Definition 8.10 — Inner product, inner-product space, Hilbert space

An inner product on a vector space $H$ over $\mathbb{F}$ is a map $\langle\cdot,\cdot\rangle:H\times H\to\mathbb{F}$ such that for all $x,y,z\in H$ and $\alpha\in\mathbb{F}$:

Linearity$\langle\alpha x+y,z\rangle=\alpha\langle x,z\rangle+\langle y,z\rangle$.
Conjugate symmetry$\langle y,x\rangle=\overline{\langle x,y\rangle}$.
Positivity$\langle x,x\rangle\gt0$ for $x\ne0$.

Then $\langle x,\alpha y+z\rangle=\bar\alpha\langle x,y\rangle+\langle x,z\rangle$ and $\langle x,x\rangle$ is real. The induced norm is $\|x\|=\sqrt{\langle x,x\rangle}$ (a norm by Theorem 8.20). A vector space with an inner product is an inner-product space (or pre-Hilbert space); if it is complete in the induced norm it is a Hilbert space.

Hilbert spaces$\mathbb{F}^n$ with $\langle x,y\rangle=\sum_{k\le n}x_k\bar y_k$; $\ \ell^2$ with $\langle x,y\rangle=\sum_nx_n\bar y_n$ (absolutely convergent by Hölder's inequality, §8.1); $\ L^2[a,b]$ with $\langle f,g\rangle=\int_a^bf\bar g$. All complete by §8.1 (Riesz–Fischer for $L^2$).
Weighted productsOn $\mathbb{F}^n$, $\langle x,y\rangle_A=y^*Ax$ for a positive definite Hermitian matrix $A$ (§3.3); on $L^2$, $\int_a^bf\bar g\,w$ with a weight $w\ge c\gt0$.
Incomplete examples$C[a,b]$ with $\int_a^bf\bar g$; the polynomials with the same product; $c_{00}$ with the $\ell^2$ product. Each is dense in a Hilbert space ($L^2[a,b]$, $L^2[a,b]$, $\ell^2$).
📐 Theorem 8.20 — Cauchy–Schwarz inequality
In an inner-product space, $|\langle x,y\rangle|\le\|x\|\,\|y\|$ for all $x,y$, with equality if and only if $x$ and $y$ are linearly dependent. Consequently $\|\cdot\|$ is a norm, and the inner product is jointly continuous: $x_n\to x$ and $y_n\to y$ imply $\langle x_n,y_n\rangle\to\langle x,y\rangle$.

Proof. If $y=0$ both sides vanish. Otherwise, for every scalar $\lambda$, $$\begin{aligned}0\le\|x-\lambda y\|^2&=\|x\|^2-\bar\lambda\langle x,y\rangle\\&\quad-\lambda\langle y,x\rangle+|\lambda|^2\|y\|^2.\end{aligned}$$ Choose $\lambda=\langle x,y\rangle/\|y\|^2$: the three last terms become $-2|\langle x,y\rangle|^2/\|y\|^2+|\langle x,y\rangle|^2/\|y\|^2$, so $0\le\|x\|^2-|\langle x,y\rangle|^2/\|y\|^2$, which is the inequality. Equality holds iff $\|x-\lambda y\|=0$, i.e. $x=\lambda y$; and if $x$, $y$ are dependent with $y\ne0$, then $x=\lambda y$ for some $\lambda$ and equality is immediate. For the triangle inequality, $$\begin{aligned}\|x+y\|^2&=\|x\|^2+2\operatorname{Re}\langle x,y\rangle+\|y\|^2\\&\le(\|x\|+\|y\|)^2.\end{aligned}$$ The other norm axioms are immediate. Continuity: $|\langle x_n,y_n\rangle-\langle x,y\rangle|$ is at most $\|x_n-x\|\,\|y_n\|+\|x\|\,\|y_n-y\|\to0$, since $(\|y_n\|)$ is bounded. ∎

📐 Theorem 8.21 — Parallelogram law, polarization and the Jordan–von Neumann theorem
  1. Parallelogram law. In an inner-product space, $\|x+y\|^2+\|x-y\|^2=2\|x\|^2+2\|y\|^2$.
  2. Polarization. Over $\mathbb{R}$, $\langle x,y\rangle=\tfrac14\big(\|x+y\|^2-\|x-y\|^2\big)$; over $\mathbb{C}$, $\langle x,y\rangle=\tfrac14\sum_{k=0}^{3}i^k\|x+i^ky\|^2$.
  3. Jordan–von Neumann. A norm on a vector space is induced by an inner product if and only if it satisfies the parallelogram law; the inner product is then given by (2).

Proof. (1), (2). Expanding as in the proof of Theorem 8.20, $\|x+cy\|^2=\|x\|^2+2\operatorname{Re}\big(\bar c\langle x,y\rangle\big)+\|y\|^2$ for $|c|=1$. With $c=\pm1$ and adding we get (1); subtracting gives $4\operatorname{Re}\langle x,y\rangle$, which is the real case of (2). Over $\mathbb{C}$ put $z=\langle x,y\rangle$; the terms $\|x\|^2+\|y\|^2$ cancel because $\sum_ki^k=0$, and $$\begin{aligned}\sum_{k=0}^3i^k\cdot2\operatorname{Re}(i^{-k}z)&=2\big(\operatorname{Re}z+i\operatorname{Im}z\\&\qquad+\operatorname{Re}z+i\operatorname{Im}z\big)=4z.\end{aligned}$$

(3) Necessity is (1). For sufficiency (real case) define $\langle x,y\rangle=\tfrac14(\|x+y\|^2-\|x-y\|^2)$. It is symmetric, $\langle x,x\rangle=\|x\|^2$ and $\langle 0,y\rangle=0$. The key step is additivity in $x$. Apply the parallelogram law to the pairs $x+y,\ x'+y$ and $x-y,\ x'-y$: $$\begin{aligned}\|x+y\|^2+\|x'+y\|^2&=\tfrac12\|x+x'+2y\|^2\\&\quad+\tfrac12\|x-x'\|^2,\\ \|x-y\|^2+\|x'-y\|^2&=\tfrac12\|x+x'-2y\|^2\\&\quad+\tfrac12\|x-x'\|^2.\end{aligned}$$ Subtracting and dividing by $4$: $\langle x,y\rangle+\langle x',y\rangle=\tfrac12\langle x+x',2y\rangle$. Taking $x'=0$ gives $\langle w,y\rangle=\tfrac12\langle w,2y\rangle$ for every $w$; with $w=x+x'$ we obtain $\langle x,y\rangle+\langle x',y\rangle=\langle x+x',y\rangle$. Additivity gives $\langle rx,y\rangle=r\langle x,y\rangle$ for rational $r$, and both sides are continuous in $r$ (the norm is continuous), so this holds for all real $r$. In the complex case one defines $\langle x,y\rangle$ by the complex polarization formula, applies the real result to the real and imaginary parts, and checks $\langle ix,y\rangle=i\langle x,y\rangle$ directly. ∎

✏️ Example 1 — Which norms come from inner products? Prove or disprove
(a) For which $p\in[1,\infty]$ is the norm of $\ell^p$ induced by an inner product? (b) Is the sup norm of $C[0,1]$ induced by an inner product? (c) Show that $C[0,1]$ with $\langle f,g\rangle=\int_0^1f\bar g$ is an inner-product space but not a Hilbert space. (d) The norm $\|(x,y)\|=\sqrt{x^2+xy+y^2}$ on $\mathbb{R}^2$ comes from an inner product; find it.
(a) Only $p=2$ Test the parallelogram law on $e_1,e_2$: $\|e_1\pm e_2\|_p=2^{1/p}$ (and $1$ for $p=\infty$), so the left side is $2\cdot2^{2/p}$ while the right side is $4$. These agree iff $2/p=1$. For $p=2$ the norm is induced by $\sum x_n\bar y_n$. By Theorem 8.21(3) no other $\ell^p$ is an inner-product space — the same test works in $\ell^p_2=(\mathbb{F}^2,\|\cdot\|_p)$.
(b) No Take $f\equiv1$ and $g(t)=t$. Then $\|f+g\|_\infty=2$ and $\|f-g\|_\infty=1$, so the left side of the parallelogram law is $4+1=5$, whereas $2\|f\|^2+2\|g\|^2=4$.
(c) Incomplete The axioms are routine; positivity holds because a continuous $f$ with $\int|f|^2=0$ vanishes identically. Let $f_k=0$ on $[0,\tfrac12]$, $f_k=k(t-\tfrac12)$ on $[\tfrac12,\tfrac12+\tfrac1k]$ and $f_k=1$ after that. For $m\gt k$ the functions differ only on $[\tfrac12,\tfrac12+\tfrac1k]$, where $|f_k-f_m|\le1$, so $\|f_k-f_m\|_2^2\le\tfrac1k$: the sequence is Cauchy. If $f_k\to f$ with $f$ continuous, then $\int_0^{1/2}|f|^2=\lim\int_0^{1/2}|f_k-f|^2=0$, so $f=0$ on $[0,\tfrac12]$; likewise $f=1$ on $[\tfrac12+\delta,1]$ for every $\delta\gt0$. No continuous function does both. (In $L^2[0,1]$ the limit exists: it is the indicator of $(\tfrac12,1]$.)
(d) Polarize $\langle u,v\rangle=\tfrac14\big(\|u+v\|^2-\|u-v\|^2\big)$. With $Q(x,y)=x^2+xy+y^2$, the difference $Q(u+v)-Q(u-v)$ equals $4u_1v_1+2(u_1v_2+u_2v_1)+4u_2v_2$, so $$\langle u,v\rangle=u_1v_1+\tfrac12(u_1v_2+u_2v_1)+u_2v_2=v^TAu,$$ with $A=\begin{pmatrix}1&\frac12\\\frac12&1\end{pmatrix}$ positive definite ($\det A=\tfrac34\gt0$). So this norm is induced by an inner product. The general test on $\mathbb{R}^n$: $\|x\|^2$ must be a positive definite quadratic form.

Every inner-product space $X$ can be completed: there is a Hilbert space $H$ containing $X$ as a dense subspace, with the inner product extended by continuity. The completion of $C[a,b]$ (or of the polynomials) in $\|\cdot\|_2$ is $L^2[a,b]$, and that of $c_{00}$ is $\ell^2$.

Orthogonality and the Projection Theorem

📘 Definition 8.11 — Orthogonality and orthogonal complement
Vectors $x,y$ are orthogonal, $x\perp y$, if $\langle x,y\rangle=0$. For a subset $S\subseteq H$, the orthogonal complement is $S^\perp=\{x\in H:\langle x,s\rangle=0\text{ for all }s\in S\}$. Pythagoras: if $x\perp y$ then $\|x+y\|^2=\|x\|^2+\|y\|^2$, and for pairwise orthogonal $x_1,\dots,x_n$, $\|\sum x_k\|^2=\sum\|x_k\|^2$. For every $S$: $S^\perp$ is a closed subspace (it is $\bigcap_{s\in S}\ker\langle\cdot,s\rangle$, an intersection of kernels of continuous functionals); $S^\perp=(\operatorname{span}S)^\perp=\big(\overline{\operatorname{span}}\,S\big)^\perp$ (by continuity); $S\subseteq S^{\perp\perp}$; and $S\cap S^\perp\subseteq\{0\}$.
📐 Theorem 8.22 — Closest point in a closed convex set
Let $H$ be a Hilbert space and $K\subseteq H$ nonempty, closed and convex. For every $x\in H$ there is exactly one $k_0\in K$ with $$\|x-k_0\|=d:=\inf_{k\in K}\|x-k\|.$$ It is characterised by $\operatorname{Re}\langle x-k_0,\,k-k_0\rangle\le0$ for all $k\in K$ (the angle at $k_0$ between $x$ and any point of $K$ is at least $90^\circ$).

Proof. Choose $k_n\in K$ with $\|x-k_n\|\to d$. The parallelogram law for the vectors $x-k_n$ and $x-k_m$ gives $$\begin{aligned}\|k_n-k_m\|^2&=2\|x-k_n\|^2+2\|x-k_m\|^2\\&\quad-4\big\|x-\tfrac12(k_n+k_m)\big\|^2.\end{aligned}$$ By convexity $\tfrac12(k_n+k_m)\in K$, so the last norm is at least $d$, and the right-hand side is at most $2\|x-k_n\|^2+2\|x-k_m\|^2-4d^2\to0$. Thus $(k_n)$ is Cauchy; $H$ is complete and $K$ closed, so $k_n\to k_0\in K$, and $\|x-k_0\|=d$ by continuity of the norm. If $k_1\in K$ also has $\|x-k_1\|=d$, the same identity with $k_0,k_1$ gives $\|k_0-k_1\|^2\le2d^2+2d^2-4d^2=0$.

For the characterisation write $x-k=(x-k_0)-(k-k_0)$, so that $$\begin{aligned}\|x-k\|^2&=\|x-k_0\|^2-2\operatorname{Re}\langle x-k_0,k-k_0\rangle\\&\quad+\|k-k_0\|^2.\end{aligned}$$ If the real part is $\le0$ for all $k$, then $\|x-k\|\ge\|x-k_0\|$. Conversely, if $k_0$ is the closest point, apply the identity to $k_0+s(k-k_0)\in K$, $0\lt s\le1$: $0\le-2s\operatorname{Re}\langle x-k_0,k-k_0\rangle+s^2\|k-k_0\|^2$; divide by $s$ and let $s\to0$. ∎

The proof uses exactly two things: the parallelogram law (for uniqueness and for the Cauchy estimate) and completeness of $K$. In a Banach space closest points can fail to exist or fail to be unique — Interactive 1 below shows the second failure in the plane.

📐 Theorem 8.23 — Projection theorem
Let $M$ be a closed subspace of a Hilbert space $H$.
  1. Every $x\in H$ can be written uniquely as $x=m+n$ with $m\in M$, $n\in M^\perp$: $\ H=M\oplus M^\perp$. Here $m=P_Mx$ is the closest point of $M$ to $x$, characterised by $m\in M$ and $x-m\perp M$.
  2. The orthogonal projection $P=P_M$ is linear and bounded, $P^2=P$, $\langle Px,y\rangle=\langle x,Py\rangle$, $\|Px\|^2+\|x-Px\|^2=\|x\|^2$, and $\|P\|=1$ if $M\ne\{0\}$. Its range is $M$, its kernel is $M^\perp$, and $I-P=P_{M^\perp}$.
  3. $M^{\perp\perp}=M$. For an arbitrary subset $S$, $S^{\perp\perp}=\overline{\operatorname{span}}\,S$; in particular a subspace $N$ is dense in $H$ if and only if $N^\perp=\{0\}$.

Proof. (1) A subspace is convex, so Theorem 8.22 gives a unique closest point $m\in M$. For a subspace, $k-m$ runs through all of $M$ as $k$ does, so the characterisation reads $\operatorname{Re}\langle x-m,w\rangle\le0$ for all $w\in M$; replacing $w$ by $-w$ and by $iw$ shows $\langle x-m,w\rangle=0$, i.e. $x-m\in M^\perp$. Conversely, if $m\in M$ and $x-m\perp M$, then for $k\in M$ Pythagoras gives $\|x-k\|^2=\|x-m\|^2+\|m-k\|^2\ge\|x-m\|^2$, so $m$ is the closest point. Uniqueness of the decomposition: if $m+n=m'+n'$, then $m-m'=n'-n\in M\cap M^\perp=\{0\}$.

(2) If $x=Px+(x-Px)$ and $y=Py+(y-Py)$, then $\alpha x+y=(\alpha Px+Py)+\big(\alpha(x-Px)+(y-Py)\big)$ is a decomposition of the required kind, so by uniqueness $P(\alpha x+y)=\alpha Px+Py$. Pythagoras gives $\|Px\|^2+\|x-Px\|^2=\|x\|^2$, hence $\|Px\|\le\|x\|$, with equality for $x\in M$, where $Px=x$; so $\|P\|=1$ and $P^2=P$. Next, $\langle Px,y\rangle=\langle Px,Py\rangle+\langle Px,y-Py\rangle=\langle Px,Py\rangle$, and symmetrically $\langle x,Py\rangle=\langle Px,Py\rangle$. The statements about range and kernel follow from (1). Finally $x=(x-Px)+Px$ with $x-Px\in M^\perp$ and $Px\in M\subseteq(M^\perp)^\perp$; this is the decomposition of $x$ with respect to the closed subspace $M^\perp$, so $P_{M^\perp}x=x-Px$.

(3) Always $M\subseteq M^{\perp\perp}$. If $x\in M^{\perp\perp}$, write $x=m+n$ as in (1). Then $n=x-m\in M^{\perp\perp}$ (a subspace containing $M$) and $n\in M^\perp$, so $\langle n,n\rangle=0$. Hence $x=m\in M$. For a subset $S$, $S^\perp=(\overline{\operatorname{span}}\,S)^\perp$, so $S^{\perp\perp}=(\overline{\operatorname{span}}\,S)^{\perp\perp}=\overline{\operatorname{span}}\,S$. Finally $N^\perp=\{0\}$ iff $N^{\perp\perp}=H$ iff $\overline N=H$. ∎

✏️ Example 2 — Two distances, computed with the projection theorem
(a) In $L^2[0,1]$ find the polynomial $p$ of degree at most $1$ closest to $f(t)=t^2$, and the distance $\|f-p\|_2$. (b) In real $\ell^2$ let $M=\{x:\sum_nx_n/n=0\}$. Show that $M$ is closed, find $M^\perp$, the closest point of $M$ to $e_1$, and $\operatorname{dist}(e_1,M)$.
(a) Normal equations $p=a+bt$ is the projection iff $t^2-a-bt\perp1$ and $\perp t$: $$\tfrac13-a-\tfrac b2=0,\qquad\tfrac14-\tfrac a2-\tfrac b3=0.$$ Doubling the second gives $a+\tfrac{2b}3=\tfrac12$; subtracting the first, $\tfrac b6=\tfrac16$, so $b=1$, $a=-\tfrac16$: $\ p(t)=t-\tfrac16$.
(a) Distance Since $f-p\perp p$, $\|f-p\|^2=\langle f-p,f\rangle$, which is $\int_0^1(t^4-t^3+\tfrac16t^2)\,dt$ $=\tfrac15-\tfrac14+\tfrac1{18}=\tfrac1{180}$. So $\operatorname{dist}=\tfrac1{\sqrt{180}}=\tfrac1{6\sqrt5}\approx0.0745$ (checked symbolically).
(b) Closed, complement With $y=(1/n)\in\ell^2$, $M=\{y\}^\perp$, a closed subspace (Definition 8.11). By Theorem 8.23(3), $M^\perp=\{y\}^{\perp\perp}=\operatorname{span}\{y\}$.
(b) Closest point $P_{M^\perp}x=\dfrac{\langle x,y\rangle}{\|y\|^2}y$, so $P_Mx=x-\dfrac{\langle x,y\rangle}{\|y\|^2}y$ and $\operatorname{dist}(x,M)=\dfrac{|\langle x,y\rangle|}{\|y\|}$. Here $\|y\|^2=\sum\frac1{n^2}=\frac{\pi^2}6$ and $\langle e_1,y\rangle=1$: the closest point is $e_1-\frac6{\pi^2}\big(1,\tfrac12,\tfrac13,\dots\big)$ and $$\operatorname{dist}(e_1,M)=\frac{\sqrt6}{\pi}\approx0.7797.$$ This is the formula $\operatorname{dist}(x_0,\ker f)=|f(x_0)|/\|f\|$ of §8.1 — in a Hilbert space the distance is always attained.
✏️ Example 3 — Without completeness the projection theorem fails
Let $X=c_{00}$ with the inner product of $\ell^2$, $y=(1/n)$, and $M=\{x\in X:\sum_nx_n/n=0\}$. Prove: (a) $M$ is a closed subspace of $X$; (b) the orthogonal complement of $M$ in $X$ is $\{0\}$, so $X\ne M\oplus M^\perp$ and $M^{\perp\perp}\ne M$; (c) $e_1$ has no closest point in $M$; (d) $M$ contains an orthonormal sequence that is maximal in $X$ but for which Parseval's identity fails at $e_1$.
(a) $\phi(x)=\sum x_n/n=\langle x,y\rangle$ is bounded on $X$ by Cauchy–Schwarz, and $M=\ker\phi$, so $M$ is closed in $X$.
(b) $M$ is dense in $Y=\{y\}^\perp\subseteq\ell^2$ For $w\in Y$ let $P_kw=(w_1,\dots,w_k,0,\dots)$ and $s_k=\langle P_kw,y\rangle\to\langle w,y\rangle=0$. Then $m_k=P_kw-s_ke_1\in M$ (because $\langle e_1,y\rangle=1$) and $m_k\to w$. So if $z\in X$ is orthogonal to $M$, it is orthogonal to $Y$ (continuity), hence $z\in Y^\perp=\operatorname{span}\{y\}$ in $\ell^2$; but $cy\in c_{00}$ only for $c=0$. Thus $M^\perp=\{0\}$ in $X$, $M\oplus M^\perp=M\ne X$, and $M^{\perp\perp}=X$.
(c) Because $M$ is dense in $Y$, $\operatorname{dist}(e_1,M)=\operatorname{dist}(e_1,Y)=\sqrt6/\pi$ (Example 2(b)). In the Hilbert space $\ell^2$ the only point of $Y$ at this distance is $e_1-\frac6{\pi^2}y$ (Theorem 8.22), which has infinitely many nonzero terms and so is not in $M$. No $m\in M$ attains the distance.
(d) $M\subseteq c_{00}$ has a countable Hamel basis; Gram–Schmidt turns it into an orthonormal sequence $(u_n)$ with $\operatorname{span}\{u_n\}=M$. If $z\in X$ is orthogonal to every $u_n$, it is orthogonal to $M$, so $z=0$ by (b): $(u_n)$ is maximal in $X$. Yet $\sum_n|\langle e_1,u_n\rangle|^2=\|P_Ye_1\|^2$ $=1-\frac6{\pi^2}\approx0.392\lt1=\|e_1\|^2$, computed in $\ell^2$ (Theorem 8.25 below). In an incomplete space "maximal orthonormal set" does not imply Parseval — the equivalences of Theorem 8.26 need completeness.
Interactive 1 — Closest Points: Why the Parallelogram Law Matters

The point $x=(0.5,\,1.3)$ and a line $M$ through $0$ at angle $\theta$. The shaded ball of the $p$-norm centred at $x$ is grown until it touches $M$: its radius is $\operatorname{dist}_p(x,M)$ and the touching points are the closest points. Only for $p=2$ is the closest point always unique and $x-Px$ perpendicular to $M$. For $p=1$ and $p=\infty$ the ball has flat sides, and at $\theta=45^\circ,135^\circ$ (resp. $0^\circ,90^\circ$) a whole segment of closest points appears; for $p=1.5$ or $4$ the point is unique but the "error vector" is no longer perpendicular.

θ—
dist_p(x, M)—
closest point(s)—
angle(x − Px, M)—
🧭 Problem-solving strategy — distances and projections in exam problems
  1. Orthonormal basis of $M$ known ($e_1,\dots,e_n$): $P_Mx=\sum\langle x,e_k\rangle e_k$ and $\operatorname{dist}(x,M)^2=\|x\|^2-\sum|\langle x,e_k\rangle|^2$ (Theorem 8.24).
  2. Spanning set $x_1,\dots,x_n$ only: solve the normal equations $\sum_jc_j\langle x_j,x_i\rangle=\langle x,x_i\rangle$; then $\operatorname{dist}^2=\langle x-P_Mx,x\rangle$, or use the Gram determinants $\operatorname{dist}^2=\dfrac{G(x_1,\dots,x_n,x)}{G(x_1,\dots,x_n)}$ with $G=\det\big(\langle x_j,x_i\rangle\big)$. For Example 2(a): $\tfrac{1/2160}{1/12}=\tfrac1{180}$.
  3. Hyperplane $M=\{y\}^\perp$: $\operatorname{dist}(x,M)=|\langle x,y\rangle|/\|y\|$. Line $\operatorname{span}\{y\}$: $\operatorname{dist}^2=\|x\|^2-|\langle x,y\rangle|^2/\|y\|^2$.
  4. Complement trick: $\operatorname{dist}(x,M)=\|P_{M^\perp}x\|$ — if $M$ is cut out by finitely many conditions, project onto the small space $M^\perp$ instead.
  5. Symmetry on $[-a,a]$: even and odd functions are orthogonal; project the even and odd parts of $x$ separately. Legendre polynomials on $[-1,1]$ (and shifted Legendre on $[0,1]$) are ready-made orthogonal bases of polynomial subspaces.

Orthonormal Sets, Bessel, Parseval and Orthonormal Bases

📘 Definition 8.12 — Orthonormal set, Fourier coefficients, orthonormal basis
A family $\{e_\alpha\}_{\alpha\in A}$ in an inner-product space is orthonormal if $\langle e_\alpha,e_\beta\rangle=1$ for $\alpha=\beta$ and $0$ otherwise. The numbers $\langle x,e_\alpha\rangle$ are the Fourier coefficients of $x$. An orthonormal set in a Hilbert space $H$ is an orthonormal basis (or complete orthonormal system) if it is maximal: it is not contained in a strictly larger orthonormal set. Theorem 8.26 shows that this is the same as having dense linear span. An orthonormal basis is not a Hamel basis: vectors are infinite sums $\sum\langle x,e_\alpha\rangle e_\alpha$, not finite combinations.
📐 Theorem 8.24 — Best approximation and Bessel's inequality
Let $\{e_\alpha\}$ be orthonormal in an inner-product space and $x$ a vector.
  1. For finitely many $e_1,\dots,e_n$ and scalars $c_k$, $$\begin{aligned}\Big\|x-\sum_{k=1}^nc_ke_k\Big\|^2&=\|x\|^2-\sum_{k=1}^n|\langle x,e_k\rangle|^2\\&\quad+\sum_{k=1}^n|c_k-\langle x,e_k\rangle|^2.\end{aligned}$$ So the closest point of $\operatorname{span}\{e_1,\dots,e_n\}$ to $x$ is $\sum\langle x,e_k\rangle e_k$, uniquely.
  2. Bessel's inequality: $\sum_\alpha|\langle x,e_\alpha\rangle|^2\le\|x\|^2$. In particular only countably many Fourier coefficients of $x$ are nonzero, and $\langle x,e_n\rangle\to0$ for every orthonormal sequence $(e_n)$.

Proof. (1) Put $a_k=\langle x,e_k\rangle$. By orthonormality, $$\begin{aligned}\Big\|x-\sum c_ke_k\Big\|^2&=\|x\|^2-\sum\bar c_ka_k\\&\quad-\sum c_k\bar a_k+\sum|c_k|^2.\end{aligned}$$ Since $$|c_k-a_k|^2=|c_k|^2-c_k\bar a_k-\bar c_ka_k+|a_k|^2,$$ this equals $\|x\|^2-\sum|a_k|^2+\sum|c_k-a_k|^2$, which is smallest exactly when every $c_k=a_k$. (2) Taking $c_k=a_k$ in (1) gives $\sum_{k\le n}|a_k|^2\le\|x\|^2$ for every finite subfamily, so the sum over all $\alpha$ (the supremum of finite partial sums) is at most $\|x\|^2$. For each $m$, fewer than $m^2\|x\|^2+1$ indices have $|\langle x,e_\alpha\rangle|\gt\frac1m$; the union over $m$ of these finite sets is countable. For a sequence, the terms of a convergent series tend to $0$. ∎

📐 Theorem 8.25 — Orthonormal series (Riesz–Fischer)
Let $(e_n)$ be an orthonormal sequence in a Hilbert space $H$ and $(c_n)$ scalars. The series $\sum_nc_ne_n$ converges in $H$ if and only if $\sum_n|c_n|^2\lt\infty$, and then $$\Big\|\sum_nc_ne_n\Big\|^2=\sum_n|c_n|^2,$$ $$\Big\langle\sum_nc_ne_n,e_k\Big\rangle=c_k,$$ and the sum does not depend on the order of the terms. In particular, for every $x\in H$ the series $\sum_n\langle x,e_n\rangle e_n$ converges, and its sum is the orthogonal projection of $x$ onto $\overline{\operatorname{span}}\{e_n\}$.

Proof. For the partial sums $s_N=\sum_{n\le N}c_ne_n$ and $M\gt N$, Pythagoras gives $\|s_M-s_N\|^2=\sum_{N\lt n\le M}|c_n|^2$. So $(s_N)$ is Cauchy in $H$ iff the partial sums of $\sum|c_n|^2$ are Cauchy in $\mathbb{R}$; as $H$ is complete, Cauchy is the same as convergent. The formulas follow from $\|s_N\|^2=\sum_{n\le N}|c_n|^2$ and $\langle s_N,e_k\rangle=c_k$ ($N\ge k$) by continuity. For a rearrangement with partial sums $s'_N$: given $\varepsilon\gt0$ choose $K$ with $\sum_{n\gt K}|c_n|^2\lt\varepsilon$; once $N$ is so large that the first $N$ rearranged terms include $c_1e_1,\dots,c_Ke_K$, the vector $s'_N-s_K$ is a finite sum of terms with indices $\gt K$, so $\|s'_N-s_K\|^2\lt\varepsilon$, while $\|s-s_K\|^2\le\varepsilon$. Hence $\|s'_N-s\|\lt2\sqrt\varepsilon$, and the rearranged series also converges to $s$. Finally, by Bessel $\sum|\langle x,e_n\rangle|^2\lt\infty$, so $p=\sum\langle x,e_n\rangle e_n$ exists; it lies in $M=\overline{\operatorname{span}}\{e_n\}$ and $\langle x-p,e_k\rangle=0$ for all $k$, so $x-p\perp M$ and $p=P_Mx$ by Theorem 8.23(1). ∎

📐 Theorem 8.26 — Characterisations and existence of orthonormal bases
Let $\{e_\alpha\}_{\alpha\in A}$ be an orthonormal set in a Hilbert space $H$. The following are equivalent:
  1. $\{e_\alpha\}$ is maximal (an orthonormal basis);
  2. if $\langle x,e_\alpha\rangle=0$ for all $\alpha$, then $x=0$;
  3. the linear span of $\{e_\alpha\}$ is dense in $H$;
  4. $x=\sum_\alpha\langle x,e_\alpha\rangle e_\alpha$ for every $x\in H$;
  5. Parseval's identity: $\|x\|^2=\sum_\alpha|\langle x,e_\alpha\rangle|^2$ for every $x$;
  6. $\langle x,y\rangle=\sum_\alpha\langle x,e_\alpha\rangle\overline{\langle y,e_\alpha\rangle}$ for all $x,y$.
Moreover every Hilbert space $H\ne\{0\}$ has an orthonormal basis; $H$ is separable if and only if it has a countable orthonormal basis; and every infinite-dimensional separable Hilbert space is isometrically isomorphic to $\ell^2$ through $x\mapsto(\langle x,e_n\rangle)_n$.

Proof. (1)⇔(2): if $x\ne0$ is orthogonal to every $e_\alpha$, then adding $x/\|x\|$ gives a larger orthonormal set; conversely a strictly larger orthonormal set contains a unit vector orthogonal to every $e_\alpha$. (2)⇔(3): by Theorem 8.23(3), the span is dense iff its orthogonal complement — which is $\{e_\alpha\}^\perp$ — is $\{0\}$. (2)⇒(4): only countably many coefficients of $x$ are nonzero (Bessel); list them as a sequence and let $p=\sum\langle x,e_\alpha\rangle e_\alpha$ (Theorem 8.25). Then $x-p$ is orthogonal to every $e_\alpha$, so $x=p$. (4)⇒(6): apply the continuity of the inner product to the partial sums of the expansions of $x$ and $y$. (6)⇒(5): take $y=x$. (5)⇒(2): if all coefficients vanish, $\|x\|^2=0$.

Existence. Order the orthonormal subsets of $H$ by inclusion. The union of a chain is orthonormal (two of its vectors lie in a common member), so by Zorn's lemma there is a maximal orthonormal set. Separability. If $\{x_k\}$ is dense, discard each $x_k$ that is a combination of the earlier ones and apply Gram–Schmidt (§3.3) to the rest: this gives a countable orthonormal set whose span equals $\operatorname{span}\{x_k\}$, which is dense; it is a basis by (3). Conversely, finite combinations of a countable basis with coefficients in $\mathbb{Q}$ (or $\mathbb{Q}+i\mathbb{Q}$) form a countable dense set, by (4). Isomorphism. $Ux=(\langle x,e_n\rangle)$ maps $H$ into $\ell^2$ by Bessel, preserves norms and inner products by (5) and (6), and is onto by Theorem 8.25: $(c_n)\in\ell^2$ is $U\big(\sum c_ne_n\big)$. ∎

$\ell^2$The unit vectors $(e_n)$: $\langle x,e_n\rangle=x_n$, and Parseval is the definition of $\|x\|_2$.
$L^2[-\pi,\pi]$$\big\{e^{ikt}/\sqrt{2\pi}\big\}_{k\in\mathbb{Z}}$, or the real system $\frac1{\sqrt{2\pi}},\frac{\cos kt}{\sqrt\pi},\frac{\sin kt}{\sqrt\pi}$ ($k\ge1$). Complete because trigonometric polynomials are uniformly dense in the continuous periodic functions (Fejér's theorem, §8.2) and these are dense in $L^2$ (§4.3).
$L^2[0,1]$$\{\sqrt2\sin n\pi t\}_{n\ge1}$ and $\{1\}\cup\{\sqrt2\cos n\pi t\}_{n\ge1}$ — odd and even extensions to $[-1,1]$ (Example 5); re-proved by the spectral theorem in §8.4.
$L^2[-1,1]$The normalised Legendre polynomials $\sqrt{\tfrac{2n+1}2}P_n$, obtained by Gram–Schmidt from $1,t,t^2,\dots$; complete by the Weierstrass approximation theorem (§4.2).

For Fourier series it is convenient to write $\hat f(k)=\frac1{2\pi}\int_{-\pi}^{\pi}f(t)e^{-ikt}dt$, so that $\langle f,e^{ikt}/\sqrt{2\pi}\rangle=\sqrt{2\pi}\,\hat f(k)$ and Parseval reads $$\frac1{2\pi}\int_{-\pi}^{\pi}|f|^2=\sum_{k\in\mathbb{Z}}|\hat f(k)|^2.$$

✏️ Example 4 — Summing series with Parseval
(a) Compute the Fourier coefficients of $f(t)=t$ on $[-\pi,\pi]$ and deduce $\sum_{n\ge1}\frac1{n^2}=\frac{\pi^2}6$. (b) Do the same with $g(t)=t^2$ to obtain $\sum_{n\ge1}\frac1{n^4}=\frac{\pi^4}{90}$.
(a) Coefficients $\hat f(0)=0$ ($f$ is odd). For $n\ne0$, integrating by parts, $$\begin{aligned}\int_{-\pi}^{\pi}te^{-int}dt&=\Big[\frac{te^{-int}}{-in}\Big]_{-\pi}^{\pi}+\frac1{in}\int_{-\pi}^{\pi}e^{-int}dt\\&=\frac{2\pi(-1)^n}{-in},\end{aligned}$$ so $\hat f(n)=\dfrac{i(-1)^n}{n}$.
(a) Parseval $\frac1{2\pi}\int_{-\pi}^{\pi}t^2dt=\frac{\pi^2}3$ and $\sum_{n\ne0}|\hat f(n)|^2=2\sum_{n\ge1}\frac1{n^2}$. Hence $\sum\frac1{n^2}=\frac{\pi^2}6$.
(b) $\hat g(0)=\frac1{2\pi}\int t^2=\frac{\pi^2}3$, and two integrations by parts give $\hat g(n)=\frac{2(-1)^n}{n^2}$ for $n\ne0$. Parseval: $$\frac1{2\pi}\int_{-\pi}^{\pi}t^4dt=\frac{\pi^4}5=\frac{\pi^4}9+2\sum_{n\ge1}\frac4{n^4},$$ so $\sum\frac1{n^4}=\frac18\Big(\frac{\pi^4}5-\frac{\pi^4}9\Big)=\frac{\pi^4}{90}\approx1.0823$.
Check Partial sums: $\sum_{n\le10^5}n^{-2}\approx1.644924$ against $\pi^2/6\approx1.644934$; $\sum_{n\le100}n^{-4}\approx1.082323$ against $\pi^4/90\approx1.082323$. Parseval converts "energy of a function" into "sum of squares of its coefficients" — a standard GATE NAT device.
✏️ Example 5 — Orthonormal, but is it a basis? Prove or disprove
(a) Is $\{e_{2n}\}_{n\ge1}$ an orthonormal basis of $\ell^2$? (b) Is $\{\sin nt/\sqrt\pi\}_{n\ge1}$ an orthonormal basis of $L^2[-\pi,\pi]$? Is $\{\sqrt{2/\pi}\,\sin nt\}_{n\ge1}$ one of $L^2[0,\pi]$? (c) Show that an orthonormal sequence has no norm-convergent subsequence, but $\langle x,e_n\rangle\to0$ for every $x$. What does this say about the closed unit ball of an infinite-dimensional Hilbert space? (d) Is $(e_n)$ a Hamel basis of $\ell^2$?
(a) No $e_1$ is orthogonal to every $e_{2n}$ (criterion (2) of Theorem 8.26 fails); Bessel is strict at $x=e_1$: $0\lt1$.
(b) No on $[-\pi,\pi]$, yes on $[0,\pi]$ The constant $1$ (indeed every even function) is orthogonal to all $\sin nt$ on $[-\pi,\pi]$. On $[0,\pi]$: orthonormality is $\frac2\pi\int_0^\pi\sin mt\sin nt\,dt=\delta_{mn}$. If $f\in L^2[0,\pi]$ is orthogonal to every $\sin nt$, its odd extension $f_o$ to $[-\pi,\pi]$ satisfies $\int_{-\pi}^\pi f_o\sin nt=2\int_0^\pi f\sin nt=0$ and $\int_{-\pi}^{\pi}f_o\cos nt=0$ (odd times even). By completeness of the trigonometric system, $f_o=0$, so $f=0$.
(c) $\|e_n-e_m\|^2=\|e_n\|^2+\|e_m\|^2=2$ for $n\ne m$, so no subsequence is Cauchy. The $e_n$ lie in the closed unit ball, which is therefore not compact in infinite dimensions. But Bessel's inequality gives $\langle x,e_n\rangle\to0$ for every $x$: by Riesz (Theorem 8.27 below) $f(e_n)\to0$ for every $f\in H^*$, i.e. $e_n\to0$ weakly although $\|e_n\|=1$.
(d) No $(1,\tfrac12,\tfrac13,\dots)\in\ell^2$ has infinitely many nonzero coordinates, so it is not a finite combination of the $e_n$; by Baire's theorem (§8.2) every Hamel basis of $\ell^2$ is uncountable. Orthonormal bases are the useful notion of basis in Hilbert spaces.
Interactive 2 — Fourier Partial Sums: From Bessel to Parseval

Top: a function $f$ on $[-\pi,\pi]$ and its Fourier partial sum $S_Nf$, the orthogonal projection of $f$ onto the trigonometric polynomials of degree $\le N$. Bottom: the energy $\pi(a_k^2+b_k^2)$ of each harmonic as a fraction of $\|f\|^2=\int_{-\pi}^{\pi}|f|^2$; the bars up to $N$ are what $S_Nf$ has captured (Bessel), and they sum to $\|f\|^2$ as $N\to\infty$ (Parseval). Move $\delta$ to replace $S_Nf$ by $S_Nf+\delta\sin t$: the error grows by exactly $\pi\delta^2$ — the projection is the best approximation.

N—
‖f‖²—
‖S_N f‖² (Bessel sum)—
‖f − g‖², g = S_N f + δ sin t—

The Riesz Representation Theorem and Adjoint Operators

In §8.1 we found $(\ell^2)^*=\ell^2$ by a computation special to sequences. In a Hilbert space the projection theorem gives the same conclusion for every space at once.

📐 Theorem 8.27 — Riesz representation theorem (Fréchet–Riesz)
Let $H$ be a Hilbert space. For each $y\in H$, $f_y(x)=\langle x,y\rangle$ defines $f_y\in H^*$ with $\|f_y\|=\|y\|$. Conversely, for every $f\in H^*$ there is exactly one $y_f\in H$ with $$f(x)=\langle x,y_f\rangle\ \text{ for all }x\in H,\qquad\|f\|=\|y_f\|.$$ The map $y\mapsto f_y$ is a bijective isometry of $H$ onto $H^*$ which is additive and conjugate-linear ($f_{\alpha y}=\bar\alpha f_y$). With $\langle f_y,f_z\rangle:=\langle z,y\rangle$, $H^*$ is a Hilbert space, and $H$ is reflexive.

Proof. By Cauchy–Schwarz $|f_y(x)|\le\|x\|\,\|y\|$, with equality at $x=y$; so $\|f_y\|=\|y\|$. If $\langle x,y\rangle=\langle x,y'\rangle$ for all $x$, take $x=y-y'$ to get $y=y'$: the representing vector is unique. Existence. If $f=0$ take $y_f=0$. Otherwise $N=\ker f$ is a closed (because $f$ is continuous) proper subspace, so by the projection theorem $N^\perp\ne\{0\}$. Choose $z\in N^\perp$ with $f(z)=1$. For every $x$, $f\big(x-f(x)z\big)=0$, so $x-f(x)z\in N$ is orthogonal to $z$: $$0=\langle x-f(x)z,z\rangle=\langle x,z\rangle-f(x)\|z\|^2.$$ Hence $f(x)=\langle x,y_f\rangle$ with $y_f=z/\|z\|^2$. Conjugate-linearity: $\langle x,\alpha y\rangle=\bar\alpha\langle x,y\rangle$. With the stated inner product, $\langle\alpha f_y,f_z\rangle=\langle f_{\bar\alpha y},f_z\rangle=\langle z,\bar\alpha y\rangle=\alpha\langle f_y,f_z\rangle$, and $\langle f_y,f_y\rangle=\|y\|^2=\|f_y\|^2$, so $H^*$ is a Hilbert space. If $\varphi\in H^{**}$, Riesz in $H^*$ gives $f_z$ with $\varphi(f_y)=\langle f_y,f_z\rangle=\langle z,y\rangle=f_y(z)$ for all $y$: $\varphi$ is evaluation at $z$, so the canonical map $H\to H^{**}$ is onto. ∎

Two consequences answer questions left open in §8.1. In a Hilbert space a bounded functional $g$ on a subspace $M$ has a unique norm-preserving extension, namely $F=g\circ P_{\overline M}$ (Exercise IV.5) — the disc has a unique tangent at each boundary point. And every bounded functional on a Hilbert space attains its norm (at $y_f/\|y_f\|$), in contrast with $c_0$.

✏️ Example 6 — Finding the representing vector (or showing there is none)
(a) On $L^2[0,1]$, find $y_f$ and $\|f\|$ for $f(u)=\int_0^1t\,u(t)\,dt$. (b) On the space $\mathcal{P}_1$ of real polynomials of degree $\le1$ with $\langle p,q\rangle=\int_0^1pq$, find $y$ with $p(0)=\langle p,y\rangle$ for all $p$, and the norm of $p\mapsto p(0)$. (c) Show that $u\mapsto u(0)$ is unbounded on $C[0,1]$ with the norm $\|\cdot\|_2$. (d) On $X=c_{00}$ with the $\ell^2$ inner product, show that $f(x)=\sum_nx_n/n$ is bounded with $\|f\|=\pi/\sqrt6$ but has no representing vector in $X$.
(a) $f(u)=\langle u,y\rangle$ with $y(t)=t$ (real), so $\|f\|=\|t\|_2=\big(\int_0^1t^2\big)^{1/2}=\frac1{\sqrt3}\approx0.5774$, attained at $u=\sqrt3\,t$.
(b) Finite dimensions $\mathcal{P}_1$ is a two-dimensional Hilbert space. Write $y=a+bt$ and impose the identity on the basis $1,t$: $\int_0^1(a+bt)=a+\frac b2=1$ and $\int_0^1t(a+bt)=\frac a2+\frac b3=0$. Thus $a=4$, $b=-6$: $y=4-6t$, and $\|f\|=\|y\|_2=\big(\int_0^1(4-6t)^2\big)^{1/2}$ $=\sqrt{16-24+12}=2$. Check: $y(0)/\|y\|=4/2=2$.
(c) Unbounded $u_n(t)=\max(0,1-nt)$ has $u_n(0)=1$ and $\|u_n\|_2^2=\int_0^{1/n}(1-nt)^2dt=\frac1{3n}$, so $|u_n(0)|/\|u_n\|_2=\sqrt{3n}\to\infty$. An unbounded functional has no representing vector: point evaluation is not an element of $(L^2)^*$, and "$\delta_0$" is not a function.
(d) Completeness is needed $|f(x)|=|\langle x,y\rangle|\le\|x\|\,\|y\|$ with $y=(1/n)$, and the truncations $x^{(k)}=P_ky\in X$ give $f(x^{(k)})/\|x^{(k)}\|=\|P_ky\|\to\|y\|=\pi/\sqrt6\approx1.2825$. If $z\in X$ represented $f$, then $\bar z_n=\langle e_n,z\rangle=f(e_n)=\frac1n$ for every $n$, so $z$ would have infinitely many nonzero terms. The representer $y$ exists only in the completion $\ell^2$.
📘 Definition 8.13 — Adjoint; self-adjoint, normal and unitary operators
Let $H,K$ be Hilbert spaces and $T\in B(H,K)$. The adjoint of $T$ is the operator $T^*\in B(K,H)$ with $$\langle Tx,y\rangle_K=\langle x,T^*y\rangle_H\qquad(x\in H,\ y\in K)$$ (it exists and is unique by Theorem 8.28). An operator $T\in B(H)$ is self-adjoint (Hermitian) if $T^*=T$, normal if $T^*T=TT^*$, unitary if $T^*T=TT^*=I$, and an isometry if $\|Tx\|=\|x\|$ for all $x$ (equivalently $T^*T=I$). For a matrix acting on $\mathbb{F}^n$ with the standard inner product, $A^*$ is the conjugate transpose (§3.3).
📐 Theorem 8.28 — Properties of the adjoint
Let $S,T$ be bounded operators between Hilbert spaces (composable where needed) and $\alpha\in\mathbb{F}$.
  1. $T^*$ exists, is unique, linear and bounded.
  2. $(S+T)^*=S^*+T^*$, $(\alpha T)^*=\bar\alpha T^*$, $(ST)^*=T^*S^*$, $T^{**}=T$, $I^*=I$.
  3. $\|T^*\|=\|T\|$ and $\|T^*T\|=\|T\|^2$.
  4. $\ker T^*=(\operatorname{ran}T)^\perp$ and $\overline{\operatorname{ran}T}=(\ker T^*)^\perp$; likewise with $T$ and $T^*$ exchanged.
  5. An operator $P\in B(H)$ is an orthogonal projection (onto a closed subspace) if and only if $P^2=P=P^*$.

Proof. (1) Fix $y\in K$. The map $x\mapsto\langle Tx,y\rangle$ is linear with $|\langle Tx,y\rangle|\le\|T\|\,\|x\|\,\|y\|$, so by Theorem 8.27 there is a unique vector $T^*y\in H$ with $\langle Tx,y\rangle=\langle x,T^*y\rangle$ for all $x$, and $\|T^*y\|\le\|T\|\,\|y\|$. Linearity: $\langle x,T^*(\alpha y+y')\rangle=\langle Tx,\alpha y+y'\rangle$, which is $\bar\alpha\langle Tx,y\rangle+\langle Tx,y'\rangle=\langle x,\alpha T^*y+T^*y'\rangle$ for all $x$. (2) Each identity follows from uniqueness, e.g. $\langle STx,y\rangle=\langle Tx,S^*y\rangle=\langle x,T^*S^*y\rangle$, and $\langle T^*y,x\rangle=\overline{\langle x,T^*y\rangle}=\overline{\langle Tx,y\rangle}=\langle y,Tx\rangle$ shows $T^{**}=T$. (3) From (1), $\|T^*\|\le\|T\|$; applied to $T^*$ it gives $\|T\|=\|T^{**}\|\le\|T^*\|$. Also $\|T^*T\|\le\|T^*\|\,\|T\|=\|T\|^2$, while $\|Tx\|^2=\langle T^*Tx,x\rangle\le\|T^*T\|\,\|x\|^2$ gives $\|T\|^2\le\|T^*T\|$. (4) $y\in\ker T^*$ iff $\langle x,T^*y\rangle=0$ for all $x$ iff $\langle Tx,y\rangle=0$ for all $x$ iff $y\perp\operatorname{ran}T$. Taking complements and using Theorem 8.23(3) gives the second identity. (5) Orthogonal projections satisfy $P^2=P=P^*$ by Theorem 8.23(2). Conversely let $P^2=P=P^*$ and $M=\operatorname{ran}P=\ker(I-P)$, a closed subspace. For every $x$, $Px\in M$, and for $m=Pz\in M$, $\langle x-Px,Pz\rangle=\langle P(x-Px),z\rangle$ $=\langle Px-P^2x,z\rangle=0$. So $x-Px\perp M$, and $Px=P_Mx$ by Theorem 8.23(1). ∎

✏️ Example 7 — Adjoints of the standard operators
Find the adjoints of (a) the right shift $S(x_1,x_2,\dots)=(0,x_1,x_2,\dots)$ on $\ell^2$, and compute $S^*S$ and $SS^*$; (b) a multiplication operator $M_\varphi u=\varphi u$ on $L^2[a,b]$, $\varphi$ bounded and measurable; (c) an integral operator $(T_Ku)(x)=\int_a^bK(x,t)u(t)\,dt$ on $L^2[a,b]$ with $K\in L^2([a,b]^2)$, in particular the Volterra operator $(Vu)(x)=\int_0^xu(t)\,dt$ on $L^2[0,1]$. Which are self-adjoint?
(a) Shifts $\langle Sx,y\rangle=\sum_{n\ge1}x_n\bar y_{n+1}=\langle x,Ly\rangle$ with the left shift $Ly=(y_2,y_3,\dots)$; so $S^*=L$. Then $S^*S=LS=I$ ($S$ is an isometry), but $SS^*x=SLx=(0,x_2,x_3,\dots)=x-\langle x,e_1\rangle e_1$, the projection onto $\{e_1\}^\perp$. So $S$ is an isometry that is neither unitary nor normal, and not self-adjoint.
(b) Multiplication $\langle\varphi u,v\rangle=\int\varphi u\bar v=\int u\,\overline{\bar\varphi v}=\langle u,\bar\varphi v\rangle$, so $M_\varphi^*=M_{\bar\varphi}$. Every $M_\varphi$ is normal; it is self-adjoint iff $\varphi$ is real a.e. and unitary iff $|\varphi|=1$ a.e. (For example $M_t$ on $L^2[0,1]$ is self-adjoint with $\|M_t\|=1$.)
(c) Kernels By Fubini, $$\begin{aligned}\langle T_Ku,v\rangle&=\int\!\!\int K(x,t)u(t)\overline{v(x)}\,dt\,dx\\&=\int u(t)\,\overline{\int\overline{K(x,t)}\,v(x)\,dx}\,dt.\end{aligned}$$ So $T_K^*=T_{K^*}$ with $K^*(x,t)=\overline{K(t,x)}$: transpose and conjugate, as for matrices. $T_K$ is self-adjoint iff $K(x,t)=\overline{K(t,x)}$ a.e. (a Hermitian kernel; for real kernels, symmetric).
(c) Volterra $V=T_K$ with $K(x,t)=1$ for $t\le x$ and $0$ otherwise; $K^*(x,t)=1$ for $x\le t$, so $(V^*v)(x)=\int_x^1v(t)\,dt$. $V$ is not self-adjoint, but $(V+V^*)u=\int_0^1u$ is the rank-one operator $u\mapsto\langle u,1\rangle1$ (Exercise IV.7). Boundedness of $T_K$ ($\|T_K\|\le\|K\|_{L^2}$) is proved in §8.4.
⚠️ Counterexamples and pitfalls
  1. Most norms are not inner-product norms. $\ell^p$ and $L^p$ for $p\ne2$, $c_0$, $\ell^\infty$ and $(C[a,b],\|\cdot\|_\infty)$ all fail the parallelogram law (test $e_1,e_2$, or $1$ and $t$).
  2. Inner-product space ≠ Hilbert space. $C[0,1]$ with $\int f\bar g$, the polynomials, and $c_{00}\subseteq\ell^2$ are incomplete.
  3. The projection theorem needs a closed subspace of a complete space. If $M$ is dense but not closed (polynomials in $L^2$, $c_{00}$ in $\ell^2$), then $M^\perp=\{0\}$ and $M\oplus M^\perp=M\ne H$; always $M^{\perp\perp}=\overline M$, not $M$. In the incomplete space $c_{00}$, even a closed subspace can have $M^\perp=\{0\}$ (Example 3).
  4. Closest points in Banach spaces may be non-unique — $(\mathbb{R}^2,\|\cdot\|_\infty)$, the $x$-axis and $(0,1)$: every $(t,0)$ with $|t|\le1$ — or may not exist: in $c_0$ no point of $\ker f$, $f(x)=\sum x_n2^{-n}$, is closest to $e_1$, because $f$ does not attain its norm (§8.1).
  5. Orthonormal basis ≠ Hamel basis; a non-separable Hilbert space such as $\ell^2(\mathbb{R})$ has uncountable orthonormal bases, and "basis" never means finite combinations.
  6. Orthonormal sequences converge weakly to $0$ but are never norm-Cauchy ($\|e_n-e_m\|=\sqrt2$); the closed unit ball of an infinite-dimensional Hilbert space is not compact.
  7. Bessel is not Parseval. Equality for every $x$ characterises orthonormal bases; $\{\sin nt\}$ on $[-\pi,\pi]$ and $\{e_{2n}\}$ in $\ell^2$ are orthonormal but incomplete. And in an incomplete space a maximal orthonormal set can violate Parseval.
  8. Riesz needs completeness and boundedness. $f(x)=\sum x_n/n$ on $c_{00}$ is bounded with no representer in $c_{00}$; $u\mapsto u(0)$ on $(C[0,1],\|\cdot\|_2)$ is unbounded and represented by nothing.
  9. The Riesz map is conjugate-linear over $\mathbb{C}$: $y_{\alpha f}=\bar\alpha y_f$.
  10. Adjoints. $\|T^*\|=\|T\|$ but $T^*\ne T$ in general; $S^*S=I$ does not imply $SS^*=I$ (the shift); a product of self-adjoint operators is self-adjoint only if they commute.

Exercises

I
Inner products, norms and completeness.
  • 1 Which of the following define inner products on $\mathbb{R}^2$? (a) $x_1y_1+2x_2y_2-x_1y_2-x_2y_1$; (b) $x_1y_1-x_2y_2$; (c) $x_1y_2+x_2y_1$; (d) $3x_1y_1+x_1y_2+x_2y_1+x_2y_2$.
  • 2 Prove that in a real inner-product space $x\perp y$ iff $\|x+y\|=\|x-y\|$ iff $\|x+y\|^2=\|x\|^2+\|y\|^2$. Show by an example in $\mathbb{C}$ that the last equivalence fails over $\mathbb{C}$.
  • 3 Using $f=\mathbf 1_{[0,1/2]}$ and $g=\mathbf 1_{(1/2,1]}$, show that the norm of $L^p[0,1]$ is induced by an inner product only for $p=2$.
  • 4 Show that the space $\mathcal{P}$ of all polynomials with $\langle p,q\rangle=\int_0^1p\bar q$ is an inner-product space that is not a Hilbert space.
  • 5 Let $D$ be a dense subset of an inner-product space $X$. Prove that if $\langle x,z\rangle=\langle y,z\rangle$ for all $z\in D$, then $x=y$.
  • 6 Prove that $\|x+y\|=\|x\|+\|y\|$ in an inner-product space iff one of $x,y$ is a non-negative multiple of the other. Show that this fails in $\ell^1$.
  • 7 Prove Apollonius' identity: $\|z-x\|^2+\|z-y\|^2$ $=\tfrac12\|x-y\|^2+2\big\|z-\tfrac12(x+y)\big\|^2$.
  • 8 On $C^1[0,1]$ let $\langle u,v\rangle=\int_0^1(uv+u'v')$. Show this is an inner product and that the space is not complete. (Hint: $u_n(x)=\sqrt{(x-\frac12)^2+\frac1{n^2}}$.)
1 A bilinear form $y^TAx$ is an inner product iff $A$ is symmetric positive definite. (a) $A=\begin{pmatrix}1&-1\\-1&2\end{pmatrix}$, minors $1$ and $1$: yes. (b) $\langle e_2,e_2\rangle=-1$: no. (c) $\langle e_1,e_1\rangle=0$: no. (d) $A=\begin{pmatrix}3&1\\1&1\end{pmatrix}$, minors $3$ and $2$: yes.
2 $\|x\pm y\|^2=\|x\|^2\pm2\langle x,y\rangle+\|y\|^2$ in the real case, so each condition is equivalent to $\langle x,y\rangle=0$. Over $\mathbb{C}$ the middle term is $2\operatorname{Re}\langle x,y\rangle$: for $x=1$, $y=i$ in $\mathbb{C}$, $|1+i|^2=2=|1|^2+|i|^2$ but $\langle1,i\rangle=\bar i=-i\ne0$.
3 $\|f\|_p=\|g\|_p=2^{-1/p}$ and $|f\pm g|=1$ on $[0,1]$, so $\|f\pm g\|_p=1$. The parallelogram law would need $2=4\cdot2^{-2/p}$, i.e. $2^{1-2/p}=1$, i.e. $p=2$ (for $p=\infty$: $2\ne4$).
4 The axioms hold (a polynomial with $\int_0^1|p|^2=0$ vanishes on $[0,1]$, hence identically). $\mathcal{P}$ has the countable Hamel basis $1,t,t^2,\dots$, and no infinite-dimensional Banach space has one (Baire, §8.2). Concretely, the Taylor polynomials of $e^t$ are $\|\cdot\|_2$-Cauchy with no polynomial limit.
5 The map $z\mapsto\langle x-y,z\rangle$ is continuous and vanishes on $D$, hence on $\overline D=X$; take $z=x-y$.
6 Squaring, equality means $\operatorname{Re}\langle x,y\rangle=\|x\|\,\|y\|$, so $|\langle x,y\rangle|=\|x\|\,\|y\|$ and $\langle x,y\rangle\ge0$. By Theorem 8.20, $x=cy$ (or $y=0$) and then $c\|y\|^2=\langle x,y\rangle\ge0$. In $\ell^1$: $\|e_1+e_2\|_1=2=\|e_1\|_1+\|e_2\|_1$ although $e_1,e_2$ are independent — the $\ell^1$ ball has flat faces.
7 Apply the parallelogram law to $u=z-x$, $v=z-y$: $u+v=2\big(z-\tfrac12(x+y)\big)$ and $u-v=y-x$, so $4\|z-\tfrac{x+y}2\|^2+\|x-y\|^2$ $=2\|z-x\|^2+2\|z-y\|^2$; divide by $2$.
8 Positivity: $\int(u^2+u'^2)=0$ forces $u=0$. The $u_n$ are $C^1$, converge uniformly to $|x-\tfrac12|$, and $u_n'(x)=(x-\tfrac12)/u_n(x)\to\operatorname{sgn}(x-\tfrac12)$ boundedly, so by dominated convergence $(u_n)$ and $(u_n')$ are Cauchy in $L^2$: $(u_n)$ is Cauchy for the new norm. A limit $u\in C^1$ would satisfy $\|u_n-u\|_2\to0$, so $u=|x-\tfrac12|$, which is not differentiable at $\tfrac12$. (The completion is the Sobolev space $H^1(0,1)$.)
II
Orthogonal complements, projections and distances.
  • 1 In $\mathbb{R}^3$ find the orthogonal projection of $(1,2,3)$ onto the plane $x+y+z=0$, and its distance from the plane.
  • 2 In real $L^2[-1,1]$ let $M$ be the subspace of even functions. Show that $M^\perp$ is the subspace of odd functions, find $P_Me^t$ and $\operatorname{dist}(e^t,M)$.
  • 3 In real $\ell^2$, find $\operatorname{dist}(e_1,M)$ for $M=\{x:\sum_nx_n2^{-n}=0\}$.
  • 4 Let $M=\{f\in L^2[0,1]:\int_0^1f=0\}$. Show $M$ is closed, find $M^\perp$, $P_M$, and $\operatorname{dist}(t,M)$.
  • 5 Let $P,Q$ be the orthogonal projections onto closed subspaces $M,N$. Prove that $PQ$ is an orthogonal projection iff $PQ=QP$, and that it is then the projection onto $M\cap N$.
  • 6 Let $M=\{f\in L^2[0,1]:f=0\text{ a.e. on }[0,\tfrac12]\}$. Show $M$ is closed and find $M^\perp$ and $P_M$.
  • 7 Give a proper dense subspace $N$ of $L^2[0,1]$, and find $N^\perp$ and $N^{\perp\perp}$.
  • 8 Find the polynomial $q$ of degree $\le2$ minimising $\int_{-1}^1\big(|t|-q(t)\big)^2dt$, and the minimum distance.
1 Normal $n=(1,1,1)$: $Px=x-\frac{\langle x,n\rangle}{\|n\|^2}n$ $=(1,2,3)-2(1,1,1)=(-1,0,1)$; distance $\frac{|\langle x,n\rangle|}{\|n\|}=\frac6{\sqrt3}=2\sqrt3\approx3.464$.
2 Each $f$ is $f_e+f_o$ with $f_e(t)=\frac{f(t)+f(-t)}2$ even and $f_o$ odd, and $\int_{-1}^1f_ef_o=0$ (odd integrand). So $L^2=M\oplus N$ with $N$ = odd functions $\subseteq M^\perp$; if $g\in M^\perp$ then $g_e=g-g_o\in M\cap M^\perp=\{0\}$, so $M^\perp=N$. $P_Me^t=\cosh t$ and $\operatorname{dist}=\|\sinh t\|_2=\big(\tfrac{\sinh2}2-1\big)^{1/2}\approx0.9019$.
3 $M=\{y\}^\perp$ with $y=(2^{-n})$, $\|y\|^2=\sum4^{-n}=\tfrac13$, $\langle e_1,y\rangle=\tfrac12$: $\operatorname{dist}=\frac{1/2}{1/\sqrt3}=\frac{\sqrt3}2\approx0.8660$.
4 $M=\{1\}^\perp$, closed; $M^\perp=\operatorname{span}\{1\}$, the constants; $P_Mf=f-\int_0^1f$. $\operatorname{dist}(t,M)=\frac{|\langle t,1\rangle|}{\|1\|}=\frac12$.
5 $(PQ)^*=Q^*P^*=QP$. If $PQ=QP$, then $(PQ)^2=PQPQ=P^2Q^2=PQ$ and $(PQ)^*=PQ$, so $PQ$ is an orthogonal projection (Theorem 8.28(5)). Its range lies in $\operatorname{ran}P=M$ and, as $PQ=QP$, in $N$; and $PQx=x$ for $x\in M\cap N$. So it projects onto $M\cap N$. Conversely, if $PQ$ is an orthogonal projection it is self-adjoint: $PQ=(PQ)^*=QP$.
6 If $f_n\in M$ and $f_n\to f$ in $L^2$, then $\int_0^{1/2}|f|^2=\lim\int_0^{1/2}|f-f_n|^2=0$, so $f\in M$. $M^\perp=\{g:g=0\text{ a.e. on }(\tfrac12,1]\}$ and $P_Mf=f\,\mathbf 1_{(1/2,1]}$, because $f=f\mathbf 1_{(1/2,1]}+f\mathbf 1_{[0,1/2]}$ with the two pieces orthogonal.
7 $N=\mathcal{P}$, the polynomials: dense (Weierstrass gives uniform, hence $L^2$, approximation of continuous functions, which are dense in $L^2$), proper ($\mathbf 1_{[0,1/2]}$ is not a.e. equal to a polynomial). $N^\perp=\{0\}$ and $N^{\perp\perp}=L^2[0,1]\ne N$.
8 $|t|$ is even, so only $P_0=1$ and $P_2=\frac{3t^2-1}2$ contribute: $\frac{\langle|t|,1\rangle}{\|1\|^2}=\frac12$ and $\frac{\langle|t|,P_2\rangle}{\|P_2\|^2}=\frac{1/4}{2/5}=\frac58$. So $q=\frac12+\frac58P_2=\frac{15t^2+3}{16}$, and $\operatorname{dist}^2=\frac23-\frac14\cdot2-\frac{25}{64}\cdot\frac25=\frac1{96}$: $\operatorname{dist}\approx0.1021$ (checked symbolically).
III
Orthonormal systems, Bessel and Parseval.
  • 1 Verify that $\{e^{ikt}/\sqrt{2\pi}\}_{k\in\mathbb{Z}}$ is orthonormal in $L^2[-\pi,\pi]$ and $\{\sqrt2\sin n\pi t\}_{n\ge1}$ is orthonormal in $L^2[0,1]$.
  • 2 Expand $f(t)=t$ in the basis $\{\sqrt2\sin n\pi t\}$ of $L^2[0,1]$ and use Parseval to show again that $\sum1/n^2=\pi^2/6$.
  • 3 Use the square wave $\operatorname{sgn}t$ on $[-\pi,\pi]$ to prove $\sum_{k\ge1}\frac1{(2k-1)^2}=\frac{\pi^2}8$.
  • 4 Apply Gram–Schmidt to $1,t,t^2$ in $L^2[0,1]$.
  • 5 Prove that every orthonormal set in a separable Hilbert space is countable.
  • 6 Prove that $\int_{-\pi}^{\pi}f(t)\cos nt\,dt\to0$ as $n\to\infty$ for every $f\in L^2[-\pi,\pi]$.
  • 7 Let $(e_n)$ be an orthonormal basis and $(f_n)$ an orthonormal sequence with $\sum_n\|e_n-f_n\|^2\lt1$. Prove that $(f_n)$ is an orthonormal basis.
  • 8 The vectors $e_n-e_{n+1}$ ($n\ge1$) are not orthogonal. Show that nevertheless their linear span is dense in $\ell^2$.
1 $\frac1{2\pi}\int_{-\pi}^{\pi}e^{i(k-l)t}dt$ is $1$ for $k=l$ and $\frac{e^{i(k-l)\pi}-e^{-i(k-l)\pi}}{2\pi i(k-l)}=0$ otherwise. And $$\begin{aligned}&2\int_0^1\sin m\pi t\sin n\pi t\,dt\\&=\int_0^1\big(\cos(m-n)\pi t-\cos(m+n)\pi t\big)dt\\&=\delta_{mn}.\end{aligned}$$
2 $\langle t,\sqrt2\sin n\pi t\rangle=\sqrt2\int_0^1t\sin n\pi t\,dt=\frac{\sqrt2(-1)^{n+1}}{n\pi}$. Parseval: $\int_0^1t^2=\frac13=\sum_n\frac2{n^2\pi^2}$, so $\sum\frac1{n^2}=\frac{\pi^2}6$.
3 $\widehat{\operatorname{sgn}}(k)=\frac1{2\pi}\int_0^\pi(e^{-ikt}-e^{ikt})dt=-\frac{i}{\pi}\cdot\frac{1-(-1)^k}{k}$, i.e. $-\frac{2i}{\pi k}$ for odd $k$ and $0$ for even $k$. Parseval: $1=\frac1{2\pi}\int|\operatorname{sgn}|^2=2\sum_{k\text{ odd}\gt0}\frac4{\pi^2k^2}$, so $\sum_{k\text{ odd}}\frac1{k^2}=\frac{\pi^2}8$.
4 $e_0=1$; $t-\langle t,1\rangle=t-\frac12$ with norm $\frac1{\sqrt{12}}$, so $e_1=\sqrt3(2t-1)$; $t^2-\frac13-\frac{\sqrt3}6e_1=t^2-t+\frac16$ with norm $\frac1{6\sqrt5}$ (Example 2(a)), so $e_2=\sqrt5(6t^2-6t+1)$ — the shifted Legendre polynomials (orthonormality checked symbolically).
5 Distinct $e_\alpha,e_\beta$ satisfy $\|e_\alpha-e_\beta\|=\sqrt2$, so the balls $B(e_\alpha,\tfrac{\sqrt2}2)$ are pairwise disjoint. Each contains a point of a fixed countable dense set, and different balls contain different points.
6 $\{\cos nt/\sqrt\pi\}_{n\ge1}$ is orthonormal, so by Bessel $\sum_n\big|\int f\cos nt\big|^2\le\pi\|f\|^2\lt\infty$ and the terms tend to $0$ (the $L^2$ Riemann–Lebesgue lemma).
7 If $x\perp f_n$ for all $n$ and $x\ne0$, Parseval for $(e_n)$ and Cauchy–Schwarz give $$\begin{aligned}\|x\|^2&=\sum|\langle x,e_n\rangle|^2=\sum|\langle x,e_n-f_n\rangle|^2\\&\le\|x\|^2\sum\|e_n-f_n\|^2\lt\|x\|^2,\end{aligned}$$ a contradiction. So $(f_n)$ satisfies criterion (2) of Theorem 8.26.
8 If $x\perp(e_n-e_{n+1})$ for all $n$, then $x_n=x_{n+1}$ for all $n$, so $x$ is a constant sequence; in $\ell^2$ that forces $x=0$. By Theorem 8.23(3) the span is dense. (It is a proper subspace: $e_1$ is not a finite combination of the $e_n-e_{n+1}$, since any such combination has coordinate sum $0$.)
IV
Riesz representation and adjoints.
  • 1 Find the representing vector and the norm of $f(u)=\int_0^1e^tu(t)\,dt$ on $L^2[0,1]$.
  • 2 Find the norm of $f(x)=\sum_{n\ge1}x_n/n$ on $\ell^2$ and a unit vector at which it is attained.
  • 3 On $\mathcal{P}_1$ with $\langle p,q\rangle=\int_{-1}^1pq$, find $y$ with $p(1)=\langle p,y\rangle$ for all $p$, and the norm of $p\mapsto p(1)$.
  • 4 Show that $u\mapsto u(\tfrac12)$ is not bounded on $(C[0,1],\|\cdot\|_2)$.
  • 5 Let $M$ be a subspace of a Hilbert space $H$ and $g\in M^*$. Prove that $g$ has exactly one extension $F\in H^*$ with $\|F\|=\|g\|$.
  • 6 Find the adjoint of the weighted shift $Tx=(0,x_1,\tfrac{x_2}2,\tfrac{x_3}3,\dots)$ on $\ell^2$, and $\|T\|$.
  • 7 For the Volterra operator $V$ on $L^2[0,1]$, show that $V+V^*$ is the rank-one operator $u\mapsto\langle u,1\rangle1$.
  • 8 Let $U\in B(H)$ be an isometry. Prove that $U^*U=I$ and that $UU^*$ is the orthogonal projection onto $\operatorname{ran}U$. When is $U$ unitary?
1 $y_f=e^t$ and $\|f\|=\big(\int_0^1e^{2t}dt\big)^{1/2}=\sqrt{\tfrac{e^2-1}2}\approx1.7873$.
2 $y=(1/n)$, $\|f\|=\|y\|=\pi/\sqrt6\approx1.2825$, attained at $x=y/\|y\|=\big(\tfrac{\sqrt6}{n\pi}\big)_n$.
3 An orthonormal basis of $\mathcal{P}_1$ is $\frac1{\sqrt2},\sqrt{\tfrac32}t$; then $y=\sum f(e_i)e_i=\frac12+\frac32t$, and $\|f\|^2=\sum|f(e_i)|^2=\frac12+\frac32=2$, so $\|f\|=\sqrt2$. Check: $\int_{-1}^1(\tfrac12+\tfrac32t)\,dt=1=p(1)$ for $p=1$, and $\int_{-1}^1t(\tfrac12+\tfrac32t)\,dt=1$ for $p=t$.
4 Tents $u_n$ of height $1$ at $\tfrac12$ with base $[\tfrac12-\tfrac1n,\tfrac12+\tfrac1n]$: $\|u_n\|_2^2=\frac2{3n}\to0$ while $u_n(\tfrac12)=1$.
5 $g$ extends uniquely and isometrically to $\overline M$ (by uniform continuity), so assume $M$ closed. Existence: $F=g\circ P_M$ has $\|F\|\le\|g\|$. Uniqueness: let $F$ be any norm-preserving extension with representer $y_F$. For $m\in M$, $g(m)=\langle m,y_F\rangle=\langle m,P_My_F\rangle$, so $P_My_F$ represents $g$ in the Hilbert space $M$ and $\|g\|=\|P_My_F\|$. Then $\|y_F\|^2=\|P_My_F\|^2+\|(I-P_M)y_F\|^2$ and $\|y_F\|=\|F\|=\|g\|$ force $(I-P_M)y_F=0$: $y_F$ is determined by $g$.
6 $(Tx)_{n+1}=x_n/n$, so $\langle Tx,y\rangle=\sum_nx_n\bar y_{n+1}/n=\langle x,T^*y\rangle$ with $T^*y=(y_2,\tfrac{y_3}2,\tfrac{y_4}3,\dots)$. $\|Tx\|^2=\sum|x_n|^2/n^2\le\|x\|^2$ with equality at $e_1$: $\|T\|=1$.
7 By Example 7(c), $(V+V^*)u(x)=\int_0^xu+\int_x^1u$, which is $\int_0^1u=\langle u,1\rangle\cdot1$.
8 $\langle U^*Ux,x\rangle=\|Ux\|^2=\|x\|^2$, so the self-adjoint operator $A=U^*U-I$ has $\langle Ax,x\rangle=0$ for all $x$. Expanding, $\langle A(x+y),x+y\rangle-\langle A(x-y),x-y\rangle$ equals $2\langle Ax,y\rangle+2\langle Ay,x\rangle=4\operatorname{Re}\langle Ax,y\rangle$, so $\operatorname{Re}\langle Ax,y\rangle=0$ for all $x,y$; with $y=Ax$ this gives $\|Ax\|^2=0$. Hence $A=0$. Then $Q=UU^*$ satisfies $Q^2=U(U^*U)U^*=Q=Q^*$, an orthogonal projection, with $\operatorname{ran}Q\subseteq\operatorname{ran}U$ and $QUx=Ux$; so it projects onto $\operatorname{ran}U$ (closed, being isometric image of a complete space). $U$ is unitary iff it is onto.
🎯 GATE corner GATE MA · MCQGATE MA · MSQGATE MA · NAT

E1. GATE MA · MCQ Which one of the following norms on $\mathbb{R}^2$ is induced by an inner product? (A) $\|(x,y)\|=|x|+2|y|$ (B) $\|(x,y)\|=\max(|x|,|y|)$ (C) $\|(x,y)\|=\sqrt{2x^2-2xy+y^2}$ (D) $\|(x,y)\|=(x^4+y^4)^{1/4}$

Idea Jordan–von Neumann: test the parallelogram law, or recognise a positive definite quadratic form.
Options (C): $2x^2-2xy+y^2=v^TAv$ with $A=\begin{pmatrix}2&-1\\-1&1\end{pmatrix}$, minors $2$ and $1$ — positive definite, so the norm comes from $\langle u,v\rangle=v^TAu$. (A): $e_1,e_2$ give $\|e_1\pm e_2\|^2=9$, so $18$ on the left and $2\cdot1+2\cdot4=10$ on the right. (B): $1+1=2\ne4$. (D): $\|e_1\pm e_2\|^2=\sqrt2$, so $2\sqrt2\ne4$.
Answer (C).

E2. GATE MA · MCQ Let $f:L^2[0,1]\to\mathbb{R}$, $f(u)=\int_0^1(1+t)\,u(t)\,dt$ (real scalars). Then $\|f\|$ equals (A) $\tfrac32$ (B) $2$ (C) $\sqrt{7/3}$ (D) $\tfrac73$

Idea Riesz: $f(u)=\langle u,y\rangle$ with $y=1+t$, so $\|f\|=\|1+t\|_2$.
Compute $\int_0^1(1+t)^2dt=1+1+\tfrac13=\tfrac73$, so $\|f\|=\sqrt{7/3}\approx1.5275$.
Distractors (A) is $\int_0^1|1+t|$, the norm of the same functional on $(C[0,1],\|\cdot\|_\infty)$ (§8.1); (B) is $\max|1+t|$, its norm on $L^1$; (D) forgets the square root.
Answer (C).

E3. GATE MA · MSQ CSIR Part C Let $M$ be a subspace (not necessarily closed) of a Hilbert space $H$. Which of the following statements are true? (One or more options may be correct.) (A) $M^\perp$ is a closed subspace of $H$. (B) $(M^\perp)^\perp=M$. (C) If $M$ is closed, then every $x\in H$ is uniquely $m+n$ with $m\in M$, $n\in M^\perp$. (D) If $M^\perp=\{0\}$, then $M=H$.

(A) True An intersection of kernels of continuous functionals (Definition 8.11).
(B) False $M^{\perp\perp}=\overline M$; for $M=c_{00}$ in $\ell^2$, $M^{\perp\perp}=\ell^2\ne M$.
(C) True The projection theorem.
(D) False $M^\perp=\{0\}$ only says $M$ is dense: again $c_{00}\subsetneq\ell^2$.
Answer (A) and (C).

E4. GATE MA · MSQ Let $(e_n)_{n\ge1}$ be an orthonormal sequence in an infinite-dimensional Hilbert space $H$. Which of the following are true? (One or more options may be correct.) (A) $\langle x,e_n\rangle\to0$ for every $x\in H$. (B) $(e_n)$ has a norm-convergent subsequence. (C) $\sum_n\langle x,e_n\rangle e_n$ converges in $H$ for every $x\in H$. (D) If $\sum_n|\langle x,e_n\rangle|^2=\|x\|^2$ for every $x\in H$, then the closed linear span of $\{e_n\}$ is $H$.

(A) True Bessel's inequality: the series $\sum|\langle x,e_n\rangle|^2$ converges.
(B) False $\|e_n-e_m\|=\sqrt2$ for $n\ne m$.
(C) True Bessel and Theorem 8.25 (the sum is the projection onto the closed span — not necessarily $x$).
(D) True Parseval for all $x$ is criterion (5) of Theorem 8.26, equivalent to density of the span.
Answer (A), (C) and (D).

E5. GATE MA · NAT In real $L^2[-1,1]$, the distance from $f(t)=t^3$ to the subspace spanned by $1$ and $t$, correct to two decimal places, is ______.

Idea $1\perp t$ on $[-1,1]$, and $t^3\perp1$ (odd), so $P(t^3)=\frac{\langle t^3,t\rangle}{\|t\|^2}t=\frac{2/5}{2/3}t=\frac35t$.
Compute $\operatorname{dist}^2=\|t^3\|^2-\frac{\langle t^3,t\rangle^2}{\|t\|^2}$ $=\frac27-\frac{4/25}{2/3}=\frac27-\frac6{25}=\frac8{175}$, so $\operatorname{dist}=\sqrt{8/175}\approx0.2138$.
Answer $0.21$ (accept $0.21$ to $0.22$).

E6. GATE MA · NAT The Fourier series of $f(t)=|t|$ on $[-\pi,\pi]$ is $\dfrac\pi2-\dfrac4\pi\displaystyle\sum_{k\ge1}\dfrac{\cos(2k-1)t}{(2k-1)^2}$. Using Parseval's identity, the value of $\displaystyle\sum_{k=1}^{\infty}\frac1{(2k-1)^4}$, correct to two decimal places, is ______.

Idea Real form of Parseval: $\frac1\pi\int_{-\pi}^{\pi}|f|^2=\frac{a_0^2}2+\sum_{n\ge1}(a_n^2+b_n^2)$, here with $\frac{a_0}2=\frac\pi2$ and $a_{2k-1}=-\frac4{\pi(2k-1)^2}$.
Compute $\frac1\pi\int_{-\pi}^{\pi}t^2dt=\frac{2\pi^2}3$ and $\frac{a_0^2}2=\frac{\pi^2}2$, so $\frac{16}{\pi^2}\sum\frac1{(2k-1)^4}=\frac{2\pi^2}3-\frac{\pi^2}2=\frac{\pi^2}6$, giving $\sum\frac1{(2k-1)^4}=\frac{\pi^4}{96}\approx1.0147$.
Answer $1.01$ (accept $1.01$ to $1.02$).
🧑‍🏫 For the teacher

Suggested plan — 3 lectures × 60 min.

  1. Inner products and geometry. Definition 8.10 with the rule cards of examples; Theorem 8.20 (the "choose $\lambda$" proof on the board); Theorem 8.21 — prove (1), (2) and the additivity step of (3), which students rarely see. Example 1 as a class quiz ("which norms are Euclidean?"), then GATE corner E1.
  2. Projections. Definition 8.11, Theorem 8.22 with a picture of the parallelogram argument, Theorem 8.23. Project Interactive 1: start at $p=2$, then switch to $p=\infty$ and let the animation stop at $\theta=0^\circ$; ask why the segment appears. Example 2 and the strategy box (Gram determinants); Example 3 to show what completeness buys. Finish with E3 and E5.
  3. Bases, Riesz and adjoints. Theorems 8.24–8.26 (prove 8.24 and the equivalences of 8.26); Interactive 2 in square-wave mode — let the class predict the Bessel sum at $N=1$ ($8/\pi^2\approx81\%$), then move $\delta$. Example 4 (Parseval sums); Theorem 8.27 in full; Definition 8.13, Theorem 8.28 and Example 7. E2, E4 and E6 as a closing drill.

Discussion questions. (1) The proof of the closest-point theorem uses the parallelogram law twice. Where exactly, and what goes wrong for the sup norm? (2) Why is "maximal orthonormal set" a better definition of basis than "orthonormal set whose span is everything"? (3) Riesz identifies $H^*$ with $H$. Why does the identification fail to be linear over $\mathbb{C}$, and does that matter for real GATE problems?

Misconceptions to watch for. "Every norm has a parallelogram law"; "$M^{\perp\perp}=M$ for every subspace"; "an orthonormal basis is a basis in the linear-algebra sense"; "Bessel's inequality is always an equality"; "every functional on an inner-product space is an inner product"; forgetting the square root in $\|f\|=\|y_f\|$; confusing $\int|g|$ (norm on $C$) with $\|g\|_2$ (norm on $L^2$).

Suggested homework. I(1, 3, 6, 8), II(1, 2, 4, 5, 8), III(2, 3, 4, 7), IV(1, 3, 5, 6, 8); GATE corner E1, E5, E6.

🔬 Research–Teaching NexusResearch-tutored · optional, about 60 minutes

Research question. How many Fourier coefficients does it take to capture 99% of the energy of a signal, and how does the answer depend on the smoothness of the signal? Parseval's identity turns this into a question about the decay of the coefficients.

Why it matters. Transform coding — the idea behind JPEG images, MP3 audio and the compression of ECG recordings in portable heart monitors — keeps only the largest coefficients in an orthonormal basis and discards the rest. By Bessel and Parseval, the squared $L^2$ error of keeping a set of coefficients is exactly the energy of the discarded ones. Signals with jumps compress badly in a Fourier basis; smooth signals compress extremely well.

What to do.

  1. For the square wave $\operatorname{sgn}t$, the triangle wave $|t|-\frac\pi2$ and the smooth pulse $e^{\cos t}-I_0(1)$ (mean removed; its coefficients are modified Bessel values $I_k(1)$), compute the fraction $F_N=\|S_Nf\|^2/\|f\|^2$ of energy captured by harmonics up to $N$, using Parseval rather than numerical integration.
  2. Find the smallest $N$ with $F_N\ge0.99$ for each signal.
  3. Plot $1-F_N$ against $N$ on log–log axes and relate the slopes to the decay rates $|\hat f(k)|\sim k^{-1}$, $k^{-2}$ and faster than any power.
  4. Check one value against Interactive 2, and explain in one paragraph why the location of a jump cannot be "seen" by a few Fourier coefficients.

Tools: Python with NumPy and SciPy (scipy.special.iv for $I_k$), or a spreadsheet.

Hand in: A table of $F_N$ for $N=1,3,5,\dots$, the log–log plot, and a short paragraph recommending a basis for compressing (i) a smooth temperature cycle and (ii) a signal with abrupt switches.

Going further. Repeat with the Haar wavelet basis of $L^2[0,1]$, keeping the $N$ largest coefficients rather than the first $N$ (nonlinear approximation): a single jump then costs only about one coefficient per scale.

Instructor note: for $\operatorname{sgn}t$, $F_N=\frac8{\pi^2}\sum_{k\text{ odd}\le N}k^{-2}$: $F_1\approx0.8106$, $F_5\approx0.9331$, $F_{39}\approx0.9899$, $F_{41}\approx0.9904$, so 99% needs harmonics up to $41$ and $1-F_N\approx\frac4{\pi^2N}$. For the triangle wave, $F_1=96/\pi^4\approx0.9855$ and $F_3\approx0.9977$ ($1-F_N\sim N^{-3}$). For $e^{\cos t}$, $F_1\approx0.9441$, $F_2\approx0.9985$, $F_3\approx0.99998$ (exponential decay). All values were computed from the coefficient formulas.

🧪 Self-check quiz

0/5 correct

Q1. Which of these is a Hilbert space?

Q2. If $P$ is the orthogonal projection onto a closed subspace $M\ne\{0\}$, then $\|P\|$ equals

Q3. Bessel's inequality $\sum|\langle x,e_\alpha\rangle|^2\le\|x\|^2$ is an equality for every $x$ exactly when the orthonormal set $\{e_\alpha\}$ is

Q4. The norm of $u\mapsto\int_0^1u(t)\,dt$ on $L^2[0,1]$ is

Q5. The adjoint of the right shift $S(x_1,x_2,\dots)=(0,x_1,x_2,\dots)$ on $\ell^2$ is