The dot product, generalised: inner products measure length, distance and angle in any vector space — and orthogonal sets turn finding coordinates into one division per vector.
In Section 4.2 the dot product $\mathbf{u}\cdot\mathbf{v}$ gave us length, distance and angle in $\mathbb{R}^n$, and in particular a test for perpendicularity: $\mathbf{u}\cdot\mathbf{v}=0$. This section takes that test seriously. Perpendicular directions are the most useful directions there are: a surveyor measures along north and east, an engineer splits a force into a part along a beam and a part across it, and a mobile-phone network lets several users share one radio channel by giving each of them a code that is perpendicular to everybody else's.
We shall do three things. First, we generalise the dot product to an inner product, so that "length" and "perpendicular" make sense for polynomials, for weighted data and even for functions. Second, we collect everything perpendicular to a subspace into a new subspace, its orthogonal complement — and discover that the null space of a matrix is exactly the set of vectors perpendicular to its rows. Third, we see why orthogonal bases are so convenient: in such a basis every coordinate is found by a single dot product and a division, with no system of equations to solve.
Inner Products
Recall that for $\mathbf{u},\mathbf{v}$ in $\mathbb{R}^n$ the dot product is $\mathbf{u}\cdot\mathbf{v}=u_1v_1+u_2v_2+\dots+u_nv_n=\mathbf{u}^{T}\mathbf{v}$, that $\|\mathbf{v}\|=\sqrt{\mathbf{v}\cdot\mathbf{v}}$, and that $\mathbf{u}\cdot\mathbf{v}=\|\mathbf{u}\|\|\mathbf{v}\|\cos\theta$. Every proof about length and angle in Section 4.2 used only four properties of the dot product. Anything else with those four properties deserves the same name.
📘 Definition 8.1 — Inner product
An inner product on a real vector space $V$ is a rule that assigns to each pair of vectors $\mathbf{u},\mathbf{v}$ in $V$ a real number $\langle\mathbf{u},\mathbf{v}\rangle$ such that, for all $\mathbf{u},\mathbf{v},\mathbf{w}$ in $V$ and every scalar $c$:
$\langle\mathbf{u},\mathbf{u}\rangle\ge0$, and $\langle\mathbf{u},\mathbf{u}\rangle=0$ only when $\mathbf{u}=\mathbf{0}$ (positivity).
A vector space together with an inner product is called an inner product space. The dot product is the standard inner product on $\mathbb{R}^n$.
Three families of inner products appear in this chapter.
Weighted inner products on $\mathbb{R}^n$. Choose positive weights $w_1,\dots,w_n$ and set $\langle\mathbf{u},\mathbf{v}\rangle=w_1u_1v_1+\dots+w_nu_nv_n$. Weights let a statistician trust some measurements more than others.
Evaluation inner products on $P_n$. Pick $n+1$ distinct numbers $t_0,\dots,t_n$ and set $\langle p,q\rangle=p(t_0)q(t_0)+\dots+p(t_n)q(t_n)$ — the dot product of the lists of values. Positivity holds because a nonzero polynomial of degree at most $n$ cannot vanish at $n+1$ different points.
The integral inner product on $C[a,b]$ (uses calculus). On the space of continuous functions on $[a,b]$ set $\langle f,g\rangle=\displaystyle\int_a^bf(t)g(t)\,dt$. This is the "continuous" version of the dot product: instead of adding finitely many products $u_iv_i$ we integrate the products $f(t)g(t)$.
📘 Definition 8.2 — Length, distance and orthogonality in an inner product space
In an inner product space $V$, the norm (length) of $\mathbf{v}$ is $\|\mathbf{v}\|=\sqrt{\langle\mathbf{v},\mathbf{v}\rangle}$, and the distance between $\mathbf{u}$ and $\mathbf{v}$ is $\operatorname{dist}(\mathbf{u},\mathbf{v})=\|\mathbf{u}-\mathbf{v}\|$. Vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if $\langle\mathbf{u},\mathbf{v}\rangle=0$. A vector of norm $1$ is a unit vector; for $\mathbf{v}\ne\mathbf{0}$, the vector $\dfrac{1}{\|\mathbf{v}\|}\mathbf{v}$ is a unit vector in the direction of $\mathbf{v}$ (we say we normalise $\mathbf{v}$).
📐 Theorem 8.1 — Pythagoras, Cauchy–Schwarz and the triangle inequality
For all vectors $\mathbf{u},\mathbf{v}$ in an inner product space:
Pythagoras: $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if and only if $\|\mathbf{u}+\mathbf{v}\|^2=\|\mathbf{u}\|^2+\|\mathbf{v}\|^2$.
Why Pythagoras holds: by the axioms, $\|\mathbf{u}+\mathbf{v}\|^2=\langle\mathbf{u}+\mathbf{v},\mathbf{u}+\mathbf{v}\rangle=\|\mathbf{u}\|^2+2\langle\mathbf{u},\mathbf{v}\rangle+\|\mathbf{v}\|^2$, and the middle term vanishes exactly when $\langle\mathbf{u},\mathbf{v}\rangle=0$.
✏️ Example 1 — A weighted inner product on $\mathbb{R}^2$
On $\mathbb{R}^2$ define $\langle\mathbf{u},\mathbf{v}\rangle=3u_1v_1+2u_2v_2$. Let $\mathbf{u}=(1,1)$ and $\mathbf{v}=(2,-3)$. (a) Show that $\mathbf{u}$ and $\mathbf{v}$ are orthogonal for this inner product but not for the dot product. (b) Find $\|\mathbf{u}\|$, $\|\mathbf{v}\|$ and $\operatorname{dist}(\mathbf{u},\mathbf{v})$, and check Pythagoras.
💡 Show solution▶
Is it an inner product? Symmetry, additivity and homogeneity are clear because each term is a product $u_iv_i$ times a constant. Positivity: $\langle\mathbf{u},\mathbf{u}\rangle=3u_1^2+2u_2^2\ge0$, and it is $0$ only when $u_1=u_2=0$ — this is where the weights must be positive.
(a) $\langle\mathbf{u},\mathbf{v}\rangle=3(1)(2)+2(1)(-3)=6-6=0$, so they are orthogonal. But $\mathbf{u}\cdot\mathbf{v}=2-3=-1\ne0$: whether two vectors are "perpendicular" depends on which inner product we use.
(b) $\|\mathbf{u}\|=\sqrt{3+2}=\sqrt5$, $\ \|\mathbf{v}\|=\sqrt{3(4)+2(9)}=\sqrt{30}$, and $\mathbf{u}-\mathbf{v}=(-1,4)$, so $\operatorname{dist}(\mathbf{u},\mathbf{v})=\sqrt{3(1)+2(16)}=\sqrt{35}$.
Check Pythagoras (with $-\mathbf{v}$, which is also orthogonal to $\mathbf{u}$): $\|\mathbf{u}-\mathbf{v}\|^2=35=5+30=\|\mathbf{u}\|^2+\|\mathbf{v}\|^2$. ✓
✏️ Example 2 — An inner product on the polynomials $P_2$
On $P_2$ use $\langle p,q\rangle=p(-1)q(-1)+p(0)q(0)+p(1)q(1)$. (a) Show that $t$ and $t^2$ are orthogonal. (b) Find $\|1\|$, $\|t\|$ and $\|t^2\|$. (c) Are $1$ and $t^2$ orthogonal? If not, find the number $a$ for which $t^2+a$ is orthogonal to $1$.
💡 Show solution▶
Values List each polynomial's values at $t=-1,0,1$: $\ 1\mapsto(1,1,1)$, $\ t\mapsto(-1,0,1)$, $\ t^2\mapsto(1,0,1)$. The inner product is just the dot product of these lists.
(c) $\langle1,t^2\rangle=1+0+1=2\ne0$, so they are not orthogonal. Now $\langle t^2+a,1\rangle=\langle t^2,1\rangle+a\langle1,1\rangle=2+3a$, which is $0$ for $a=-\tfrac23$. Check with values: $t^2-\tfrac23\mapsto\left(\tfrac13,-\tfrac23,\tfrac13\right)$, whose entries add to $0$. ✓
✏️ Example 3 — Orthogonal functions (uses calculus)
On $C[-1,1]$ use $\langle f,g\rangle=\displaystyle\int_{-1}^{1}f(t)g(t)\,dt$. (a) Show that $1$ and $t$ are orthogonal, and so are $t$ and $t^2$. (b) Find $\|1\|$ and $\|t\|$. (c) Show that $t^2-\tfrac13$ is orthogonal to $1$.
💡 Show solution▶
(a) $\langle1,t\rangle=\displaystyle\int_{-1}^1t\,dt=0$ and $\langle t,t^2\rangle=\displaystyle\int_{-1}^1t^3\,dt=0$: an odd function integrates to $0$ over an interval symmetric about $0$.
(b) $\|1\|^2=\displaystyle\int_{-1}^11\,dt=2$, so $\|1\|=\sqrt2$; $\ \|t\|^2=\displaystyle\int_{-1}^1t^2\,dt=\tfrac23$, so $\|t\|=\sqrt{2/3}$.
(c) $\left\langle t^2-\tfrac13,1\right\rangle=\displaystyle\int_{-1}^1\left(t^2-\tfrac13\right)dt=\tfrac23-\tfrac23=0$. ✓ The polynomials $1,\ t,\ t^2-\tfrac13$ are pairwise orthogonal; Section 8.2 shows how to produce such lists automatically.
⚠️ Common mistakes with inner products
1. Not every formula is an inner product. $\langle\mathbf{u},\mathbf{v}\rangle=u_1v_1-u_2v_2$ fails positivity: for $\mathbf{u}=(0,1)$ it gives $\langle\mathbf{u},\mathbf{u}\rangle=-1$. Always check axiom 4.
2. Orthogonality depends on the inner product. Example 1 shows two vectors that are orthogonal for one inner product and not for another. When a problem mentions an inner product, use that one for every length, distance and angle.
3. The norm needs a square root. $\langle\mathbf{v},\mathbf{v}\rangle$ is $\|\mathbf{v}\|^2$, not $\|\mathbf{v}\|$.
Orthogonal Complements
A vector that is perpendicular to a whole plane through the origin is perpendicular to every vector in it — think of a flagpole standing on flat ground. Collecting all such vectors gives a new subspace.
📘 Definition 8.3 — Orthogonal complement
Let $W$ be a subspace of $\mathbb{R}^n$. A vector $\mathbf{z}$ is orthogonal to $W$ if $\mathbf{z}\cdot\mathbf{w}=0$ for every $\mathbf{w}$ in $W$. The set of all vectors orthogonal to $W$ is the orthogonal complement of $W$, written $W^{\perp}$ ("$W$ perp"):
$$W^{\perp}=\left\{\mathbf{z}\in\mathbb{R}^n:\ \mathbf{z}\cdot\mathbf{w}=0\ \text{for all}\ \mathbf{w}\in W\right\}.$$
📐 Theorem 8.2 — Properties of the orthogonal complement
Let $W$ be a subspace of $\mathbb{R}^n$.
$W^{\perp}$ is a subspace of $\mathbb{R}^n$.
If $W=\operatorname{span}\{\mathbf{w}_1,\dots,\mathbf{w}_k\}$, then $\mathbf{z}$ is in $W^{\perp}$ if and only if $\mathbf{z}\cdot\mathbf{w}_i=0$ for each $i=1,\dots,k$. (It is enough to test a spanning set.)
$W\cap W^{\perp}=\{\mathbf{0}\}$.
$\dim W+\dim W^{\perp}=n$, and $\left(W^{\perp}\right)^{\perp}=W$.
Part 2 turns "find $W^{\perp}$" into "solve a homogeneous system": put the spanning vectors $\mathbf{w}_i$ as the rows of a matrix $A$; then $\mathbf{z}\cdot\mathbf{w}_i=0$ for all $i$ says exactly $A\mathbf{z}=\mathbf{0}$. That is the content of the next theorem, which ties this chapter to the four subspaces of Section 5.4.
📐 Theorem 8.3 — The row space and the null space are orthogonal complements
For any $m\times n$ matrix $A$,
$$\left(\operatorname{Row}A\right)^{\perp}=\operatorname{Nul}A\qquad\text{and}\qquad\left(\operatorname{Col}A\right)^{\perp}=\operatorname{Nul}A^{T}.$$
Reason: the $i$-th entry of $A\mathbf{x}$ is (row $i$ of $A$)$\,\cdot\,\mathbf{x}$. So $A\mathbf{x}=\mathbf{0}$ says precisely that $\mathbf{x}$ is orthogonal to every row, hence (by Theorem 8.2) to $\operatorname{Row}A$. Apply this to $A^{T}$, whose rows are the columns of $A$, for the second statement. Together with $\operatorname{rank}A+\operatorname{nullity}A=n$ this is part 4 of Theorem 8.2 again.
✏️ Example 4 — A basis for an orthogonal complement in $\mathbb{R}^4$
Let $W=\operatorname{span}\{\mathbf{w}_1,\mathbf{w}_2\}$ in $\mathbb{R}^4$ with $\mathbf{w}_1=(1,2,0,-1)$ and $\mathbf{w}_2=(2,4,1,0)$. Find a basis for $W^{\perp}$ and check that $\dim W+\dim W^{\perp}=4$.
💡 Show solution▶
Set up Put $\mathbf{w}_1,\mathbf{w}_2$ as rows: $W=\operatorname{Row}A$ with $A=\begin{bmatrix}1&2&0&-1\\2&4&1&0\end{bmatrix}$, so $W^{\perp}=\operatorname{Nul}A$ by Theorem 8.3.
Reduce $R_2\to R_2-2R_1$: $\begin{bmatrix}1&2&0&-1\\0&0&1&2\end{bmatrix}$, already in reduced echelon form. Pivots in columns 1 and 3; $x_2$ and $x_4$ are free.
Solve $x_1=-2x_2+x_4$, $\ x_3=-2x_4$, so
$$\mathbf{x}=x_2\begin{bmatrix}-2\\1\\0\\0\end{bmatrix}+x_4\begin{bmatrix}1\\0\\-2\\1\end{bmatrix},\qquad W^{\perp}=\operatorname{span}\left\{\begin{bmatrix}-2\\1\\0\\0\end{bmatrix},\begin{bmatrix}1\\0\\-2\\1\end{bmatrix}\right\}.$$
Interactive 1 — A Subspace and Its Orthogonal Complement in $\mathbb{R}^3$
The subspace $W$ is drawn in blue and its orthogonal complement $W^{\perp}$ in red. A line through the origin always pairs with a plane, because $\dim W+\dim W^{\perp}=3$. The green vector travels round the plane; its dot product with the line's direction stays $0$ the whole time. Press Play to turn the picture, or use the slider.
basis of $W$—
basis of $W^{\perp}$—
green vector $\mathbf{z}$—
$\mathbf{z}\cdot(\text{line direction})$—
Orthogonal and Orthonormal Sets
📘 Definition 8.4 — Orthogonal and orthonormal sets and bases
A set of vectors $\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ in $\mathbb{R}^n$ (or in an inner product space) is an orthogonal set if every pair of distinct vectors in it is orthogonal: $\mathbf{u}_i\cdot\mathbf{u}_j=0$ whenever $i\ne j$. It is an orthonormal set if, in addition, every vector in it is a unit vector. An orthogonal basis (or orthonormal basis) of a subspace $W$ is a basis of $W$ that is also an orthogonal (orthonormal) set.
📐 Theorem 8.4 — Orthogonal sets are linearly independent
If $S=\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ is an orthogonal set of nonzero vectors, then $S$ is linearly independent, and so it is a basis for the subspace it spans.
Proof. Suppose $c_1\mathbf{u}_1+\dots+c_p\mathbf{u}_p=\mathbf{0}$. Take the dot product of both sides with $\mathbf{u}_1$. Every term $c_j\,\mathbf{u}_j\cdot\mathbf{u}_1$ with $j\ne1$ is $0$, leaving $c_1\,(\mathbf{u}_1\cdot\mathbf{u}_1)=0$. Since $\mathbf{u}_1\ne\mathbf{0}$, $\mathbf{u}_1\cdot\mathbf{u}_1\gt0$, so $c_1=0$. The same argument with $\mathbf{u}_2,\dots,\mathbf{u}_p$ gives every $c_j=0$. $\blacksquare$
The proof contains a gift. Dotting with $\mathbf{u}_j$ "switches off" every term except the $j$-th, so if a vector $\mathbf{y}$ is written as a combination of the $\mathbf{u}_j$, each coefficient can be read off separately.
📐 Theorem 8.5 — Coordinates in an orthogonal basis
Let $\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ be an orthogonal basis for a subspace $W$ of $\mathbb{R}^n$. Then every $\mathbf{y}$ in $W$ is
$$\mathbf{y}=c_1\mathbf{u}_1+\dots+c_p\mathbf{u}_p\qquad\text{with}\qquad c_j=\frac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}\quad(j=1,\dots,p).$$
If the basis is orthonormal, simply $c_j=\mathbf{y}\cdot\mathbf{u}_j$. (In an inner product space replace each dot product by $\langle\ ,\ \rangle$.)
Geometrically, $c_j\mathbf{u}_j$ is the projection of $\mathbf{y}$ onto the line through $\mathbf{u}_j$, the vector $\operatorname{proj}_{\mathbf{u}_j}\mathbf{y}$ of Section 4.2. In an orthogonal basis, $\mathbf{y}$ is just the sum of its projections onto the basis directions.
✏️ Example 5 — Coordinates without solving a system
Let $\mathbf{u}_1=(1,1,1)$, $\mathbf{u}_2=(1,-2,1)$, $\mathbf{u}_3=(1,0,-1)$. Show that $\{\mathbf{u}_1,\mathbf{u}_2,\mathbf{u}_3\}$ is an orthogonal basis of $\mathbb{R}^3$, and write $\mathbf{y}=(5,-1,2)$ as a combination of these vectors.
💡 Show solution▶
Orthogonal? $\mathbf{u}_1\cdot\mathbf{u}_2=1-2+1=0$, $\ \mathbf{u}_1\cdot\mathbf{u}_3=1+0-1=0$, $\ \mathbf{u}_2\cdot\mathbf{u}_3=1+0-1=0$. The vectors are nonzero, so by Theorem 8.4 they are independent — three independent vectors in $\mathbb{R}^3$ form a basis.
Coefficients $c_1=\tfrac63=2$, $\ c_2=\tfrac96=\tfrac32$, $\ c_3=\tfrac32$, so $$\mathbf{y}=2\mathbf{u}_1+\tfrac32\mathbf{u}_2+\tfrac32\mathbf{u}_3 .$$
Check $2(1,1,1)+\tfrac32(1,-2,1)+\tfrac32(1,0,-1)=(2+\tfrac32+\tfrac32,\ 2-3+0,\ 2+\tfrac32-\tfrac32)=(5,-1,2)$. ✓ With a non-orthogonal basis we would have had to row-reduce a $3\times4$ augmented matrix.
Interactive 2 — Coordinates as Projection Lengths
Drag the black vector $\mathbf{y}$. Dashed lines drop perpendicularly from its tip onto the lines through $\mathbf{u}_1$ (blue) and $\mathbf{u}_2$ (red); the feet are $c_1\mathbf{u}_1$ and $c_2\mathbf{u}_2$ with $c_j=\dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$. For an orthogonal basis the two projections add up to $\mathbf{y}$ exactly. Switch to the non-orthogonal basis: the orange sum of projections misses $\mathbf{y}$, while the true coordinates (green parallelogram) are different numbers. Play turns the basis.
Put orthonormal vectors side by side as the columns of a matrix $U$. The $(i,j)$ entry of $U^{T}U$ is (column $i$)$\,\cdot\,$(column $j$), which is $1$ on the diagonal and $0$ elsewhere. So "orthonormal columns" is the same as "$U^{T}U=I$".
📐 Theorem 8.6 — Matrices with orthonormal columns
An $m\times n$ matrix $U$ has orthonormal columns if and only if $U^{T}U=I_n$. In that case, for all $\mathbf{x},\mathbf{y}$ in $\mathbb{R}^n$:
$\|U\mathbf{x}\|=\|\mathbf{x}\|$ (lengths are preserved);
$(U\mathbf{x})\cdot(U\mathbf{y})=\mathbf{x}\cdot\mathbf{y}$ (dot products, hence angles, are preserved);
$(U\mathbf{x})\cdot(U\mathbf{y})=0$ if and only if $\mathbf{x}\cdot\mathbf{y}=0$.
Why: $(U\mathbf{x})\cdot(U\mathbf{y})=(U\mathbf{x})^{T}(U\mathbf{y})=\mathbf{x}^{T}U^{T}U\mathbf{y}=\mathbf{x}^{T}\mathbf{y}$; take $\mathbf{y}=\mathbf{x}$ for part 1.
📘 Definition 8.5 — Orthogonal matrix
A square matrix $U$ is an orthogonal matrix if $U^{T}U=I$, that is, if its columns form an orthonormal basis of $\mathbb{R}^n$. Then $U$ is invertible with
$$U^{-1}=U^{T},$$
so $UU^{T}=I$ as well, which says that the rows of $U$ are orthonormal too. Since $\det\left(U^{T}U\right)=(\det U)^2=1$, every orthogonal matrix has $\det U=\pm1$.
✏️ Example 6 — An orthogonal matrix preserves length
Let $U=\begin{bmatrix}\tfrac35&-\tfrac45\\[2pt]\tfrac45&\tfrac35\end{bmatrix}$. (a) Show that $U$ is orthogonal and write down $U^{-1}$. (b) For $\mathbf{x}=(2,1)$, verify that $\|U\mathbf{x}\|=\|\mathbf{x}\|$. (c) Describe $U$ geometrically.
💡 Show solution▶
(a) Columns: $\left(\tfrac35,\tfrac45\right)\cdot\left(-\tfrac45,\tfrac35\right)=-\tfrac{12}{25}+\tfrac{12}{25}=0$, and each has length $\sqrt{\tfrac{9}{25}+\tfrac{16}{25}}=1$. So $U^{T}U=I$ and $$U^{-1}=U^{T}=\begin{bmatrix}\tfrac35&\tfrac45\\[2pt]-\tfrac45&\tfrac35\end{bmatrix}.$$ No row reduction needed.
(b) $U\mathbf{x}=\left(\tfrac65-\tfrac45,\ \tfrac85+\tfrac35\right)=\left(\tfrac25,\tfrac{11}5\right)$, and $\|U\mathbf{x}\|^2=\tfrac{4}{25}+\tfrac{121}{25}=\tfrac{125}{25}=5=\|\mathbf{x}\|^2$. ✓
(c) $U$ has the form $\begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$ with $\cos\theta=\tfrac35$, $\sin\theta=\tfrac45$, so it is a rotation through $\theta\approx53.13^\circ$; $\det U=\tfrac{9}{25}+\tfrac{16}{25}=1$. Rotations and reflections are the orthogonal transformations of the plane — they move shapes without stretching them.
⚠️ Orthogonal or orthonormal? — words that trip people up
1. An "orthogonal matrix" has orthonormal columns. $\begin{bmatrix}1&1\\1&-1\end{bmatrix}$ has orthogonal columns of length $\sqrt2$; it is not an orthogonal matrix ($A^{T}A=2I$). Dividing by $\sqrt2$ fixes it.
2. The formula $c_j=\dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$ needs an orthogonal basis. For any other basis it gives wrong numbers — see the third setting of Interactive 2.
3. Watch out for $\mathbf{0}$. The zero vector is orthogonal to everything, so $\{\mathbf{u},\mathbf{0}\}$ is an orthogonal set — but it is dependent. Theorem 8.4 needs nonzero vectors.
4. Non-square $U$. If $U$ is $m\times n$ with $m\gt n$ and orthonormal columns, then $U^{T}U=I_n$ but $UU^{T}\ne I_m$; such a $U$ has no inverse.
Exercises
I
Inner products, lengths and distances.
1 For $\mathbf{u}=(2,-1,3)$ and $\mathbf{v}=(1,4,-2)$ in $\mathbb{R}^3$ (dot product), find $\mathbf{u}\cdot\mathbf{v}$, $\|\mathbf{u}\|$, $\|\mathbf{v}\|$ and $\operatorname{dist}(\mathbf{u},\mathbf{v})$.
2 On $\mathbb{R}^2$ let $\langle\mathbf{u},\mathbf{v}\rangle=2u_1v_1+5u_2v_2$. For $\mathbf{u}=(1,-1)$, $\mathbf{v}=(3,2)$ find $\langle\mathbf{u},\mathbf{v}\rangle$, $\|\mathbf{u}\|$ and $\|\mathbf{v}\|$.
3 With the inner product of part 2, find all vectors orthogonal to $(1,1)$.
4 Show that $\langle\mathbf{u},\mathbf{v}\rangle=u_1v_1-u_2v_2$ is not an inner product on $\mathbb{R}^2$.
5 On $P_2$ with $\langle p,q\rangle=p(-1)q(-1)+p(0)q(0)+p(1)q(1)$, let $p=1+t$ and $q=2-t^2$. Find $\langle p,q\rangle$, $\|p\|$ and $\|q\|$.
6 With the inner product of part 5, find $a$ so that $t^2+a$ is orthogonal to the constant polynomial $1$.
7 (uses calculus) On $C[0,1]$ with $\langle f,g\rangle=\displaystyle\int_0^1f(t)g(t)\,dt$, find $\langle t,t^2\rangle$, $\|t\|$, $\|t^2\|$, and the number $c$ for which $t^2-ct$ is orthogonal to $t$.
8 Verify the Pythagorean theorem for $\mathbf{u}=(1,2,2)$ and $\mathbf{v}=(2,1,-2)$.
9 (uses calculus) On $C[-\pi,\pi]$ with $\langle f,g\rangle=\displaystyle\int_{-\pi}^{\pi}f(t)g(t)\,dt$, show that $\sin t$ and $\cos t$ are orthogonal, and find $\|\sin t\|$. (This orthogonality is the starting point of Fourier analysis, used to compress sound.)
💡 Show solutions▶
1 $\mathbf{u}\cdot\mathbf{v}=2-4-6=-8$; $\|\mathbf{u}\|=\sqrt{4+1+9}=\sqrt{14}$; $\|\mathbf{v}\|=\sqrt{1+16+4}=\sqrt{21}$; $\mathbf{u}-\mathbf{v}=(1,-5,5)$, so $\operatorname{dist}=\sqrt{51}\approx7.141$.
3 $\langle(x,y),(1,1)\rangle=2x+5y=0$, so the vectors orthogonal to $(1,1)$ are the multiples of $(5,-2)$: the line $\operatorname{span}\{(5,-2)\}$. (For the dot product it would be $\operatorname{span}\{(1,-1)\}$.)
4 Positivity fails: for $\mathbf{u}=(0,1)$, $\langle\mathbf{u},\mathbf{u}\rangle=0-1=-1\lt0$. (Also $(1,1)\ne\mathbf{0}$ has $\langle(1,1),(1,1)\rangle=0$.)
5 Values at $-1,0,1$: $p\mapsto(0,1,2)$, $q\mapsto(1,2,1)$. $\langle p,q\rangle=0+2+2=4$; $\|p\|=\sqrt{0+1+4}=\sqrt5$; $\|q\|=\sqrt{1+4+1}=\sqrt6$.
6 $\langle t^2+a,1\rangle=(1+a)+a+(1+a)=2+3a=0$, so $a=-\tfrac23$ (as in Example 2).
8 $\mathbf{u}\cdot\mathbf{v}=2+2-4=0$, so they are orthogonal. $\|\mathbf{u}\|^2=9$, $\|\mathbf{v}\|^2=9$, and $\mathbf{u}+\mathbf{v}=(3,3,0)$ has $\|\mathbf{u}+\mathbf{v}\|^2=18=9+9$. ✓
5 $A$ has two pivots, so $\operatorname{Col}A$ is a 2-dimensional subspace of $\mathbb{R}^2$, i.e. all of $\mathbb{R}^2$. Hence $\left(\operatorname{Col}A\right)^{\perp}=\operatorname{Nul}A^{T}=\{\mathbf{0}\}$ (dimension $0$, empty basis).
6 $W=\operatorname{Nul}\begin{bmatrix}3&-1&2\end{bmatrix}=\left(\operatorname{Row}\begin{bmatrix}3&-1&2\end{bmatrix}\right)^{\perp}$, so $W^{\perp}=\operatorname{span}\{(3,-1,2)\}$, the normal line. $\dim W=2$ (two free variables), $\dim W^{\perp}=1$, total $3$. ✓
7 If $\mathbf{x}$ is in both, then $\mathbf{x}$ is orthogonal to every vector of $W$ — including itself. So $\mathbf{x}\cdot\mathbf{x}=\|\mathbf{x}\|^2=0$, which forces $\mathbf{x}=\mathbf{0}$.
III
Orthogonal sets, orthogonal bases and coordinates.
1 Is $\{(1,2,-1),\ (1,0,1),\ (1,-1,-1)\}$ an orthogonal set? Is it a basis of $\mathbb{R}^3$?
2 Is $\{(1,1,0),\ (1,-1,1),\ (0,1,1)\}$ an orthogonal set?
3 Write $\mathbf{y}=(4,1,0)$ as a combination of the basis in part 1.
4 Write $\mathbf{y}=(3,-3,6)$ as a combination of the basis in part 1.
5 Normalise the basis of part 1 to obtain an orthonormal basis of $\mathbb{R}^3$.
6 Find a nonzero vector $\mathbf{w}$ such that $\{(1,1,1,1),\ (1,-1,1,-1),\ (1,1,-1,-1),\ \mathbf{w}\}$ is an orthogonal basis of $\mathbb{R}^4$.
7 Show that $\{(2,1),(-1,2)\}$ is an orthogonal basis of $\mathbb{R}^2$ and find the coordinates of $(7,-4)$ in it.
8Signal coding. Four phones share one channel. Phone $j$ is given the code $\mathbf{u}_j$ — the four vectors of part 6, with $\mathbf{w}=(1,-1,-1,1)$ — and sends the bit $b_j=\pm1$ by transmitting $b_j\mathbf{u}_j$. The receiver hears the sum $\mathbf{y}=(2,2,-2,2)$. Recover the four bits.
Check: the table of dot products $\mathbf{u}_i\cdot\mathbf{u}_j$ and the coordinates $c_j=\mathbf{y}\cdot\mathbf{u}_j/\mathbf{u}_j\cdot\mathbf{u}_j$
💡 Show solutions▶
1 $(1,2,-1)\cdot(1,0,1)=0$, $(1,2,-1)\cdot(1,-1,-1)=1-2+1=0$, $(1,0,1)\cdot(1,-1,-1)=0$. Yes, orthogonal; the vectors are nonzero, so by Theorem 8.4 they are independent and form a basis of $\mathbb{R}^3$.
2 No: $(1,1,0)\cdot(0,1,1)=1\ne0$ (the other two pairs do give $0$).
6 First check the three given vectors are mutually orthogonal (each pair gives $0$). Then $\mathbf{w}$ must lie in the orthogonal complement of their span: solve $w_1+w_2+w_3+w_4=0$, $w_1-w_2+w_3-w_4=0$, $w_1+w_2-w_3-w_4=0$. Adding and subtracting gives $w_1=w_4$, $w_2=w_3=-w_1$, so $\mathbf{w}=(1,-1,-1,1)$ (or any nonzero multiple). Four orthogonal nonzero vectors in $\mathbb{R}^4$ form a basis.
7 $(2,1)\cdot(-1,2)=0$ and both are nonzero, so they form an orthogonal basis of $\mathbb{R}^2$. $c_1=\tfrac{14-4}{5}=2$, $c_2=\tfrac{-7-8}{5}=-3$: $\ (7,-4)=2(2,1)-3(-1,2)$. ✓
8 Every code has $\mathbf{u}_j\cdot\mathbf{u}_j=4$, so $b_j=\tfrac14\,\mathbf{y}\cdot\mathbf{u}_j$: $b_1=\tfrac{2+2-2+2}{4}=1$, $b_2=\tfrac{2-2-2-2}{4}=-1$, $b_3=\tfrac{2+2+2-2}{4}=1$, $b_4=\tfrac{2-2+2+2}{4}=1$. The bits are $(+1,-1,+1,+1)$; check: $\mathbf{u}_1-\mathbf{u}_2+\mathbf{u}_3+\mathbf{u}_4=(2,2,-2,2)$. ✓ Each phone's signal is separated from the others by one dot product.
IV
Orthogonal matrices.
1 Show that $U=\dfrac13\begin{bmatrix}1&2&2\\2&1&-2\\2&-2&1\end{bmatrix}$ is an orthogonal matrix and find $U^{-1}$.
2 Is $\begin{bmatrix}1&1\\1&-1\end{bmatrix}$ an orthogonal matrix? If not, find a scalar multiple of it that is.
3 Find all $a,b$ for which $\begin{bmatrix}0.6&a\\0.8&b\end{bmatrix}$ is orthogonal.
4 With $U$ from part 1 and $\mathbf{x}=(3,0,-3)$, compute $U\mathbf{x}$ and check that $\|U\mathbf{x}\|=\|\mathbf{x}\|$.
5 Prove that the product of two $n\times n$ orthogonal matrices is orthogonal.
6 Prove that every orthogonal matrix has determinant $1$ or $-1$. Give an example of each.
7Robotics. A robot arm's controller turns its gripper with $R_\theta=\begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$. Show that $R_\theta$ is orthogonal and explain why turning the gripper can never change the distance between two marked points on the object it holds.
8 Let $U$ be $n\times n$ orthogonal. Prove that the angle between $U\mathbf{x}$ and $U\mathbf{y}$ equals the angle between $\mathbf{x}$ and $\mathbf{y}$ (for nonzero $\mathbf{x},\mathbf{y}$).
💡 Show solutions▶
1 Columns (times 3): $(1,2,2)$, $(2,1,-2)$, $(2,-2,1)$. Dot products $2+2-4=0$, $2-4+2=0$, $4-2-2=0$; each has length $3$, so the columns of $U$ have length $1$. Hence $U^{T}U=I$ and $U^{-1}=U^{T}=U$ (this $U$ is symmetric).
2 The columns are orthogonal but have length $\sqrt2$, so $A^{T}A=2I\ne I$: not orthogonal. $\tfrac1{\sqrt2}\begin{bmatrix}1&1\\1&-1\end{bmatrix}$ is orthogonal.
3 The first column already has length $1$. The second must be a unit vector orthogonal to $(0.6,0.8)$: $0.6a+0.8b=0$ and $a^2+b^2=1$ give $(a,b)=(-0.8,0.6)$ or $(0.8,-0.6)$.
5 If $U^{T}U=I$ and $V^{T}V=I$, then $(UV)^{T}(UV)=V^{T}U^{T}UV=V^{T}IV=V^{T}V=I$. $\blacksquare$
6 $1=\det I=\det\left(U^{T}U\right)=\det U^{T}\det U=(\det U)^2$, so $\det U=\pm1$. Examples: $I$ (det $1$); the reflection $\begin{bmatrix}1&0\\0&-1\end{bmatrix}$ (det $-1$).
7 The columns $(\cos\theta,\sin\theta)$ and $(-\sin\theta,\cos\theta)$ have length $\sqrt{\cos^2\theta+\sin^2\theta}=1$ and dot product $-\cos\theta\sin\theta+\sin\theta\cos\theta=0$, so $R_\theta^{T}R_\theta=I$. For marked points $\mathbf{p},\mathbf{q}$: $\|R_\theta\mathbf{p}-R_\theta\mathbf{q}\|=\|R_\theta(\mathbf{p}-\mathbf{q})\|=\|\mathbf{p}-\mathbf{q}\|$ by Theorem 8.6 — a rotation is rigid.
8 $\cos\theta'=\dfrac{(U\mathbf{x})\cdot(U\mathbf{y})}{\|U\mathbf{x}\|\|U\mathbf{y}\|}=\dfrac{\mathbf{x}\cdot\mathbf{y}}{\|\mathbf{x}\|\|\mathbf{y}\|}=\cos\theta$ by Theorem 8.6, and both angles lie in $[0,\pi]$, so $\theta'=\theta$.
🔬 Research–Teaching NexusResearch-tutored · optional, about 30 minutes
Research question. How can several mobile phones transmit on the same frequency at the same time without their signals becoming a useless jumble — and how much noise can the method tolerate?
Why it matters. Code-division multiple access (CDMA), used in 3G mobile networks and in satellite navigation systems such as GPS, gives each transmitter a code vector. The codes are chosen to be mutually orthogonal (or nearly so), and the receiver separates the users with dot products — exactly Theorem 8.5.
What to do.
Use the four codes of Exercise III.8: $(1,1,1,1)$, $(1,-1,1,-1)$, $(1,1,-1,-1)$, $(1,-1,-1,1)$. Users send bits $b=(+1,-1,-1,+1)$. Compute the transmitted sum $\mathbf{y}=\sum b_j\mathbf{u}_j$.
The channel adds noise $\mathbf{n}=(0.3,-0.2,0.1,-0.4)$. Decode $\hat b_j=\tfrac14(\mathbf{y}+\mathbf{n})\cdot\mathbf{u}_j$ and round each to the nearest of $\pm1$. Are all bits recovered?
In a spreadsheet, add random noise with entries between $-a$ and $a$ and record the fraction of wrongly decoded bits over 200 trials for $a=0.5,1,1.5,2$.
Replace the codes by a set that is not orthogonal, e.g. $(1,1,1,1)$, $(1,1,1,-1)$, $(1,1,-1,-1)$, $(1,-1,-1,-1)$, decode with the same formula (no noise) and explain what goes wrong.
Tools: Calculator, spreadsheet (or Python)
Hand in: A half-page note with your decoded bits, a small table of error rate against noise level, and one sentence on why orthogonality matters.
Going further. Longer codes resist noise better. Build the eight orthogonal codes of length 8 (Walsh–Hadamard codes) by the rule $H_{2n}=\begin{bmatrix}H_n&H_n\\H_n&-H_n\end{bmatrix}$ and repeat step 3. How does the error rate change?
Instructor note: $\mathbf{y}=(0,0,0,4)$; with the noise, $\mathbf{y}+\mathbf{n}=(0.3,-0.2,0.1,3.6)$ decodes to $(0.95,-0.75,-0.90,1.00)$, which rounds to the correct bits $(+1,-1,-1,+1)$. With the non-orthogonal codes, $\mathbf{y}=(0,-2,0,2)$ and the formula gives $(0,-1,-1,0)$ — users 1 and 4 are lost even without noise. In step 3 no errors are possible for $a\le1$ (the decoded noise is an average of four numbers in $[-a,a]$); simulation gives about $1\%$ wrong bits at $a=1.5$ and about $4\%$ at $a=2$.
🧪 Self-check quiz
0/5 correct
Q1. Which pair of vectors in $\mathbb{R}^2$ is orthogonal for the dot product?
Q2. For any matrix $A$, the orthogonal complement of $\operatorname{Row}A$ is
Q3. In the orthogonal basis $\mathbf{u}_1=(1,1)$, $\mathbf{u}_2=(1,-1)$, the coefficient of $\mathbf{u}_1$ for $\mathbf{y}=(5,1)$ is
Q4. Which statement is true for every orthogonal matrix $U$?
Q5. $W$ is a 2-dimensional subspace of $\mathbb{R}^5$. Then $\dim W^{\perp}$ is