Chapter 8 · Orthogonality and Least Squares

8.2 Orthogonal Bases and the Gram–Schmidt Process

Any basis can be straightened into an orthonormal one: subtract from each vector its projections onto the ones before, then normalise. Recorded as a matrix factorisation, that is $A=QR$.

Section 8.1 showed that orthogonal bases are a pleasure to work with: coordinates come from single dot products, and orthogonal matrices are inverted by transposing. But the bases that problems hand us are rarely orthogonal. The null space of a matrix comes out of row reduction with a basis that points in awkward directions; the columns of a data matrix are whatever the measurements happened to be; the natural basis $1,t,t^2$ of $P_2$ is not orthogonal for most inner products.

The Gram–Schmidt process repairs this. It takes any basis of a subspace and, one vector at a time, removes the part that points along the directions already chosen — what is left is perpendicular to all of them. The same calculation, written as matrices, factorises a matrix $A$ as $A=QR$ with $Q$ having orthonormal columns and $R$ upper triangular. This QR factorisation is one of the workhorses of scientific computing: it is how computers solve least-squares problems reliably (Section 8.3), and the starting point of the standard algorithm for computing eigenvalues.

The Idea: Subtract the Projection

Recall from Section 4.2 the projection of $\mathbf{x}$ onto the line through a nonzero vector $\mathbf{v}$: $$\operatorname{proj}_{\mathbf{v}}\mathbf{x}=\frac{\mathbf{x}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}\,\mathbf{v}.$$ What is left over, $\mathbf{x}-\operatorname{proj}_{\mathbf{v}}\mathbf{x}$, is orthogonal to $\mathbf{v}$: $$\left(\mathbf{x}-\frac{\mathbf{x}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}\mathbf{v}\right)\cdot\mathbf{v}=\mathbf{x}\cdot\mathbf{v}-\frac{\mathbf{x}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}\,(\mathbf{v}\cdot\mathbf{v})=0.$$ So from two independent vectors $\mathbf{x}_1,\mathbf{x}_2$ we get two orthogonal ones: keep $\mathbf{v}_1=\mathbf{x}_1$ and replace $\mathbf{x}_2$ by $\mathbf{v}_2=\mathbf{x}_2-\operatorname{proj}_{\mathbf{v}_1}\mathbf{x}_2$. Since $\mathbf{v}_2$ is $\mathbf{x}_2$ minus a multiple of $\mathbf{x}_1$, the new pair spans the same plane as the old one.

✏️ Example 1 — An orthogonal basis for a plane in $\mathbb{R}^3$
Let $W=\operatorname{span}\{\mathbf{x}_1,\mathbf{x}_2\}$ with $\mathbf{x}_1=(1,0,1)$ and $\mathbf{x}_2=(2,1,0)$. Find an orthogonal basis for $W$.
Keep the first $\mathbf{v}_1=\mathbf{x}_1=(1,0,1)$.
Project $\mathbf{x}_2\cdot\mathbf{v}_1=2+0+0=2$ and $\mathbf{v}_1\cdot\mathbf{v}_1=2$, so $\operatorname{proj}_{\mathbf{v}_1}\mathbf{x}_2=\tfrac22\mathbf{v}_1=(1,0,1)$.
Subtract $\mathbf{v}_2=\mathbf{x}_2-\operatorname{proj}_{\mathbf{v}_1}\mathbf{x}_2=(2,1,0)-(1,0,1)=(1,1,-1)$.
Check $\mathbf{v}_1\cdot\mathbf{v}_2=1+0-1=0$. ✓ So $\{(1,0,1),(1,1,-1)\}$ is an orthogonal basis of $W$ (two orthogonal nonzero vectors in a 2-dimensional space).

The Gram–Schmidt Process

With more vectors we repeat the idea: each new vector has its projections onto all the earlier $\mathbf{v}$'s removed.

📐 Theorem 8.7 — The Gram–Schmidt process
Let $\{\mathbf{x}_1,\dots,\mathbf{x}_p\}$ be a basis for a subspace $W$ of $\mathbb{R}^n$ (or of any inner product space, with $\langle\ ,\ \rangle$ in place of the dot product). Define $$\begin{aligned}\mathbf{v}_1&=\mathbf{x}_1,\\ \mathbf{v}_2&=\mathbf{x}_2-\frac{\mathbf{x}_2\cdot\mathbf{v}_1}{\mathbf{v}_1\cdot\mathbf{v}_1}\mathbf{v}_1,\\ \mathbf{v}_3&=\mathbf{x}_3-\frac{\mathbf{x}_3\cdot\mathbf{v}_1}{\mathbf{v}_1\cdot\mathbf{v}_1}\mathbf{v}_1-\frac{\mathbf{x}_3\cdot\mathbf{v}_2}{\mathbf{v}_2\cdot\mathbf{v}_2}\mathbf{v}_2,\\ &\ \ \vdots\\ \mathbf{v}_p&=\mathbf{x}_p-\frac{\mathbf{x}_p\cdot\mathbf{v}_1}{\mathbf{v}_1\cdot\mathbf{v}_1}\mathbf{v}_1-\dots-\frac{\mathbf{x}_p\cdot\mathbf{v}_{p-1}}{\mathbf{v}_{p-1}\cdot\mathbf{v}_{p-1}}\mathbf{v}_{p-1}.\end{aligned}$$ Then every $\mathbf{v}_k$ is nonzero, $\{\mathbf{v}_1,\dots,\mathbf{v}_p\}$ is an orthogonal basis for $W$, and moreover $$\operatorname{span}\{\mathbf{v}_1,\dots,\mathbf{v}_k\}=\operatorname{span}\{\mathbf{x}_1,\dots,\mathbf{x}_k\}\qquad\text{for each }k=1,\dots,p.$$ Dividing each $\mathbf{v}_k$ by its length gives an orthonormal basis $\{\mathbf{q}_1,\dots,\mathbf{q}_p\}$, $\ \mathbf{q}_k=\dfrac{\mathbf{v}_k}{\|\mathbf{v}_k\|}$.

Why it works: suppose $\mathbf{v}_1,\dots,\mathbf{v}_{k-1}$ are already orthogonal. Dotting the formula for $\mathbf{v}_k$ with $\mathbf{v}_j$ ($j\lt k$) kills every subtracted term except the $j$-th, and that one cancels $\mathbf{x}_k\cdot\mathbf{v}_j$ exactly, as in the two-vector calculation above. And $\mathbf{v}_k\ne\mathbf{0}$, because otherwise $\mathbf{x}_k$ would lie in $\operatorname{span}\{\mathbf{v}_1,\dots,\mathbf{v}_{k-1}\}=\operatorname{span}\{\mathbf{x}_1,\dots,\mathbf{x}_{k-1}\}$, contradicting independence.

🧭 Problem-solving strategy — running Gram–Schmidt by hand
  1. Start with a basis $\mathbf{x}_1,\dots,\mathbf{x}_p$ of $W$. (If you are given a spanning set that might be dependent, just run the process: a $\mathbf{v}_k$ that comes out as $\mathbf{0}$ tells you $\mathbf{x}_k$ was redundant — drop it and carry on.)
  2. Set $\mathbf{v}_1=\mathbf{x}_1$. For each later $\mathbf{x}_k$, compute the coefficients $\dfrac{\mathbf{x}_k\cdot\mathbf{v}_j}{\mathbf{v}_j\cdot\mathbf{v}_j}$ for every earlier $\mathbf{v}_j$ and subtract.
  3. Clear fractions as you go. You may replace any $\mathbf{v}_k$ by a nonzero multiple of itself (e.g. $(\tfrac12,-\tfrac12,1)\to(1,-1,2)$) before using it in later steps: the projection onto a line does not depend on which vector along the line you use.
  4. Check that every pair $\mathbf{v}_i\cdot\mathbf{v}_j$ is $0$ — it catches almost every arithmetic slip.
  5. Normalise last, if an orthonormal basis is wanted: $\mathbf{q}_k=\mathbf{v}_k/\|\mathbf{v}_k\|$.
✏️ Example 2 — An orthonormal basis of $\mathbb{R}^3$
Apply the Gram–Schmidt process to $\mathbf{x}_1=(1,1,0)$, $\mathbf{x}_2=(1,0,1)$, $\mathbf{x}_3=(0,1,1)$, and normalise to obtain an orthonormal basis of $\mathbb{R}^3$.
$\mathbf{v}_1$ $\mathbf{v}_1=\mathbf{x}_1=(1,1,0)$, $\ \mathbf{v}_1\cdot\mathbf{v}_1=2$.
$\mathbf{v}_2$ $\mathbf{x}_2\cdot\mathbf{v}_1=1$, so $\mathbf{v}_2=(1,0,1)-\tfrac12(1,1,0)=\left(\tfrac12,-\tfrac12,1\right)$. Scale by $2$: use $\mathbf{v}_2=(1,-1,2)$, $\ \mathbf{v}_2\cdot\mathbf{v}_2=6$.
$\mathbf{v}_3$ $\mathbf{x}_3\cdot\mathbf{v}_1=1$ and $\mathbf{x}_3\cdot\mathbf{v}_2=0-1+2=1$, so $$\mathbf{v}_3=(0,1,1)-\tfrac12(1,1,0)-\tfrac16(1,-1,2)=\left(-\tfrac23,\tfrac23,\tfrac23\right).$$ Scale by $\tfrac32$: use $\mathbf{v}_3=(-1,1,1)$.
Check $\mathbf{v}_1\cdot\mathbf{v}_2=1-1+0=0$, $\ \mathbf{v}_1\cdot\mathbf{v}_3=-1+1+0=0$, $\ \mathbf{v}_2\cdot\mathbf{v}_3=-1-1+2=0$. ✓
Normalise Lengths $\sqrt2,\ \sqrt6,\ \sqrt3$: $$\mathbf{q}_1=\frac{1}{\sqrt2}\begin{bmatrix}1\\1\\0\end{bmatrix},\quad\mathbf{q}_2=\frac{1}{\sqrt6}\begin{bmatrix}1\\-1\\2\end{bmatrix},\quad\mathbf{q}_3=\frac{1}{\sqrt3}\begin{bmatrix}-1\\1\\1\end{bmatrix}.$$
Interactive 1 — Gram–Schmidt, One Step at a Time

The given vectors $\mathbf{x}_k$ are grey. At each stage the orange arrow is the projection of the next $\mathbf{x}_k$ onto an earlier $\mathbf{v}_j$ (the dashed line drops at a right angle); subtracting all of them slides $\mathbf{x}_k$ onto the new, perpendicular $\mathbf{v}_k$. Finally every $\mathbf{v}_k$ shrinks or stretches to length $1$. Press Play, or step through with Next step. In $\mathbb{R}^3$ use the slider to turn the view.

stage—
$\mathbf{v}_1$—
$\mathbf{v}_2$—
$\mathbf{v}_3$—
largest $|\mathbf{v}_i\cdot\mathbf{v}_j|$, $i\ne j$—
✏️ Example 3 — Gram–Schmidt in $\mathbb{R}^4$
Find an orthogonal basis, and then an orthonormal basis, for $W=\operatorname{span}\{\mathbf{x}_1,\mathbf{x}_2,\mathbf{x}_3\}\subseteq\mathbb{R}^4$, where $$\mathbf{x}_1=\begin{bmatrix}1\\1\\1\\-1\end{bmatrix},\quad\mathbf{x}_2=\begin{bmatrix}0\\2\\2\\0\end{bmatrix},\quad\mathbf{x}_3=\begin{bmatrix}0\\1\\2\\1\end{bmatrix}.$$
$\mathbf{v}_1$ $\mathbf{v}_1=\mathbf{x}_1=(1,1,1,-1)$, $\ \mathbf{v}_1\cdot\mathbf{v}_1=4$.
$\mathbf{v}_2$ $\mathbf{x}_2\cdot\mathbf{v}_1=0+2+2+0=4$, so $\mathbf{v}_2=\mathbf{x}_2-\tfrac44\mathbf{v}_1=(0,2,2,0)-(1,1,1,-1)=(-1,1,1,1)$, $\ \mathbf{v}_2\cdot\mathbf{v}_2=4$.
$\mathbf{v}_3$ $\mathbf{x}_3\cdot\mathbf{v}_1=0+1+2-1=2$ and $\mathbf{x}_3\cdot\mathbf{v}_2=0+1+2+1=4$, so $$\mathbf{v}_3=\mathbf{x}_3-\tfrac24\mathbf{v}_1-\tfrac44\mathbf{v}_2=\begin{bmatrix}0\\1\\2\\1\end{bmatrix}-\begin{bmatrix}\tfrac12\\\tfrac12\\\tfrac12\\-\tfrac12\end{bmatrix}-\begin{bmatrix}-1\\1\\1\\1\end{bmatrix}=\begin{bmatrix}\tfrac12\\-\tfrac12\\\tfrac12\\\tfrac12\end{bmatrix}.$$ Scale by $2$: $\mathbf{v}_3=(1,-1,1,1)$.
Check $\mathbf{v}_1\cdot\mathbf{v}_2=-1+1+1-1=0$, $\ \mathbf{v}_1\cdot\mathbf{v}_3=1-1+1-1=0$, $\ \mathbf{v}_2\cdot\mathbf{v}_3=-1-1+1+1=0$. ✓ Orthogonal basis: $\{(1,1,1,-1),\ (-1,1,1,1),\ (1,-1,1,1)\}$.
Normalise Each vector has length $\sqrt4=2$, so $\mathbf{q}_1=\tfrac12(1,1,1,-1)$, $\ \mathbf{q}_2=\tfrac12(-1,1,1,1)$, $\ \mathbf{q}_3=\tfrac12(1,-1,1,1)$.
✏️ Example 4 — An orthonormal basis of $P_2$
Give $P_2$ the inner product $\langle p,q\rangle=p(0)q(0)+p(1)q(1)+p(2)q(2)$. Apply the Gram–Schmidt process to the basis $1,\ t,\ t^2$, then normalise.
Values Work with values at $t=0,1,2$: $\ 1\mapsto(1,1,1)$, $\ t\mapsto(0,1,2)$, $\ t^2\mapsto(0,1,4)$. Every inner product is a dot product of these lists.
$p_1$ $p_1=1$, $\ \langle p_1,p_1\rangle=3$.
$p_2$ $\langle t,1\rangle=0+1+2=3$, so $p_2=t-\tfrac33\cdot1=t-1$, with values $(-1,0,1)$ and $\langle p_2,p_2\rangle=2$.
$p_3$ $\langle t^2,1\rangle=0+1+4=5$ and $\langle t^2,t-1\rangle=0+0+4=4$, so $$p_3=t^2-\tfrac53\cdot1-\tfrac42(t-1)=t^2-2t+\tfrac13,$$ with values $\left(\tfrac13,-\tfrac23,\tfrac13\right)$. Check: $\langle p_3,1\rangle=\tfrac13-\tfrac23+\tfrac13=0$ and $\langle p_3,p_2\rangle=-\tfrac13+0+\tfrac13=0$. ✓
Normalise $\|p_1\|=\sqrt3$, $\ \|p_2\|=\sqrt2$, $\ \|p_3\|=\sqrt{\tfrac19+\tfrac49+\tfrac19}=\tfrac{\sqrt6}{3}$. Orthonormal basis: $$\frac{1}{\sqrt3},\qquad\frac{t-1}{\sqrt2},\qquad\frac{3t^2-6t+1}{\sqrt6}.$$

Orthogonal polynomials (uses calculus). With the integral inner product $\langle f,g\rangle=\int_{-1}^1f(t)g(t)\,dt$ on $C[-1,1]$, Gram–Schmidt turns $1,t,t^2,t^3$ into $1,\ t,\ t^2-\tfrac13,\ t^3-\tfrac35t$ (Exercise III.5). Up to scaling these are the Legendre polynomials, used by engineers to approximate functions and by computers to evaluate integrals very accurately (Gaussian quadrature).

⚠️ Common mistakes in Gram–Schmidt

1. Projecting onto the old vectors. Subtract projections onto the new orthogonal vectors $\mathbf{v}_1,\dots,\mathbf{v}_{k-1}$, never onto $\mathbf{x}_1,\dots,\mathbf{x}_{k-1}$. The $\mathbf{x}$'s are not orthogonal, so subtracting their projections does not produce a vector perpendicular to them.

2. Forgetting a term. $\mathbf{v}_3$ needs two subtractions, $\mathbf{v}_4$ three, and so on.

3. Mixing up normalised and unnormalised formulas. If you normalise as you go, the projection onto $\mathbf{q}_j$ is $(\mathbf{x}_k\cdot\mathbf{q}_j)\,\mathbf{q}_j$ — no denominator, because $\mathbf{q}_j\cdot\mathbf{q}_j=1$. Using $\dfrac{\mathbf{x}_k\cdot\mathbf{v}_j}{\mathbf{v}_j\cdot\mathbf{v}_j}$ with the unnormalised $\mathbf{v}_j$ is equally correct; just do not combine a $\mathbf{q}$ in the numerator with a $\mathbf{v}$ elsewhere.

4. The order matters. Gram–Schmidt on $(1,1),(1,0)$ gives $(1,1),\left(\tfrac12,-\tfrac12\right)$; on $(1,0),(1,1)$ it gives $(1,0),(0,1)$. Both answers are correct orthogonal bases of $\mathbb{R}^2$ — a subspace has many.

The QR Factorisation

Run Gram–Schmidt on the columns $\mathbf{a}_1,\dots,\mathbf{a}_n$ of a matrix $A$ and normalise. Because $\operatorname{span}\{\mathbf{q}_1,\dots,\mathbf{q}_k\}=\operatorname{span}\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$, each column $\mathbf{a}_k$ is a combination of $\mathbf{q}_1,\dots,\mathbf{q}_k$ only: $$\mathbf{a}_k=r_{1k}\mathbf{q}_1+r_{2k}\mathbf{q}_2+\dots+r_{kk}\mathbf{q}_k,\qquad r_{jk}=\mathbf{q}_j\cdot\mathbf{a}_k$$ (the coefficients come from Theorem 8.5, since the $\mathbf{q}$'s are orthonormal). Collecting these equations column by column gives a matrix identity.

📐 Theorem 8.8 — The QR factorisation
Let $A$ be an $m\times n$ matrix whose columns are linearly independent. Then $A$ can be factorised as $$A=QR,$$ where $Q$ is an $m\times n$ matrix whose columns form an orthonormal basis of $\operatorname{Col}A$, and $R$ is an $n\times n$ upper triangular invertible matrix with positive entries on its diagonal. Since $Q^{T}Q=I$, the factor $R$ can be computed as $$R=Q^{T}A,\qquad r_{jk}=\mathbf{q}_j\cdot\mathbf{a}_k .$$

The diagonal entries are $r_{kk}=\|\mathbf{v}_k\|\gt0$, the lengths of the Gram–Schmidt vectors before normalising; the entries below the diagonal are $0$ because $\mathbf{a}_k$ needs no $\mathbf{q}_j$ with $j\gt k$.

✏️ Example 5 — A QR factorisation
Find a QR factorisation of $A=\begin{bmatrix}1&0&0\\1&2&1\\1&2&2\\-1&0&1\end{bmatrix}$, whose columns are the vectors $\mathbf{x}_1,\mathbf{x}_2,\mathbf{x}_3$ of Example 3.
$Q$ Example 3 produced the orthonormal basis $\mathbf{q}_1=\tfrac12(1,1,1,-1)$, $\mathbf{q}_2=\tfrac12(-1,1,1,1)$, $\mathbf{q}_3=\tfrac12(1,-1,1,1)$ of $\operatorname{Col}A$, so $$Q=\frac12\begin{bmatrix}1&-1&1\\1&1&-1\\1&1&1\\-1&1&1\end{bmatrix}.$$
$R=Q^{T}A$ $r_{jk}=\mathbf{q}_j\cdot\mathbf{a}_k$: $$R=\frac12\begin{bmatrix}1&1&1&-1\\-1&1&1&1\\1&-1&1&1\end{bmatrix}\begin{bmatrix}1&0&0\\1&2&1\\1&2&2\\-1&0&1\end{bmatrix}=\frac12\begin{bmatrix}4&4&2\\0&4&4\\0&0&2\end{bmatrix}=\begin{bmatrix}2&2&1\\0&2&2\\0&0&1\end{bmatrix}.$$ The zeros below the diagonal appear automatically — a good check. The diagonal $2,2,1$ is the list of lengths of the unscaled $\mathbf{v}_1,\mathbf{v}_2$ and $\left(\tfrac12,-\tfrac12,\tfrac12,\tfrac12\right)$.
Check Column 3 of $QR$ is $1\,\mathbf{q}_1+2\,\mathbf{q}_2+1\,\mathbf{q}_3=\left(\tfrac12-1+\tfrac12,\ \tfrac12+1-\tfrac12,\ \tfrac12+1+\tfrac12,\ -\tfrac12+1+\tfrac12\right)=(0,1,2,1)=\mathbf{a}_3$. ✓ (Columns 1 and 2 check the same way.)
Interactive 2 — Checking $A=QR$ Live

The matrix $A$ is factorised by Gram–Schmidt into $Q$ (orthonormal columns) times $R$ (upper triangular). The slider changes one entry of $A$ (outlined); Play wobbles it continuously. Watch $Q$ and $R$ change while three things never do: $Q^{T}Q=I$, the zeros below the diagonal of $R$ (green outlines), and $QR=A$. Blue cells are positive, red cells negative; darker means larger.

0
outlined entry of $A$—
diagonal of $R$—
largest $|Q^{T}Q-I|$ entry—
largest $|QR-A|$ entry—

Using the QR Factorisation

Suppose $A=QR$ and we want to solve $A\mathbf{x}=\mathbf{b}$. Multiply $QR\mathbf{x}=\mathbf{b}$ on the left by $Q^{T}$ and use $Q^{T}Q=I$: $$R\mathbf{x}=Q^{T}\mathbf{b}.$$ Since $R$ is upper triangular, this is solved by back substitution — no row reduction and no inverse. If $A$ is square (and invertible), the solution of $R\mathbf{x}=Q^{T}\mathbf{b}$ is the solution of $A\mathbf{x}=\mathbf{b}$. If $A$ has more rows than columns, the same $\mathbf{x}$ solves $A\mathbf{x}=\mathbf{b}$ whenever $\mathbf{b}$ is in $\operatorname{Col}A$ — and when $\mathbf{b}$ is not in $\operatorname{Col}A$, it is the best approximate (least-squares) solution, as Section 8.3 will show.

✏️ Example 6 — Solving a system with $A=QR$
With $A=QR$ from Example 5, solve $A\mathbf{x}=\mathbf{b}$ for $\mathbf{b}=(2,3,6,1)$.
$Q^{T}\mathbf{b}$ $\mathbf{q}_1\cdot\mathbf{b}=\tfrac12(2+3+6-1)=5$, $\ \mathbf{q}_2\cdot\mathbf{b}=\tfrac12(-2+3+6+1)=4$, $\ \mathbf{q}_3\cdot\mathbf{b}=\tfrac12(2-3+6+1)=3$.
Back substitution $$\begin{bmatrix}2&2&1\\0&2&2\\0&0&1\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}5\\4\\3\end{bmatrix}:\qquad x_3=3,\quad 2x_2+2(3)=4\Rightarrow x_2=-1,\quad 2x_1+2(-1)+3=5\Rightarrow x_1=2.$$
Check $A$ is $4\times3$, so we must confirm that $\mathbf{x}=(2,-1,3)$ really solves the system: $2\mathbf{a}_1-\mathbf{a}_2+3\mathbf{a}_3=(2,2,2,-2)+(0,-2,-2,0)+(0,3,6,3)=(2,3,6,1)=\mathbf{b}$. ✓

Exercises

I
Gram–Schmidt in $\mathbb{R}^2$ and $\mathbb{R}^3$. Find an orthogonal basis for the span of the given vectors, then an orthonormal one.
  • 1 $\mathbf{x}_1=(3,4)$, $\ \mathbf{x}_2=(1,0)$
  • 2 $\mathbf{x}_1=(1,2)$, $\ \mathbf{x}_2=(4,3)$
  • 3 $\mathbf{x}_1=(1,1,1)$, $\ \mathbf{x}_2=(0,1,2)$
  • 4 $\mathbf{x}_1=(1,2,2)$, $\ \mathbf{x}_2=(3,3,0)$
  • 5 $\mathbf{x}_1=(1,1,1)$, $\ \mathbf{x}_2=(1,1,0)$, $\ \mathbf{x}_3=(1,0,0)$
  • 6 The plane $x-y+2z=0$ in $\mathbb{R}^3$ (find a basis first).
  • 7 $\operatorname{Col}A$ for $A=\begin{bmatrix}1&3\\1&1\\1&1\end{bmatrix}$.
  • 8 $\mathbf{x}_1=(1,2,3)$, $\ \mathbf{x}_2=(2,4,6)$. What goes wrong, and why?
Check: exact Gram–Schmidt (fractions) with the orthogonality test
1 $\mathbf{v}_1=(3,4)$; $\mathbf{v}_2=(1,0)-\tfrac{3}{25}(3,4)=\left(\tfrac{16}{25},-\tfrac{12}{25}\right)$, rescaled $(4,-3)$. Orthonormal: $\left(\tfrac35,\tfrac45\right)$, $\left(\tfrac45,-\tfrac35\right)$.
2 $\mathbf{v}_1=(1,2)$; $\mathbf{x}_2\cdot\mathbf{v}_1=10$, $\mathbf{v}_1\cdot\mathbf{v}_1=5$: $\mathbf{v}_2=(4,3)-2(1,2)=(2,-1)$. Orthonormal: $\tfrac1{\sqrt5}(1,2)$, $\tfrac1{\sqrt5}(2,-1)$.
3 $\mathbf{x}_2\cdot\mathbf{v}_1=3$, $\mathbf{v}_1\cdot\mathbf{v}_1=3$: $\mathbf{v}_2=(0,1,2)-(1,1,1)=(-1,0,1)$. Orthonormal: $\tfrac1{\sqrt3}(1,1,1)$, $\tfrac1{\sqrt2}(-1,0,1)$.
4 $\mathbf{x}_2\cdot\mathbf{v}_1=9$, $\mathbf{v}_1\cdot\mathbf{v}_1=9$: $\mathbf{v}_2=(3,3,0)-(1,2,2)=(2,1,-2)$. Both have length $3$: $\tfrac13(1,2,2)$, $\tfrac13(2,1,-2)$.
5 $\mathbf{v}_1=(1,1,1)$. $\mathbf{v}_2=(1,1,0)-\tfrac23(1,1,1)=\left(\tfrac13,\tfrac13,-\tfrac23\right)$, rescaled $(1,1,-2)$. $\mathbf{x}_3\cdot\mathbf{v}_1=1$, $\mathbf{x}_3\cdot\mathbf{v}_2=1$: $\mathbf{v}_3=(1,0,0)-\tfrac13(1,1,1)-\tfrac16(1,1,-2)=\left(\tfrac12,-\tfrac12,0\right)$, rescaled $(1,-1,0)$. Orthonormal: $\tfrac1{\sqrt3}(1,1,1)$, $\tfrac1{\sqrt6}(1,1,-2)$, $\tfrac1{\sqrt2}(1,-1,0)$.
6 $x=y-2z$ gives the basis $\mathbf{x}_1=(1,1,0)$, $\mathbf{x}_2=(-2,0,1)$. $\mathbf{x}_2\cdot\mathbf{x}_1=-2$, so $\mathbf{v}_2=(-2,0,1)+(1,1,0)=(-1,1,1)$ (check: it satisfies $-1-1+2=0$). Orthonormal: $\tfrac1{\sqrt2}(1,1,0)$, $\tfrac1{\sqrt3}(-1,1,1)$.
7 The columns $(1,1,1)$, $(3,1,1)$ are independent. $\mathbf{v}_2=(3,1,1)-\tfrac53(1,1,1)=\left(\tfrac43,-\tfrac23,-\tfrac23\right)$, rescaled $(2,-1,-1)$. Orthonormal: $\tfrac1{\sqrt3}(1,1,1)$, $\tfrac1{\sqrt6}(2,-1,-1)$.
8 $\mathbf{v}_2=(2,4,6)-\tfrac{28}{14}(1,2,3)=\mathbf{0}$. The process "fails" because $\mathbf{x}_2=2\mathbf{x}_1$: the vectors are dependent, the span is a line, and $\left\{\tfrac1{\sqrt{14}}(1,2,3)\right\}$ is an orthonormal basis for it.
II
Gram–Schmidt in $\mathbb{R}^4$.
  • 1 $\mathbf{x}_1=(1,1,0,0)$, $\ \mathbf{x}_2=(1,0,1,0)$
  • 2 $\mathbf{x}_1=(-1,0,1,1)$, $\ \mathbf{x}_2=(0,1,1,2)$, $\ \mathbf{x}_3=(0,0,2,2)$
  • 3 Find an orthonormal basis for the subspace $x_1+x_2+x_3+x_4=0$ of $\mathbb{R}^4$.
  • 4 $\mathbf{x}_1=(1,-1,1,-1)$, $\ \mathbf{x}_2=(2,0,2,0)$, $\ \mathbf{x}_3=(2,1,1,2)$
  • 5 $\mathbf{x}_1=(1,1,1,1)$, $\ \mathbf{x}_2=(1,2,3,4)$, $\ \mathbf{x}_3=(2,3,4,5)$. What does Gram–Schmidt reveal about these vectors?
  • 6 Extend the orthogonal set $\{(1,1,1,1),\ (-3,-1,1,3)\}$ to an orthogonal basis of $\mathbb{R}^4$.
1 $\mathbf{v}_2=(1,0,1,0)-\tfrac12(1,1,0,0)=\left(\tfrac12,-\tfrac12,1,0\right)$, rescaled $(1,-1,2,0)$. Orthonormal: $\tfrac1{\sqrt2}(1,1,0,0)$, $\tfrac1{\sqrt6}(1,-1,2,0)$.
2 $\mathbf{x}_2\cdot\mathbf{v}_1=3$, $\mathbf{v}_1\cdot\mathbf{v}_1=3$: $\mathbf{v}_2=(0,1,1,2)-(-1,0,1,1)=(1,1,0,1)$. $\mathbf{x}_3\cdot\mathbf{v}_1=4$, $\mathbf{x}_3\cdot\mathbf{v}_2=2$, $\mathbf{v}_2\cdot\mathbf{v}_2=3$: $\mathbf{v}_3=(0,0,2,2)-\tfrac43(-1,0,1,1)-\tfrac23(1,1,0,1)=\left(\tfrac23,-\tfrac23,\tfrac23,0\right)$, rescaled $(1,-1,1,0)$. All three have length $\sqrt3$: orthonormal basis $\tfrac1{\sqrt3}(-1,0,1,1)$, $\tfrac1{\sqrt3}(1,1,0,1)$, $\tfrac1{\sqrt3}(1,-1,1,0)$.
3 With $x_2,x_3,x_4$ free: basis $(-1,1,0,0)$, $(-1,0,1,0)$, $(-1,0,0,1)$. Gram–Schmidt: $\mathbf{v}_1=(-1,1,0,0)$; $\mathbf{v}_2=(-1,0,1,0)-\tfrac12\mathbf{v}_1=\left(-\tfrac12,-\tfrac12,1,0\right)\to(-1,-1,2,0)$; $\mathbf{v}_3=(-1,0,0,1)-\tfrac12\mathbf{v}_1-\tfrac16(-1,-1,2,0)=\left(-\tfrac13,-\tfrac13,-\tfrac13,1\right)\to(-1,-1,-1,3)$. Orthonormal: $\tfrac1{\sqrt2}(-1,1,0,0)$, $\tfrac1{\sqrt6}(-1,-1,2,0)$, $\tfrac1{2\sqrt3}(-1,-1,-1,3)$.
4 $\mathbf{x}_2\cdot\mathbf{v}_1=4$, $\mathbf{v}_1\cdot\mathbf{v}_1=4$: $\mathbf{v}_2=(2,0,2,0)-(1,-1,1,-1)=(1,1,1,1)$. $\mathbf{x}_3\cdot\mathbf{v}_1=0$, $\mathbf{x}_3\cdot\mathbf{v}_2=6$: $\mathbf{v}_3=(2,1,1,2)-\tfrac64(1,1,1,1)=\left(\tfrac12,-\tfrac12,-\tfrac12,\tfrac12\right)\to(1,-1,-1,1)$. Orthonormal: halve each of $(1,-1,1,-1)$, $(1,1,1,1)$, $(1,-1,-1,1)$.
5 $\mathbf{v}_2=(1,2,3,4)-\tfrac{10}{4}(1,1,1,1)=\left(-\tfrac32,-\tfrac12,\tfrac12,\tfrac32\right)\to(-3,-1,1,3)$. Then $\mathbf{x}_3\cdot\mathbf{v}_1=14$, $\mathbf{x}_3\cdot(-3,-1,1,3)=10$, and $\mathbf{v}_3=(2,3,4,5)-\tfrac{14}{4}(1,1,1,1)-\tfrac{10}{20}(-3,-1,1,3)=\mathbf{0}$. So $\mathbf{x}_3$ lies in the span of $\mathbf{x}_1,\mathbf{x}_2$ (indeed $\mathbf{x}_3=\mathbf{x}_1+\mathbf{x}_2$); the span is 2-dimensional with orthogonal basis $\{(1,1,1,1),(-3,-1,1,3)\}$.
6 The missing vectors form the orthogonal complement $\operatorname{Nul}\begin{bmatrix}1&1&1&1\\-3&-1&1&3\end{bmatrix}$. $R_2\to R_2+3R_1$, $R_2\to\tfrac12R_2$, $R_1\to R_1-R_2$ give $\begin{bmatrix}1&0&-1&-2\\0&1&2&3\end{bmatrix}$, so a basis is $(1,-2,1,0)$, $(2,-3,0,1)$. Gram–Schmidt on these: $(2,-3,0,1)-\tfrac{8}{6}(1,-2,1,0)=\left(\tfrac23,-\tfrac13,-\tfrac43,1\right)\to(2,-1,-4,3)$. Orthogonal basis of $\mathbb{R}^4$: $\{(1,1,1,1),\ (-3,-1,1,3),\ (1,-2,1,0),\ (2,-1,-4,3)\}$. (Another valid answer: $(1,-1,-1,1)$ and $(-1,3,-3,1)$.)
III
Other inner products.
  • 1 On $P_2$ with $\langle p,q\rangle=p(-1)q(-1)+p(0)q(0)+p(1)q(1)$, apply Gram–Schmidt to $1,t,t^2$.
  • 2 With the inner product of part 1, apply Gram–Schmidt to $1+t$, $t^2$ (in that order).
  • 3 On $\mathbb{R}^2$ with $\langle\mathbf{u},\mathbf{v}\rangle=2u_1v_1+u_2v_2$, apply Gram–Schmidt to $(1,1)$, $(1,0)$ and normalise.
  • 4 (uses calculus) On $C[0,1]$ with $\langle f,g\rangle=\displaystyle\int_0^1f(t)g(t)\,dt$, apply Gram–Schmidt to $1,t,t^2$.
  • 5 (uses calculus) On $C[-1,1]$ with $\langle f,g\rangle=\displaystyle\int_{-1}^1f(t)g(t)\,dt$, apply Gram–Schmidt to $1,t,t^2,t^3$.
1 Values at $-1,0,1$: $1\mapsto(1,1,1)$, $t\mapsto(-1,0,1)$, $t^2\mapsto(1,0,1)$. $\langle t,1\rangle=0$, so $p_2=t$. $\langle t^2,1\rangle=2$, $\langle1,1\rangle=3$, $\langle t^2,t\rangle=0$: $p_3=t^2-\tfrac23$. Orthogonal basis $1,\ t,\ t^2-\tfrac23$ (compare 8.1 Example 2).
2 $1+t\mapsto(0,1,2)$, $\langle1+t,1+t\rangle=5$, $\langle t^2,1+t\rangle=0+0+2=2$: $p_2=t^2-\tfrac25(1+t)=t^2-\tfrac25t-\tfrac25$. Check: values $\left(1,-\tfrac25,\tfrac15\right)$, and $(0,1,2)\cdot\left(1,-\tfrac25,\tfrac15\right)=0$. ✓
3 $\langle(1,0),(1,1)\rangle=2$, $\langle(1,1),(1,1)\rangle=3$: $\mathbf{v}_2=(1,0)-\tfrac23(1,1)=\left(\tfrac13,-\tfrac23\right)\to(1,-2)$; check $\langle(1,1),(1,-2)\rangle=2-2=0$. Norms $\sqrt3$ and $\sqrt{2+4}=\sqrt6$: orthonormal basis $\tfrac1{\sqrt3}(1,1)$, $\tfrac1{\sqrt6}(1,-2)$. (They are not perpendicular for the dot product.)
4 $\langle t,1\rangle=\tfrac12$, $\langle1,1\rangle=1$: $p_2=t-\tfrac12$. $\langle t^2,1\rangle=\tfrac13$; $\langle t^2,t-\tfrac12\rangle=\tfrac14-\tfrac16=\tfrac1{12}$; $\langle t-\tfrac12,t-\tfrac12\rangle=\tfrac1{12}$: $p_3=t^2-\tfrac13-\left(t-\tfrac12\right)=t^2-t+\tfrac16$.
5 Odd times even integrates to $0$ on $[-1,1]$, so many terms vanish. $p_1=1$, $p_2=t$, $p_3=t^2-\dfrac{\langle t^2,1\rangle}{\langle1,1\rangle}=t^2-\dfrac{2/3}{2}=t^2-\tfrac13$, $p_4=t^3-\dfrac{\langle t^3,t\rangle}{\langle t,t\rangle}t=t^3-\dfrac{2/5}{2/3}t=t^3-\tfrac35t$.
IV
QR factorisations and their use.
  • 1 Find a QR factorisation of $A=\begin{bmatrix}1&2\\0&1\\1&0\end{bmatrix}$ (see Example 1).
  • 2 Find a QR factorisation of $A=\begin{bmatrix}3&1\\4&0\end{bmatrix}$.
  • 3 Find a QR factorisation of $A=\begin{bmatrix}1&3\\2&3\\2&0\end{bmatrix}$.
  • 4 Find a QR factorisation of $A=\begin{bmatrix}1&2&2\\-1&0&1\\1&2&1\\-1&0&2\end{bmatrix}$ (see Exercise II.4).
  • 5 Use your answer to part 4 to solve $A\mathbf{x}=\mathbf{b}$ with $\mathbf{b}=(3,1,1,3)$, and check the solution.
  • 6 If $A=QR$ where $Q$ has orthonormal columns, show that $R=Q^{T}A$. Why must $R$ be invertible when the columns of $A$ are independent?
  • 7 What is the QR factorisation of an orthogonal matrix $U$? Of a matrix whose columns are orthogonal but not unit vectors?
1 From Example 1, $\mathbf{q}_1=\tfrac1{\sqrt2}(1,0,1)$, $\mathbf{q}_2=\tfrac1{\sqrt3}(1,1,-1)$. $r_{11}=\mathbf{q}_1\cdot\mathbf{a}_1=\sqrt2$, $r_{12}=\mathbf{q}_1\cdot\mathbf{a}_2=\tfrac2{\sqrt2}=\sqrt2$, $r_{22}=\mathbf{q}_2\cdot\mathbf{a}_2=\tfrac3{\sqrt3}=\sqrt3$: $$Q=\begin{bmatrix}\tfrac1{\sqrt2}&\tfrac1{\sqrt3}\\0&\tfrac1{\sqrt3}\\\tfrac1{\sqrt2}&-\tfrac1{\sqrt3}\end{bmatrix},\qquad R=\begin{bmatrix}\sqrt2&\sqrt2\\0&\sqrt3\end{bmatrix}.$$
2 From Exercise I.1, $Q=\begin{bmatrix}\tfrac35&\tfrac45\\\tfrac45&-\tfrac35\end{bmatrix}$, and $R=Q^{T}A=\begin{bmatrix}5&\tfrac35\\0&\tfrac45\end{bmatrix}$. Check column 2: $\tfrac35\left(\tfrac35,\tfrac45\right)+\tfrac45\left(\tfrac45,-\tfrac35\right)=(1,0)$. ✓
3 From Exercise I.4, $Q=\tfrac13\begin{bmatrix}1&2\\2&1\\2&-2\end{bmatrix}$, $R=Q^{T}A=\begin{bmatrix}3&3\\0&3\end{bmatrix}$. Check: $3\mathbf{q}_1+3\mathbf{q}_2=(1,2,2)+(2,1,-2)=(3,3,0)$. ✓
4 From Exercise II.4, $Q=\tfrac12\begin{bmatrix}1&1&1\\-1&1&-1\\1&1&-1\\-1&1&1\end{bmatrix}$ and $R=Q^{T}A=\begin{bmatrix}2&2&0\\0&2&3\\0&0&1\end{bmatrix}$. Check column 3: $0\mathbf{q}_1+3\mathbf{q}_2+\mathbf{q}_3=\left(\tfrac32+\tfrac12,\ \tfrac32-\tfrac12,\ \tfrac32-\tfrac12,\ \tfrac32+\tfrac12\right)=(2,1,1,2)$. ✓
5 $Q^{T}\mathbf{b}=\left(\tfrac{3-1+1-3}{2},\ \tfrac{3+1+1+3}{2},\ \tfrac{3-1-1+3}{2}\right)=(0,4,2)$. Back substitution: $x_3=2$; $2x_2+3(2)=4\Rightarrow x_2=-1$; $2x_1+2(-1)+0=0\Rightarrow x_1=1$. Check: $\mathbf{a}_1-\mathbf{a}_2+2\mathbf{a}_3=(1-2+4,\,-1-0+2,\,1-2+2,\,-1-0+4)=(3,1,1,3)$. ✓
6 Multiply $A=QR$ on the left by $Q^{T}$: $Q^{T}A=Q^{T}QR=IR=R$. $R$ is upper triangular with diagonal entries $r_{kk}=\|\mathbf{v}_k\|$, and every $\mathbf{v}_k\ne\mathbf{0}$ when the columns are independent (Theorem 8.7); a triangular matrix with nonzero diagonal has nonzero determinant, so it is invertible.
7 For orthogonal $U$, Gram–Schmidt leaves the columns unchanged, so $Q=U$ and $R=I$. If the columns $\mathbf{a}_k$ are orthogonal, each $\mathbf{v}_k=\mathbf{a}_k$, so $\mathbf{q}_k=\mathbf{a}_k/\|\mathbf{a}_k\|$ and $R$ is diagonal with entries $\|\mathbf{a}_1\|,\dots,\|\mathbf{a}_n\|$.
V
Thinking about Gram–Schmidt.
  • 1 Explain why the Gram–Schmidt formula for $\mathbf{v}_k$ uses $\mathbf{v}_1,\dots,\mathbf{v}_{k-1}$ and not $\mathbf{x}_1,\dots,\mathbf{x}_{k-1}$. Illustrate with $\mathbf{x}_1=(1,0,0)$, $\mathbf{x}_2=(1,1,0)$, $\mathbf{x}_3=(1,1,1)$.
  • 2 Apply Gram–Schmidt to $(1,1),(1,0)$ and then to $(1,0),(1,1)$. Are the answers the same? Are both correct?
  • 3 Show that $\operatorname{span}\{\mathbf{v}_1,\mathbf{v}_2\}=\operatorname{span}\{\mathbf{x}_1,\mathbf{x}_2\}$ in Theorem 8.7.
  • 4 Surveying. A drone's camera records two direction vectors $\mathbf{x}_1=(2,1,2)$ and $\mathbf{x}_2=(1,1,1)$ (in metres) along the edges of a sloping roof. Find an orthonormal pair of directions in the roof plane, with the first along $\mathbf{x}_1$, so that the roof can be mapped with a right-angled grid of $1$ m squares.
1 Gram–Schmidt gives $\mathbf{v}_1=(1,0,0)$, $\mathbf{v}_2=(1,1,0)-(1,0,0)=(0,1,0)$, $\mathbf{v}_3=(1,1,1)-(1,0,0)-(0,1,0)=(0,0,1)$. Using the old vectors instead: $(1,1,1)-\operatorname{proj}_{\mathbf{x}_1}\mathbf{x}_3-\operatorname{proj}_{\mathbf{x}_2}\mathbf{x}_3=(1,1,1)-(1,0,0)-(1,1,0)=(-1,0,1)$, which is not orthogonal to $\mathbf{x}_1$ ($(-1,0,1)\cdot(1,0,0)=-1$). Subtracting projections onto non-orthogonal directions removes some components twice.
2 First order: $(1,1)$ and $(1,0)-\tfrac12(1,1)=\left(\tfrac12,-\tfrac12\right)$. Second order: $(1,0)$ and $(1,1)-(1,0)=(0,1)$. Different answers, both orthogonal bases of $\mathbb{R}^2$ — Gram–Schmidt depends on the order of the input.
3 $\mathbf{v}_1=\mathbf{x}_1$ and $\mathbf{v}_2=\mathbf{x}_2-c\,\mathbf{x}_1$ are combinations of $\mathbf{x}_1,\mathbf{x}_2$, so $\operatorname{span}\{\mathbf{v}_1,\mathbf{v}_2\}\subseteq\operatorname{span}\{\mathbf{x}_1,\mathbf{x}_2\}$. Conversely $\mathbf{x}_1=\mathbf{v}_1$ and $\mathbf{x}_2=\mathbf{v}_2+c\,\mathbf{v}_1$, giving the reverse inclusion.
4 $\mathbf{x}_2\cdot\mathbf{x}_1=5$, $\mathbf{x}_1\cdot\mathbf{x}_1=9$: $\mathbf{v}_2=(1,1,1)-\tfrac59(2,1,2)=\left(-\tfrac19,\tfrac49,-\tfrac19\right)\to(-1,4,-1)$. Check $(2,1,2)\cdot(-1,4,-1)=-2+4-2=0$. ✓ Normalise: $\mathbf{q}_1=\tfrac13(2,1,2)$, $\mathbf{q}_2=\tfrac{1}{3\sqrt2}(-1,4,-1)$. Grid points are $a\mathbf{q}_1+b\mathbf{q}_2$ with whole numbers $a,b$ (metres).
🔬 Research–Teaching NexusResearch-tutored · optional, about 30 minutes

Research question. Gram–Schmidt is exact on paper. What happens to it inside a computer, which rounds every number — and why do numerical libraries use a "modified" version?

Why it matters. Weather centres, GPS receivers and engineering simulation codes compute QR factorisations of huge matrices millions of times a day. If rounding destroys orthogonality, the computed $Q$ no longer satisfies $Q^{T}Q=I$ and the answers of least-squares problems drift. Understanding this is a first step into numerical analysis.

What to do.

  1. Take $\varepsilon=0.01$ and $\mathbf{x}_1=(1,\varepsilon,0,0)$, $\mathbf{x}_2=(1,0,\varepsilon,0)$, $\mathbf{x}_3=(1,0,0,\varepsilon)$. These are independent but nearly parallel.
  2. Run classical Gram–Schmidt (the formulas of Theorem 8.7, normalising each $\mathbf{v}_k$), rounding every intermediate result to 4 significant digits — as a cheap computer would. Note that $1+\varepsilon^2=1.0001$ rounds to $1.000$.
  3. Run modified Gram–Schmidt: compute $\mathbf{w}=\mathbf{x}_3-(\mathbf{x}_3\cdot\mathbf{q}_1)\mathbf{q}_1$ first, then subtract $(\mathbf{w}\cdot\mathbf{q}_2)\mathbf{q}_2$ using the updated vector $\mathbf{w}$. Round in the same way.
  4. For each method compute $\mathbf{q}_2\cdot\mathbf{q}_3$. Which method keeps the vectors orthogonal?

Tools: Spreadsheet with the ROUND function, or Python

Hand in: A half-page note with the two computed $\mathbf{q}_3$ vectors, the values of $\mathbf{q}_2\cdot\mathbf{q}_3$, and a sentence explaining why the modified version helps.

Going further. Look up Householder reflections, the method most numerical libraries actually use for QR. A Householder matrix $H=I-2\mathbf{u}\mathbf{u}^{T}$ ($\|\mathbf{u}\|=1$) is orthogonal and symmetric — verify both facts.

Instructor note: in exact arithmetic both methods give $\mathbf{q}_2\approx(0.0071,-0.7071,0.7071,0)$ and $\mathbf{q}_3\approx(0.0041,-0.4082,-0.4082,0.8165)$, very close to $\tfrac1{\sqrt2}(0,-1,1,0)$ and $\tfrac1{\sqrt6}(0,-1,-1,2)$. With 4-digit rounding the classical method gives $\mathbf{q}_3\approx(0,-0.7072,0,0.7072)$ and $\mathbf{q}_2\cdot\mathbf{q}_3\approx0.50$ — the vectors are $60^\circ$ apart, not $90^\circ$. The modified method gives $\mathbf{q}_3\approx(0,-0.4081,-0.4082,0.8163)$ and $\mathbf{q}_2\cdot\mathbf{q}_3\approx-0.0001$.

🧪 Self-check quiz

0/5 correct

Q1. Gram–Schmidt on $\mathbf{x}_1=(1,0)$, $\mathbf{x}_2=(1,1)$ gives $\mathbf{v}_2=$

Q2. In a QR factorisation of a matrix with independent columns, $R$ is

Q3. If $A=QR$ and $Q$ has orthonormal columns, then $R$ equals

Q4. Running Gram–Schmidt on $\mathbf{x}_1,\mathbf{x}_2,\mathbf{x}_3$ produces $\mathbf{v}_3=\mathbf{0}$. This means

Q5. Normalising $\mathbf{v}=(2,-1,2)$ gives

← 8.1 Inner Products and Orthogonal Sets8.3 Orthogonal Projections and Least Squares →