Reference

Formula Sheet

Every key formula of the course on one page, grouped by chapter, with a link back to the section that develops it.

Chapter 1 · Systems of Linear Equations

Linear equation; number of solutions§ 1.1 →

$a_1x_1+a_2x_2+\cdots+a_nx_n=b$ ($a_i$, $b$ real constants). Every linear system has no solution, exactly one solution, or infinitely many solutions.

Two equations, two unknowns§ 1.1 →

$a_1x+b_1y=c_1,\ a_2x+b_2y=c_2$ (each with $a_i$, $b_i$ not both $0$): if $a_1b_2-a_2b_1\ne0$, exactly one solution $x=\dfrac{c_1b_2-c_2b_1}{a_1b_2-a_2b_1}$, $y=\dfrac{a_1c_2-a_2c_1}{a_1b_2-a_2b_1}$; if $a_1b_2-a_2b_1=0$, the lines are parallel (no solution) or identical (infinitely many).

Augmented matrix§ 1.1 →

$\begin{aligned}x+2y&=5\\3x+4y&=6\end{aligned}\ \longleftrightarrow\ \left[\begin{array}{cc|c}1&2&5\\3&4&6\end{array}\right]$

Elementary row operations§ 1.1 →

$R_i\leftrightarrow R_j$; $\ R_i\to cR_i$ with $c\ne0$; $\ R_i\to R_i+cR_j$ with $j\ne i$. Row-equivalent augmented matrices give systems with the same solution set.

Back-substitution§ 1.1 →

Triangular system: solve the last equation first, substitute upwards; a variable with no equation of its own becomes a free parameter $t\in\mathbb{R}$.

Echelon forms§ 1.2 →

REF: zero rows at the bottom; each leading entry strictly right of the one above; zeros below leading entries. RREF: in addition every leading entry is $1$ and is the only nonzero entry in its column, e.g. $\begin{bmatrix}1&0&*&0\\0&1&*&0\\0&0&0&1\end{bmatrix}$. Each matrix has exactly one RREF.

Row-reduction algorithm§ 1.2 →

Forward: leftmost nonzero column → pivot (swap if $0$) → zeros below → repeat on lower rows (REF). Backward: from the last pivot up, scale to $1$ and make zeros above (RREF). Gaussian = forward + back-substitution; Gauss–Jordan = forward + backward.

Existence and uniqueness§ 1.2 →

$\left[\,A\mid\mathbf{b}\,\right]$ is consistent $\iff$ no echelon row $\left[\,0\ \cdots\ 0\mid c\,\right]$ with $c\ne0$. If consistent: no free variables $\Rightarrow$ exactly one solution; at least one free variable $\Rightarrow$ infinitely many. Free variables $=$ unknowns $-$ pivots.

Homogeneous systems§ 1.2 →

$A\mathbf{x}=\mathbf{0}$ always has $\mathbf{x}=\mathbf{0}$. If it has $n$ unknowns and $m\lt n$ equations, it has infinitely many (nontrivial) solutions.

Parametric vector form§ 1.2 →

Free variables $=s,t,\dots$: $\ \mathbf{x}=\mathbf{p}+s\mathbf{v}_1+t\mathbf{v}_2+\cdots$, where $\mathbf{p}$ solves $A\mathbf{x}=\mathbf{b}$ and each $\mathbf{v}_i$ solves $A\mathbf{x}=\mathbf{0}$.

Conservation of flow§ 1.3 →

At every junction: total flow in $=$ total flow out. A connected network with $n$ junctions, $m$ branches and known external flows has $m-n+1$ free flows (if consistent).

Circuits: Ohm and Kirchhoff§ 1.3 →

Ohm: $V=IR$. Current law: at each node, $\sum I_{\text{in}}=\sum I_{\text{out}}$. Voltage law: around each closed loop, $\sum E=\sum IR$. A negative current flows opposite to the chosen direction.

Polynomial interpolation§ 1.3 →

$n+1$ points with distinct $x_i$ determine exactly one $p(x)=a_0+a_1x+\cdots+a_nx^n$ (degree $\le n$) with $p(x_i)=y_i$; solve $\left[\begin{array}{cccc|c}1&x_i&\cdots&x_i^n&y_i\end{array}\right]$ (one row per point).

Balancing chemical equations§ 1.3 →

One homogeneous equation per element (atoms in $=$ atoms out); choose the free variable to make all coefficients the smallest positive whole numbers.

Chapter 2 · Matrices

Matrix product§ 2.1 →

$A$ is $m\times n$, $B$ is $n\times p$: $\ (AB)_{ij}=\displaystyle\sum_{k=1}^{n}a_{ik}b_{kj}$, $\ AB=\begin{bmatrix}A\mathbf{b}_1&\cdots&A\mathbf{b}_p\end{bmatrix}$ is $m\times p$

$A\mathbf{x}$ as a combination of columns§ 2.1 →

$A=\begin{bmatrix}\mathbf{a}_1&\cdots&\mathbf{a}_n\end{bmatrix}$: $\ A\mathbf{x}=x_1\mathbf{a}_1+\cdots+x_n\mathbf{a}_n$

Rules for products§ 2.1 →

$A(BC)=(AB)C$, $\ A(B+C)=AB+AC$, $\ (B+C)A=BA+CA$, $\ I_mA=A=AI_n$; in general $AB\ne BA$

Transpose rules§ 2.1 →

$\left(A^{T}\right)^{T}=A$, $\ (A+B)^{T}=A^{T}+B^{T}$, $\ (AB)^{T}=B^{T}A^{T}$

Symmetric matrix§ 2.1 →

$A^{T}=A$ ($a_{ij}=a_{ji}$); $\ A^{T}A$ and $AA^{T}$ are symmetric for every $A$

Powers of a square matrix§ 2.1 →

$A^0=I$, $\ A^rA^s=A^{r+s}$; $\ (A+B)^2=A^2+AB+BA+B^2$

Inverse of a $2\times2$ matrix§ 2.2 →

If $ad-bc\ne0$: $\ \begin{bmatrix}a&b\\c&d\end{bmatrix}^{-1}=\dfrac{1}{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}$; if $ad-bc=0$ the matrix is singular

Properties of inverses§ 2.2 →

$A$, $B$ invertible $n\times n$: $\ \left(A^{-1}\right)^{-1}=A$, $\ (AB)^{-1}=B^{-1}A^{-1}$, $\ \left(A^{T}\right)^{-1}=\left(A^{-1}\right)^{T}$, $\ (cA)^{-1}=\tfrac1cA^{-1}$ ($c\ne0$)

Solving with the inverse§ 2.2 →

$A$ invertible: $A\mathbf{x}=\mathbf{b}$ has the unique solution $\mathbf{x}=A^{-1}\mathbf{b}$

Finding $A^{-1}$ by row reduction§ 2.2 →

$\left[\begin{array}{c|c}A&I\end{array}\right]\to\left[\begin{array}{c|c}I&A^{-1}\end{array}\right]$; a zero row on the left means $A$ is singular

Elementary matrices§ 2.2 →

Row operation on $A$ $=$ $EA$; $\ E^{-1}$ is the reverse operation; $\ E_k\cdots E_1A=I\Rightarrow A^{-1}=E_k\cdots E_1$

LU factorisation§ 2.3 →

If elimination $R_i\to R_i-m_{ij}R_j$ ($j\lt i$) reduces $A$ to upper triangular $U$ with no row swaps: $A=LU$, $L$ unit lower triangular with $\ell_{ij}=m_{ij}$

Solving with $A=LU$§ 2.3 →

$A\mathbf{x}=\mathbf{b}$: solve $L\mathbf{y}=\mathbf{b}$ (forward), then $U\mathbf{x}=\mathbf{y}$ (back); unique solution when every $u_{ii}\ne0$

Row swaps: $PA=LU$§ 2.3 →

Every square $A$ has a permutation matrix $P$ with $PA=LU$; then $A\mathbf{x}=\mathbf{b}\iff LU\mathbf{x}=P\mathbf{b}$; $P^{-1}=P^{T}$

Operation counts ($n\times n$)§ 2.3 →

$A=LU$: $\tfrac23n^3-\tfrac12n^2-\tfrac16n\approx\tfrac23n^3$ flops; forward $n^2-n$, back $n^2$ (pair $\approx2n^2$); $A^{-1}$: $\approx2n^3$

Markov chain§ 2.4 →

$P$ stochastic (entries $\ge0$, columns sum to $1$): $\mathbf{x}_{k+1}=P\mathbf{x}_k$, $\ \mathbf{x}_k=P^k\mathbf{x}_0$

Steady-state vector§ 2.4 →

$P\mathbf{q}=\mathbf{q}$: solve $(P-I)\mathbf{x}=\mathbf{0}$, scale so entries sum to $1$. If $P$ is regular (some $P^k$ all positive), $\mathbf{q}$ is unique and $\mathbf{x}_k\to\mathbf{q}$ from every start

Two-state chain§ 2.4 →

$P=\begin{bmatrix}1-a&b\\a&1-b\end{bmatrix}$, $0\lt a,b\lt1$: $\ \mathbf{q}=\dfrac{1}{a+b}\begin{bmatrix}b\\a\end{bmatrix}$

Leontief input–output model§ 2.4 →

$\mathbf{x}=C\mathbf{x}+\mathbf{d}\iff(I-C)\mathbf{x}=\mathbf{d}$. If $C\ge0$ and every column sum $\lt1$: $\mathbf{x}=(I-C)^{-1}\mathbf{d}\ge\mathbf{0}$, $\ (I-C)^{-1}=I+C+C^2+\cdots$

Chapter 3 · Determinants

2×2 determinant§ 3.1 →

$\begin{vmatrix}a&b\\c&d\end{vmatrix}=ad-bc$

Minor and cofactor§ 3.1 →

$M_{ij}=\det$ of $A$ with row $i$ and column $j$ deleted; $\ C_{ij}=(-1)^{i+j}M_{ij}$

Cofactor expansion (any row $i$ or column $j$)§ 3.1 →

$A$ is $n\times n$, $n\ge2$: $\ \det A=\displaystyle\sum_{k=1}^{n}a_{ik}C_{ik}=\sum_{k=1}^{n}a_{kj}C_{kj}$

Rule of Sarrus ($3\times3$ only)§ 3.1 →

$\det A=a_{11}a_{22}a_{33}+a_{12}a_{23}a_{31}+a_{13}a_{21}a_{32}-a_{13}a_{22}a_{31}-a_{11}a_{23}a_{32}-a_{12}a_{21}a_{33}$

Triangular matrix§ 3.1 →

$A$ upper or lower triangular: $\ \det A=a_{11}a_{22}\cdots a_{nn}$; $\ \det I_n=1$

Row operations and $\det$§ 3.2 →

$R_i\leftrightarrow R_j$: $\det\to-\det$; $\ R_i\to kR_i$: $\det\to k\det$; $\ R_i\to R_i+kR_j$ ($i\ne j$): $\det$ unchanged. Same for columns.

Invertibility test§ 3.2 →

$A$ square: $\ A$ invertible $\iff\det A\ne0$

Products and transposes§ 3.2 →

$A,B$ both $n\times n$: $\ \det(AB)=\det A\,\det B$, $\ \det A^{T}=\det A$

Inverses, powers, scalar multiples§ 3.2 →

$A$ is $n\times n$: $\ \det(A^{-1})=\dfrac1{\det A}$ (if $A$ invertible), $\ \det(A^m)=(\det A)^m$, $\ \det(kA)=k^n\det A$

Determinant by row reduction§ 3.2 →

Reduce to triangular $U$ with $r$ interchanges, taking factors $k_1,\dots,k_s$ out of rows: $\ \det A=(-1)^r\,k_1\cdots k_s\,u_{11}\cdots u_{nn}$

Vandermonde determinant§ 3.2 →

$\begin{vmatrix}1&a&a^2\\1&b&b^2\\1&c&c^2\end{vmatrix}=(b-a)(c-a)(c-b)$

Cramer's rule§ 3.3 →

$A$ is $n\times n$ with $\det A\ne0$: the solution of $A\mathbf{x}=\mathbf{b}$ is $x_i=\dfrac{\det A_i(\mathbf{b})}{\det A}$, where $A_i(\mathbf{b})$ is $A$ with column $i$ replaced by $\mathbf{b}$

Adjugate and inverse§ 3.3 →

$\operatorname{adj}A=[C_{ij}]^{T}$, $\ A\,\operatorname{adj}A=(\det A)I$; if $\det A\ne0$: $\ A^{-1}=\dfrac{1}{\det A}\operatorname{adj}A$

Area in the plane§ 3.3 →

Parallelogram on $\mathbf{u},\mathbf{v}$: $\left|\det\begin{bmatrix}\mathbf{u}&\mathbf{v}\end{bmatrix}\right|$; triangle with vertices $(x_k,y_k)$: $\dfrac12\left|\det\begin{bmatrix}x_1&y_1&1\\x_2&y_2&1\\x_3&y_3&1\end{bmatrix}\right|$

Volume in space§ 3.3 →

Parallelepiped on $\mathbf{u},\mathbf{v},\mathbf{w}$: $\left|\det\begin{bmatrix}\mathbf{u}&\mathbf{v}&\mathbf{w}\end{bmatrix}\right|$; tetrahedron $\mathbf{0},\mathbf{u},\mathbf{v},\mathbf{w}$: $\tfrac16$ of this; $=0\iff$ coplanar

Determinant as a scaling factor§ 3.3 →

$T(\mathbf{x})=A\mathbf{x}$, $A$ $2\times2$ ($3\times3$): area (volume) of $T(S)=|\det A|\cdot$ area (volume) of $S$; $\det A\lt 0$ reverses orientation

Chapter 4 · Vectors in ℝⁿ

Vector between two points§ 4.1 →

$\overrightarrow{PQ}=\mathbf{q}-\mathbf{p}=(q_1-p_1,\,q_2-p_2,\,q_3-p_3)$

Addition and scalar multiples in ℝⁿ§ 4.1 →

$\mathbf{u}+\mathbf{v}=(u_1+v_1,\dots,u_n+v_n)$, $\ c\mathbf{v}=(cv_1,\dots,cv_n)$, $\ \mathbf{u}-\mathbf{v}=\mathbf{u}+(-1)\mathbf{v}$ (same $n$ only)

Linear combination§ 4.1 →

$\mathbf{w}=c_1\mathbf{v}_1+c_2\mathbf{v}_2+\dots+c_k\mathbf{v}_k$, scalars $c_i$

Standard unit vectors§ 4.1 →

$\mathbf{v}=v_1\mathbf{e}_1+\dots+v_n\mathbf{e}_n$; in $\mathbb{R}^3$: $(a,b,c)=a\mathbf{i}+b\mathbf{j}+c\mathbf{k}$

Vector form of a linear system§ 4.1 →

$x_1\mathbf{a}_1+\dots+x_n\mathbf{a}_n=\mathbf{b}$ ($\mathbf{a}_j$ = columns); consistent $\iff$ $\mathbf{b}$ is a linear combination of the columns

Dot product§ 4.2 →

$\mathbf{u}\cdot\mathbf{v}=u_1v_1+\dots+u_nv_n=\|\mathbf{u}\|\|\mathbf{v}\|\cos\theta$

Norm, unit vector, distance§ 4.2 →

$\|\mathbf{v}\|=\sqrt{\mathbf{v}\cdot\mathbf{v}}$, $\ \|c\mathbf{v}\|=|c|\|\mathbf{v}\|$, $\ \dfrac{\mathbf{v}}{\|\mathbf{v}\|}$ ($\mathbf{v}\ne\mathbf{0}$), $\ d(\mathbf{u},\mathbf{v})=\|\mathbf{u}-\mathbf{v}\|$

Angle and orthogonality§ 4.2 →

$\cos\theta=\dfrac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}$ ($\mathbf{u},\mathbf{v}\ne\mathbf{0}$, $0\le\theta\le\pi$); $\ \mathbf{u}\perp\mathbf{v}\iff\mathbf{u}\cdot\mathbf{v}=0$

Inequalities and Pythagoras§ 4.2 →

$|\mathbf{u}\cdot\mathbf{v}|\le\|\mathbf{u}\|\|\mathbf{v}\|$, $\ \|\mathbf{u}+\mathbf{v}\|\le\|\mathbf{u}\|+\|\mathbf{v}\|$; $\ \|\mathbf{u}+\mathbf{v}\|^2=\|\mathbf{u}\|^2+\|\mathbf{v}\|^2\iff\mathbf{u}\cdot\mathbf{v}=0$

Projection and component§ 4.2 →

$\mathbf{u}\ne\mathbf{0}$: $\ \operatorname{proj}_{\mathbf{u}}\mathbf{v}=\dfrac{\mathbf{v}\cdot\mathbf{u}}{\mathbf{u}\cdot\mathbf{u}}\,\mathbf{u}$, $\ \operatorname{comp}_{\mathbf{u}}\mathbf{v}=\dfrac{\mathbf{v}\cdot\mathbf{u}}{\|\mathbf{u}\|}$; $\ \mathbf{v}-\operatorname{proj}_{\mathbf{u}}\mathbf{v}\perp\mathbf{u}$

Work and cosine similarity§ 4.2 →

$W=\mathbf{F}\cdot\mathbf{d}$ (J, for N and m); similarity $=\dfrac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}$

Cross product (ℝ³ only)§ 4.3 →

$\mathbf{u}\times\mathbf{v}=\begin{vmatrix}\mathbf{i}&\mathbf{j}&\mathbf{k}\\u_1&u_2&u_3\\v_1&v_2&v_3\end{vmatrix}=(u_2v_3-u_3v_2,\ u_3v_1-u_1v_3,\ u_1v_2-u_2v_1)$; $\ \mathbf{v}\times\mathbf{u}=-\mathbf{u}\times\mathbf{v}$

Area from the cross product§ 4.3 →

$\|\mathbf{u}\times\mathbf{v}\|=\|\mathbf{u}\|\|\mathbf{v}\|\sin\theta$ = parallelogram area; triangle $=\tfrac12\|\mathbf{u}\times\mathbf{v}\|$

Triple product and volume§ 4.3 →

$V=|\mathbf{u}\cdot(\mathbf{v}\times\mathbf{w})|=\left|\det\begin{bmatrix}\mathbf{u}\\\mathbf{v}\\\mathbf{w}\end{bmatrix}\right|$; $\ 0\iff$ coplanar

Line through P with direction d§ 4.3 →

$\mathbf{x}=\mathbf{p}+t\mathbf{d}$ ($\mathbf{d}\ne\mathbf{0}$): $x=x_0+ta$, $y=y_0+tb$, $z=z_0+tc$

Plane through P with normal n§ 4.3 →

$\mathbf{n}\cdot(\mathbf{x}-\mathbf{p})=0\iff ax+by+cz=d$, $\ \mathbf{n}=(a,b,c)\ne\mathbf{0}$; through $P,Q,R$: $\mathbf{n}=\overrightarrow{PQ}\times\overrightarrow{PR}$

Distance from a point§ 4.3 →

To $ax+by+cz=d$: $\ D=\dfrac{|ax_1+by_1+cz_1-d|}{\sqrt{a^2+b^2+c^2}}$; $\ $ to the line $\mathbf{p}+t\mathbf{d}$: $\ D=\dfrac{\|\overrightarrow{PQ}\times\mathbf{d}\|}{\|\mathbf{d}\|}$

Chapter 5 · Vector Spaces

Subspace test§ 5.1 →

A subset $W$ of a vector space $V$ is a subspace $\iff$ $\mathbf{0}\in W$; $\ \mathbf{u},\mathbf{v}\in W\Rightarrow\mathbf{u}+\mathbf{v}\in W$; $\ \mathbf{u}\in W,\ c\in\mathbb{R}\Rightarrow c\mathbf{u}\in W$

Rules in every vector space§ 5.1 →

$0\mathbf{u}=\mathbf{0}$, $\ c\mathbf{0}=\mathbf{0}$, $\ (-1)\mathbf{u}=-\mathbf{u}$; $\ c\mathbf{u}=\mathbf{0}\Rightarrow c=0$ or $\mathbf{u}=\mathbf{0}$

Null space§ 5.1 →

$A$ is $m\times n$: $\ \operatorname{Nul}A=\{\mathbf{x}\in\mathbb{R}^n: A\mathbf{x}=\mathbf{0}\}$ is a subspace of $\mathbb{R}^n$ (but $A\mathbf{x}=\mathbf{b}$, $\mathbf{b}\ne\mathbf{0}$, is not)

Span§ 5.1 →

$\operatorname{span}\{\mathbf{v}_1,\dots,\mathbf{v}_k\}=\{c_1\mathbf{v}_1+\dots+c_k\mathbf{v}_k: c_i\in\mathbb{R}\}$ is the smallest subspace containing $\mathbf{v}_1,\dots,\mathbf{v}_k$

Subspaces of ℝ² and ℝ³§ 5.1 →

$\{\mathbf{0}\}$, lines through $\mathbf{0}$, planes through $\mathbf{0}$ (in $\mathbb{R}^3$), and the whole space

Is b in the span?§ 5.2 →

$\mathbf{b}\in\operatorname{span}\{\mathbf{v}_1,\dots,\mathbf{v}_k\}\iff\left[\begin{array}{c|c}\mathbf{v}_1\ \cdots\ \mathbf{v}_k&\mathbf{b}\end{array}\right]$ is consistent

Spanning ℝⁿ§ 5.2 →

Columns of $A$ ($n\times k$) span $\mathbb{R}^n\iff$ a pivot in every row (needs $k\ge n$)

Linear independence test§ 5.2 →

$c_1\mathbf{v}_1+\dots+c_k\mathbf{v}_k=\mathbf{0}$ only for $c_1=\dots=c_k=0$ $\iff$ every column of $\begin{bmatrix}\mathbf{v}_1&\cdots&\mathbf{v}_k\end{bmatrix}$ is a pivot column

Automatically dependent§ 5.2 →

The set contains $\mathbf{0}$; or two vectors are multiples; or $k\gt n$ vectors in $\mathbb{R}^n$

Redundant vectors§ 5.2 →

Dependent ($k\ge2$) $\iff$ some $\mathbf{v}_j$ is a combination of the others; removing it leaves the span unchanged

Basis§ 5.3 →

$\{\mathbf{b}_1,\dots,\mathbf{b}_n\}$ is a basis of $V$ $\iff$ it is linearly independent and spans $V$; then each $\mathbf{x}\in V$ is $c_1\mathbf{b}_1+\dots+c_n\mathbf{b}_n$ in exactly one way

Standard dimensions§ 5.3 →

$\dim\mathbb{R}^n=n,\quad\dim P_n=n+1,\quad\dim M_{m\times n}=mn,\quad\dim\{\mathbf{0}\}=0$

Basis test in $\mathbb{R}^n$§ 5.3 →

$n$ vectors in $\mathbb{R}^n$ form a basis $\iff\det\begin{bmatrix}\mathbf{v}_1&\cdots&\mathbf{v}_n\end{bmatrix}\ne0$

Basis theorem and subspaces§ 5.3 →

If $\dim V=n\ge1$: $n$ independent vectors, or $n$ spanning vectors, form a basis; more than $n$ vectors are dependent. A subspace $H$ of $V$ has $\dim H\le\dim V$, with equality only if $H=V$.

Trimming and extending§ 5.3 →

Basis of $\operatorname{span}\{\mathbf{v}_1,\dots,\mathbf{v}_p\}$: the original $\mathbf{v}_j$ in the pivot columns of $\begin{bmatrix}\mathbf{v}_1&\cdots&\mathbf{v}_p\end{bmatrix}$. To extend an independent set, append $\mathbf{e}_1,\dots,\mathbf{e}_n$ and keep the pivot columns.

The three subspaces of an $m\times n$ matrix§ 5.4 →

$\operatorname{Col}A=\{A\mathbf{x}\}\subseteq\mathbb{R}^m$, $\quad\operatorname{Row}A=\operatorname{Col}A^{T}\subseteq\mathbb{R}^n$, $\quad\operatorname{Nul}A=\{\mathbf{x}:A\mathbf{x}=\mathbf{0}\}\subseteq\mathbb{R}^n$

Bases from one row reduction§ 5.4 →

$\operatorname{Col}A$: pivot columns of the original $A$; $\ \operatorname{Row}A$: nonzero rows of the echelon form; $\ \operatorname{Nul}A$: one vector per free variable from the parametric solution of $A\mathbf{x}=\mathbf{0}$

Rank–Nullity Theorem§ 5.4 →

$A$ is $m\times n$: $\ \operatorname{rank}A+\operatorname{nullity}A=n,\quad\operatorname{rank}A=\operatorname{rank}A^{T}\le\min(m,n)$

Solving $A\mathbf{x}=\mathbf{b}$§ 5.4 →

Consistent $\iff\mathbf{b}\in\operatorname{Col}A$; then all solutions are $\mathbf{x}=\mathbf{p}+\mathbf{v}_h$, $\ \mathbf{v}_h\in\operatorname{Nul}A$

Invertible Matrix Theorem (rank form)§ 5.4 →

$A$ is $n\times n$: $\ A$ invertible $\iff\det A\ne0\iff\operatorname{rank}A=n\iff\operatorname{nullity}A=0\iff\operatorname{Col}A=\mathbb{R}^n\iff$ columns (or rows) form a basis of $\mathbb{R}^n$

Coordinate vector§ 5.5 →

$\mathcal B=\{\mathbf{b}_1,\dots,\mathbf{b}_n\}$ a basis: $\ \mathbf{x}=c_1\mathbf{b}_1+\dots+c_n\mathbf{b}_n\iff[\mathbf{x}]_{\mathcal B}=\begin{bmatrix}c_1\\\vdots\\c_n\end{bmatrix}$

To and from standard coordinates§ 5.5 →

$P_{\mathcal B}=\begin{bmatrix}\mathbf{b}_1&\cdots&\mathbf{b}_n\end{bmatrix}$ (invertible): $\ \mathbf{x}=P_{\mathcal B}[\mathbf{x}]_{\mathcal B},\quad[\mathbf{x}]_{\mathcal B}=P_{\mathcal B}^{-1}\mathbf{x}$

Change of basis $\mathcal B\to\mathcal C$§ 5.5 →

$[\mathbf{x}]_{\mathcal C}=P_{\mathcal C\leftarrow\mathcal B}[\mathbf{x}]_{\mathcal B}$, $\ P_{\mathcal C\leftarrow\mathcal B}=\begin{bmatrix}[\mathbf{b}_1]_{\mathcal C}&\cdots&[\mathbf{b}_n]_{\mathcal C}\end{bmatrix}$; in $\mathbb{R}^n$: $[\,\mathcal C\,|\,\mathcal B\,]\sim[\,I\,|\,P_{\mathcal C\leftarrow\mathcal B}\,]$; $\ \left(P_{\mathcal C\leftarrow\mathcal B}\right)^{-1}=P_{\mathcal B\leftarrow\mathcal C}$

Rotation of axes through $\theta$§ 5.5 →

$x'=x\cos\theta+y\sin\theta,\quad y'=-x\sin\theta+y\cos\theta$; $\quad x=x'\cos\theta-y'\sin\theta,\quad y=x'\sin\theta+y'\cos\theta$

Chapter 6 · Linear Transformations

Linear transformation§ 6.1 →

$T:V\to W$ is linear if $T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v})$ and $T(c\mathbf{u})=cT(\mathbf{u})$ for all $\mathbf{u},\mathbf{v}\in V$ and scalars $c$; equivalently $T(c\mathbf{u}+d\mathbf{v})=cT(\mathbf{u})+dT(\mathbf{v})$

Consequences of linearity§ 6.1 →

$T(\mathbf{0})=\mathbf{0}$, $\ T(-\mathbf{v})=-T(\mathbf{v})$, $\ T\big(\sum c_i\mathbf{v}_i\big)=\sum c_iT(\mathbf{v}_i)$

Standard matrix§ 6.1 →

$T:\mathbb{R}^n\to\mathbb{R}^m$ linear $\Rightarrow$ $T(\mathbf{x})=A\mathbf{x}$ with the $m\times n$ matrix $A=\begin{bmatrix}T(\mathbf{e}_1)&\cdots&T(\mathbf{e}_n)\end{bmatrix}$

Composition§ 6.1 →

$T$ has matrix $A$, $S$ has matrix $B$: $\ (S\circ T)(\mathbf{x})=B(A\mathbf{x})=(BA)\mathbf{x}$ — first $T$, then $S$

Kernel and range§ 6.2 →

$\ker T=\{\mathbf{v}\in V:T(\mathbf{v})=\mathbf{0}\}$, $\ \operatorname{range}T=\{T(\mathbf{v}):\mathbf{v}\in V\}$; for $T(\mathbf{x})=A\mathbf{x}$: $\ker T=\operatorname{Nul}A$, $\operatorname{range}T=\operatorname{Col}A$

One-to-one test§ 6.2 →

Linear $T$ is one-to-one $\iff\ker T=\{\mathbf{0}\}$

Pivot test for $\mathbf{x}\mapsto A\mathbf{x}$ ($A$ is $m\times n$)§ 6.2 →

one-to-one $\iff$ pivot in every column $\iff\operatorname{rank}A=n$; $\quad$ onto $\iff$ pivot in every row $\iff\operatorname{rank}A=m$

Rank–nullity theorem§ 6.2 →

$V$ finite-dimensional, $T:V\to W$ linear: $\ \dim\ker T+\dim\operatorname{range}T=\dim V$

Isomorphisms§ 6.2 →

$\mathbf{x}\mapsto A\mathbf{x}$ ($A$ $n\times n$) is an isomorphism $\iff\det A\ne0$, with inverse $\mathbf{y}\mapsto A^{-1}\mathbf{y}$; $\ \dim V=n\Rightarrow V\cong\mathbb{R}^n$ via $\mathbf{v}\mapsto[\mathbf{v}]_{\mathcal B}$

Matrix of $T:V\to W$ relative to $\mathcal B$, $\mathcal C$§ 6.3 →

$[T]_{\mathcal C\leftarrow\mathcal B}=\begin{bmatrix}[T(\mathbf{b}_1)]_{\mathcal C}&\cdots&[T(\mathbf{b}_n)]_{\mathcal C}\end{bmatrix}$, $\ [T(\mathbf{x})]_{\mathcal C}=[T]_{\mathcal C\leftarrow\mathcal B}[\mathbf{x}]_{\mathcal B}$ ($T$ linear, $V$, $W$ finite-dimensional)

Operator in a new basis§ 6.3 →

$T(\mathbf{x})=A\mathbf{x}$ on $\mathbb{R}^n$, $P=\begin{bmatrix}\mathbf{b}_1&\cdots&\mathbf{b}_n\end{bmatrix}$: $\ [T]_{\mathcal B}=P^{-1}AP$, $\ A=P[T]_{\mathcal B}P^{-1}$

Two bases of $V$§ 6.3 →

$[T]_{\mathcal C}=P^{-1}[T]_{\mathcal B}P$ with $P=P_{\mathcal B\leftarrow\mathcal C}$

Similar matrices share§ 6.3 →

$B=P^{-1}AP$ ($P$ invertible) $\Rightarrow$ $\det B=\det A$, $\operatorname{tr}B=\operatorname{tr}A$, $\operatorname{rank}B=\operatorname{rank}A$, $B^k=P^{-1}A^kP$

Rotation and reflection in the plane§ 6.4 →

Rotation by $\theta$ about $\mathbf{0}$: $R_\theta=\begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$; reflection in the line through $\mathbf{0}$ at angle $\theta$: $F_\theta=\begin{bmatrix}\cos2\theta&\sin2\theta\\\sin2\theta&-\cos2\theta\end{bmatrix}$

Projection onto / reflection in a line with direction $(a,b)\ne\mathbf{0}$§ 6.4 →

$P=\dfrac{1}{a^2+b^2}\begin{bmatrix}a^2&ab\\ab&b^2\end{bmatrix}$, $\ F=2P-I$; $\ P^2=P$, $F^2=I$

Shears and scalings§ 6.4 →

Horizontal shear $\begin{bmatrix}1&k\\0&1\end{bmatrix}$, vertical shear $\begin{bmatrix}1&0\\k&1\end{bmatrix}$, scaling $\begin{bmatrix}k_1&0\\0&k_2\end{bmatrix}$, dilation $kI$

Compositions§ 6.4 →

"First $A$, then $B$" has matrix $BA$; $\ R_\alpha R_\beta=R_{\alpha+\beta}$, $\ F_\varphi F_\theta=R_{2(\varphi-\theta)}$

Area and orientation§ 6.4 →

$T(\mathbf{x})=A\mathbf{x}$ on $\mathbb{R}^2$: $\operatorname{area}(T(S))=|\det A|\operatorname{area}(S)$; orientation preserved if $\det A\gt0$, reversed if $\det A\lt0$

Homogeneous coordinates§ 6.4 →

$(x,y)\leftrightarrow(x,y,1)$; translation $T_{(h,k)}=\begin{bmatrix}1&0&h\\0&1&k\\0&0&1\end{bmatrix}$; map $A$ about the point $\mathbf{p}$: $T_{\mathbf{p}}\begin{bmatrix}A&\mathbf{0}\\\mathbf{0}^{T}&1\end{bmatrix}T_{-\mathbf{p}}$

Chapter 7 · Eigenvalues and Eigenvectors

Eigenvalue and eigenvector§ 7.1 →

$A\mathbf{x}=\lambda\mathbf{x}$ with $\mathbf{x}\ne\mathbf{0}$ ($A$ square). Every nonzero multiple of $\mathbf{x}$ is also an eigenvector.

Characteristic equation and eigenspace§ 7.1 →

$\lambda$ is an eigenvalue $\iff\det(A-\lambda I)=0$; $\quad E_\lambda=\operatorname{Nul}(A-\lambda I)$ (eigenvectors together with $\mathbf{0}$).

$2\times2$ characteristic polynomial§ 7.1 →

$\det(A-\lambda I)=\lambda^2-(\operatorname{tr}A)\lambda+\det A$

Trace and determinant§ 7.1 →

With the $n$ roots of $\det(A-\lambda I)$ counted with multiplicity (complex ones included): $\lambda_1+\dots+\lambda_n=\operatorname{tr}A$, $\ \lambda_1\cdots\lambda_n=\det A$.

Related matrices (same eigenvectors)§ 7.1 →

If $A\mathbf{x}=\lambda\mathbf{x}$: $A^k\mathbf{x}=\lambda^k\mathbf{x}$, $(A+cI)\mathbf{x}=(\lambda+c)\mathbf{x}$, and if $A$ is invertible $A^{-1}\mathbf{x}=\lambda^{-1}\mathbf{x}$. $\ A$ invertible $\iff0$ is not an eigenvalue.

Multiplicities; triangular matrices§ 7.1 →

$1\le\dim E_\lambda\le$ algebraic multiplicity. Triangular $A$: eigenvalues $=$ diagonal entries.

Diagonalisation theorem§ 7.2 →

$A$ ($n\times n$) is diagonalisable $\iff$ it has $n$ independent eigenvectors. Then $A=PDP^{-1}$, $P=[\mathbf{v}_1\cdots\mathbf{v}_n]$, $D=\operatorname{diag}(\lambda_1,\dots,\lambda_n)$ in the same order.

When is $A$ diagonalisable?§ 7.2 →

$n$ distinct real eigenvalues $\Rightarrow$ diagonalisable. In general: diagonalisable $\iff\sum_\lambda\dim E_\lambda=n$ $\iff$ all eigenvalues real and geometric $=$ algebraic multiplicity for each.

Powers§ 7.2 →

$A=PDP^{-1}\Rightarrow A^k=PD^kP^{-1}$, $\ D^k=\operatorname{diag}(\lambda_1^k,\dots,\lambda_n^k)$

$A^k\mathbf{x}$ by eigenvectors§ 7.2 →

$\mathbf{x}=\sum c_i\mathbf{v}_i\Rightarrow A^k\mathbf{x}=\sum c_i\lambda_i^k\mathbf{v}_i$

Similar matrices§ 7.2 →

$B=P^{-1}AP\Rightarrow\det(B-\lambda I)=\det(A-\lambda I)$: same eigenvalues, trace, determinant.

Eigenbasis as change of basis§ 7.2 →

$\mathcal B=$ columns of $P$: $\ [T]_{\mathcal B}=D$, $\ [A\mathbf{x}]_{\mathcal B}=D[\mathbf{x}]_{\mathcal B}$

Discrete dynamical system $\mathbf{x}_{k+1}=A\mathbf{x}_k$§ 7.3 →

$\mathbf{x}_k=c_1\lambda_1^k\mathbf{v}_1+\dots+c_n\lambda_n^k\mathbf{v}_n$ ($n$ independent eigenvectors). If $\lambda_1$ is dominant and $c_1\ne0$: $\mathbf{x}_k\approx c_1\lambda_1^k\mathbf{v}_1$.

Origin of $\mathbf{x}_{k+1}=A\mathbf{x}_k$ ($2\times2$)§ 7.3 →

Markov steady state§ 7.3 →

$P\mathbf{q}=\mathbf{q}$: $\mathbf{q}\in E_1$, entries adding to $1$. Stochastic $\Rightarrow|\lambda|\le1$; regular $\Rightarrow\mathbf{x}_k\to\mathbf{q}$.

Power method§ 7.3 →

$\mu_k=$ entry of $A\mathbf{x}_k$ of largest size, $\mathbf{x}_{k+1}=A\mathbf{x}_k/\mu_k$. If $A$ is diagonalisable, $|\lambda_1|\gt|\lambda_2|\ge\cdots$ and $c_1\ne0$: $\mu_k\to\lambda_1$, error $\approx$ factor $|\lambda_2/\lambda_1|$ per step.

$\mathbf{x}'=A\mathbf{x}$ (uses calculus)§ 7.3 →

$\mathbf{x}(t)=c_1e^{\lambda_1t}\mathbf{v}_1+\dots+c_ne^{\lambda_nt}\mathbf{v}_n$ ($n$ independent eigenvectors, real $\lambda_i$); all $\lambda_i\lt0\Rightarrow\mathbf{x}(t)\to\mathbf{0}$.

Chapter 8 · Orthogonality and Least Squares

Inner product: norm, distance, orthogonality§ 8.1 →

$\|\mathbf{v}\|=\sqrt{\langle\mathbf{v},\mathbf{v}\rangle}$, $\ \operatorname{dist}(\mathbf{u},\mathbf{v})=\|\mathbf{u}-\mathbf{v}\|$, $\ \mathbf{u}\perp\mathbf{v}\iff\langle\mathbf{u},\mathbf{v}\rangle=0$; weighted: $\langle\mathbf{u},\mathbf{v}\rangle=\sum w_iu_iv_i$ with all $w_i\gt0$

Orthogonal complements§ 8.1 →

$(\operatorname{Row}A)^{\perp}=\operatorname{Nul}A$, $\ (\operatorname{Col}A)^{\perp}=\operatorname{Nul}A^{T}$; for $W\subseteq\mathbb{R}^n$: $\dim W+\dim W^{\perp}=n$

Coordinates in an orthogonal basis§ 8.1 →

$\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ orthogonal basis of $W$, $\mathbf{y}\in W$: $\ \mathbf{y}=\sum c_j\mathbf{u}_j$, $\ c_j=\dfrac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$ ($=\mathbf{y}\cdot\mathbf{u}_j$ if orthonormal)

Orthogonal matrices§ 8.1 →

$U$ square with orthonormal columns: $U^{T}U=I$, $U^{-1}=U^{T}$, $\|U\mathbf{x}\|=\|\mathbf{x}\|$, $(U\mathbf{x})\cdot(U\mathbf{y})=\mathbf{x}\cdot\mathbf{y}$, $\det U=\pm1$

The Gram–Schmidt process§ 8.2 →

$\{\mathbf{x}_1,\dots,\mathbf{x}_p\}$ a basis of $W$: $\ \mathbf{v}_1=\mathbf{x}_1$, $\ \mathbf{v}_k=\mathbf{x}_k-\displaystyle\sum_{j=1}^{k-1}\frac{\mathbf{x}_k\cdot\mathbf{v}_j}{\mathbf{v}_j\cdot\mathbf{v}_j}\mathbf{v}_j$; then $\{\mathbf{v}_k\}$ is an orthogonal basis of $W$ and $\mathbf{q}_k=\mathbf{v}_k/\|\mathbf{v}_k\|$ an orthonormal one

QR factorisation§ 8.2 →

Columns of $A$ independent: $A=QR$, $Q^{T}Q=I$, $R=Q^{T}A$ upper triangular with $r_{kk}=\|\mathbf{v}_k\|\gt0$

Solving with QR§ 8.2 →

$A\mathbf{x}=\mathbf{b}$ with $A=QR$: solve $R\mathbf{x}=Q^{T}\mathbf{b}$ by back substitution ($A$ square invertible, or $\mathbf{b}\in\operatorname{Col}A$)

Projection onto a subspace§ 8.3 →

$\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ an orthogonal basis of $W$: $\ \hat{\mathbf{y}}=\operatorname{proj}_W\mathbf{y}=\displaystyle\sum_{i=1}^{p}\frac{\mathbf{y}\cdot\mathbf{u}_i}{\mathbf{u}_i\cdot\mathbf{u}_i}\mathbf{u}_i$, and $\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}$ with $\mathbf{z}=\mathbf{y}-\hat{\mathbf{y}}\in W^{\perp}$

Best approximation§ 8.3 →

$\|\mathbf{y}-\hat{\mathbf{y}}\|\lt\|\mathbf{y}-\mathbf{v}\|$ for all $\mathbf{v}\in W$, $\mathbf{v}\ne\hat{\mathbf{y}}$; distance from $\mathbf{y}$ to $W$ is $\|\mathbf{y}-\hat{\mathbf{y}}\|$

Projection matrix§ 8.3 →

Columns of $A$ independent: $P=A\left(A^{T}A\right)^{-1}A^{T}$ projects onto $\operatorname{Col}A$; $P^{T}=P$, $P^2=P$; orthonormal columns: $P=UU^{T}$

Least squares — normal equations§ 8.3 →

$A^{T}A\hat{\mathbf{x}}=A^{T}\mathbf{b}$ (always consistent); if the columns of $A$ are independent, $\hat{\mathbf{x}}=\left(A^{T}A\right)^{-1}A^{T}\mathbf{b}$; error $\|\mathbf{b}-A\hat{\mathbf{x}}\|$

Least-squares line $y=c_0+c_1x$§ 8.3 →

$\begin{bmatrix}m&\sum x_i\\\sum x_i&\sum x_i^2\end{bmatrix}\begin{bmatrix}\hat c_0\\\hat c_1\end{bmatrix}=\begin{bmatrix}\sum y_i\\\sum x_iy_i\end{bmatrix}$ ($x_i$ not all equal)

Least squares by QR§ 8.3 →

$A=QR$ (independent columns): solve $R\hat{\mathbf{x}}=Q^{T}\mathbf{b}$, i.e. $\hat{\mathbf{x}}=R^{-1}Q^{T}\mathbf{b}$

Spectral Theorem§ 8.4 →

$A$ real symmetric $\iff A=PDP^{T}$ with $P^{T}P=I$, $D$ diagonal; eigenvalues real, eigenvectors for distinct eigenvalues orthogonal

Spectral decomposition§ 8.4 →

$A=\displaystyle\sum_{i=1}^{n}\lambda_i\mathbf{u}_i\mathbf{u}_i^{T}$, $\ A^k=\displaystyle\sum_{i=1}^{n}\lambda_i^k\mathbf{u}_i\mathbf{u}_i^{T}$ ($\mathbf{u}_i$ orthonormal eigenvectors, $A$ symmetric)

Quadratic form — principal axes§ 8.4 →

$Q(\mathbf{x})=\mathbf{x}^{T}A\mathbf{x}$, $a_{ij}=a_{ji}=\tfrac12(\text{coefficient of }x_ix_j)$; with $\mathbf{x}=P\mathbf{y}$: $\ Q=\lambda_1y_1^2+\cdots+\lambda_ny_n^2$

Classifying quadratic forms§ 8.4 →

$A$ symmetric: positive definite $\iff$ all $\lambda_i\gt0$; negative definite $\iff$ all $\lambda_i\lt0$; indefinite $\iff$ eigenvalues of both signs. $2\times2$: positive definite $\iff a\gt0$ and $ad-b^2\gt0$

Extremes on the unit sphere§ 8.4 →

$A$ symmetric: $\ \lambda_{\min}\le\mathbf{x}^{T}A\mathbf{x}\le\lambda_{\max}$ for $\|\mathbf{x}\|=1$, with equality at unit eigenvectors

Singular values§ 8.4 →

$\sigma_i=\sqrt{\lambda_i(A^{T}A)}$; $\ A=U\Sigma V^{T}=\displaystyle\sum_{i=1}^{r}\sigma_i\mathbf{u}_i\mathbf{v}_i^{T}$

← 8.4 Symmetric Matrices and the Spectral Theorem Glossary — English / العربية →