6.3 Matrices of Linear Transformations

In Section 6.1 we saw that every linear transformation $T:\mathbb{R}^n\to\mathbb{R}^m$ is multiplication by its standard matrix $A=\begin{bmatrix}T(\mathbf{e}_1)&\cdots&T(\mathbf{e}_n)\end{bmatrix}$. That is wonderful for computation — a computer stores a few numbers and multiplies — but two questions remain. What about maps on other vector spaces, such as "shift a polynomial", "differentiate", or "transpose a matrix", where there are no columns to speak of? And is the standard basis always the best point of view?

The answer to both questions is coordinates (Section 5.5). Once we fix a basis $\mathcal B$ for the domain and a basis $\mathcal C$ for the codomain, every vector becomes a column of numbers and every linear map becomes a matrix. Different bases give different matrices for the same map, and choosing the basis well can turn a messy matrix into a diagonal one — a reflection becomes $\begin{bmatrix}1&0\\0&-1\end{bmatrix}$, a projection becomes $\begin{bmatrix}1&0\\0&0\end{bmatrix}$. Engineers call such a basis the "natural axes" of a problem; finding it systematically is the subject of Chapter 7.

The Matrix of $T$ Relative to Bases $\mathcal B$ and $\mathcal C$

Let $V$ have basis $\mathcal B=\{\mathbf{b}_1,\dots,\mathbf{b}_n\}$ and $W$ have basis $\mathcal C=\{\mathbf{c}_1,\dots,\mathbf{c}_m\}$. Any $\mathbf{x}\in V$ can be written uniquely as $\mathbf{x}=x_1\mathbf{b}_1+\cdots+x_n\mathbf{b}_n$, so $[\mathbf{x}]_{\mathcal B}=(x_1,\dots,x_n)$. Because $T$ is linear, $$T(\mathbf{x})=x_1T(\mathbf{b}_1)+\cdots+x_nT(\mathbf{b}_n),$$ and since taking $\mathcal C$-coordinates is itself linear, $$[T(\mathbf{x})]_{\mathcal C}=x_1[T(\mathbf{b}_1)]_{\mathcal C}+\cdots+x_n[T(\mathbf{b}_n)]_{\mathcal C}.$$ The right-hand side is a linear combination of $n$ fixed columns with weights $x_1,\dots,x_n$ — in other words, a matrix times $[\mathbf{x}]_{\mathcal B}$.

The arrow in the subscript reads right to left, like the matrix product itself: feed in $\mathcal B$-coordinates on the right, get $\mathcal C$-coordinates out on the left. Uniqueness holds because multiplying any matrix with this property by $[\mathbf{b}_j]_{\mathcal B}=\mathbf{e}_j$ must return the $j$-th column $[T(\mathbf{b}_j)]_{\mathcal C}$.

The real power of Definition 6.6 is that $V$ and $W$ need not be $\mathbb{R}^n$ at all. In $P_n$ the standard basis $\{1,x,\dots,x^n\}$ makes coordinates trivial to read: $[a_0+a_1x+\cdots+a_nx^n]=(a_0,a_1,\dots,a_n)$.

Linear Operators and Change of Basis: Similarity

A linear map $T:V\to V$ from a space to itself is called a linear operator. For operators it is natural to use the same basis on both sides.

How are the matrices of one operator in two different bases related? Recall from Section 5.5 that the change-of-basis matrix $P_{\mathcal B\leftarrow\mathcal C}$ converts $\mathcal C$-coordinates into $\mathcal B$-coordinates, and $P_{\mathcal C\leftarrow\mathcal B}=\left(P_{\mathcal B\leftarrow\mathcal C}\right)^{-1}$. Follow a vector round the diagram: start with $[\mathbf{x}]_{\mathcal C}$, convert to $\mathcal B$-coordinates, apply $T$ in the $\mathcal B$-world, convert back.

Why it is true. For any $\mathbf{x}$, $$\begin{aligned}[T(\mathbf{x})]_{\mathcal C}&=P^{-1}[T(\mathbf{x})]_{\mathcal B}=P^{-1}[T]_{\mathcal B}[\mathbf{x}]_{\mathcal B}\\&=P^{-1}[T]_{\mathcal B}P\,[\mathbf{x}]_{\mathcal C}.\end{aligned}$$ By the uniqueness in Theorem 6.11 the matrix in front of $[\mathbf{x}]_{\mathcal C}$ must be $[T]_{\mathcal C}$. Read $P^{-1}AP$ from right to left: $P$ turns $\mathcal B$-coordinates into ordinary coordinates, $A$ does the map, $P^{-1}$ turns the answer back into $\mathcal B$-coordinates.

So Theorem 6.12 says: the matrices of one linear operator relative to different bases are similar. Conversely, similar matrices can always be read as the same operator seen in two bases (take the columns of $P$ as the new basis). Properties that belong to the operator itself — not to the coordinates — must therefore be shared by similar matrices.

Choosing a Good Basis

Examples 4–6 show a pattern. The standard matrix depends on how the map sits relative to the coordinate axes, which usually has nothing to do with the map itself. If we instead build the basis from directions that the map treats simply — a mirror line and its perpendicular, the line onto which we project, directions that are only stretched — the matrix becomes diagonal or nearly so. Simple matrices make everything easier:

• Powers: if $[T]_{\mathcal B}=D=\operatorname{diag}(d_1,\dots,d_n)$ then $A^k=PD^kP^{-1}$ and $D^k=\operatorname{diag}\left(d_1^k,\dots,d_n^k\right)$ — compare Exercise III.7.
• Understanding: a diagonal $\mathcal B$-matrix says exactly how much each basis direction is stretched, flipped or squashed.
• Computation: engineers rotate to "principal axes" (see the Research–Teaching Nexus) so that a coupled problem splits into independent one-variable problems.

Exercises