Linear Algebra
Master vector spaces, linear transformations, eigenvalue theory, inner product spaces, and canonical forms for GATE Mathematics.
The foundation of linear algebra: understanding vector spaces, subspaces, bases, and dimension over arbitrary fields.
A vector space is the central object of study in linear algebra. It provides the abstract framework for understanding systems of linear equations, transformations, and geometric intuition in higher dimensions.
Common examples include $\mathbb{R}^n$ over $\mathbb{R}$, the space of $m \times n$ matrices $M_{m \times n}(\mathbb{R})$, and the space of polynomials $P_n(\mathbb{R})$ of degree at most $n$.
- A subspace $W \subseteq V$ is a non-empty subset closed under addition and scalar multiplication
- The span of a set $S$ is the set of all finite linear combinations of elements in $S$
- A set is linearly independent if no vector can be written as a linear combination of the others
A basis of a vector space $V$ is a linearly independent set that spans $V$. The number of elements in any basis is the dimension of $V$.
Linear independence: Suppose $a_1(1,0,0) + a_2(0,1,0) + a_3(0,0,1) = (0,0,0)$. Then $(a_1, a_2, a_3) = (0,0,0)$, so $a_1 = a_2 = a_3 = 0$. The set is linearly independent.
Spanning: Any vector $(x,y,z) \in \mathbb{R}^3$ can be written as $x(1,0,0) + y(0,1,0) + z(0,0,1)$.
Since $\beta$ is linearly independent and spans $\mathbb{R}^3$, it is a basis. Hence $\dim(\mathbb{R}^3) = 3$.
Change of basis: If $\beta$ and $\beta'$ are two bases for $V$, the change of basis matrix $P$ satisfies $[\mathbf{v}]_{\beta'} = P^{-1}[\mathbf{v}]_{\beta}$, where $P$ is the matrix whose columns are the $\beta$-coordinates of the vectors in $\beta'$.
- Always verify both closure properties when checking if a subset is a subspace
- $\dim(\mathbb{R}^n) = n$, $\dim(M_{m \times n}) = mn$, $\dim(P_n) = n+1$
- A set of $k$ vectors in $\mathbb{R}^n$ is linearly dependent if $k > n$
Functions between vector spaces that preserve the linear structure, along with their fundamental properties and matrix representations.
A function $T: V \to W$ between vector spaces is a linear transformation if $T(\alpha u + \beta v) = \alpha T(u) + \beta T(v)$ for all $u, v \in V$ and scalars $\alpha, \beta$.
- Kernel: $\ker(T) = \{v \in V : T(v) = \mathbf{0}\}$ — a subspace of $V$
- Image: $\text{Im}(T) = \{T(v) : v \in V\}$ — a subspace of $W$
- $T$ is injective (one-to-one) if and only if $\ker(T) = \{\mathbf{0}\}$
Every linear transformation between finite-dimensional vector spaces can be represented by a matrix. If $T: V \to W$ with ordered bases $\beta$ for $V$ and $\gamma$ for $W$, then $[T]_{\beta}^{\gamma}$ is the matrix whose $j$-th column is $[T(v_j)]_{\gamma}$.
Step 1: Set $T(x,y,z) = (0,0)$, giving the system $x + y = 0$ and $y + z = 0$.
Step 2: From the first equation, $x = -y$. From the second, $z = -y$.
Step 3: Let $y = t$ (free variable). Then $(x,y,z) = (-t, t, -t) = t(-1, 1, -1)$.
Therefore $\ker(T) = \text{span}\{(-1, 1, -1)\}$ and $\dim(\ker T) = 1$.
Verification: By rank-nullity, $\text{rank}(T) = 3 - 1 = 2 = \dim(\mathbb{R}^2)$, so $T$ is surjective.
- The rank-nullity theorem is one of the most frequently tested results in GATE
- For a square matrix $A$: $\text{rank}(A) + \text{nullity}(A) = n$
- Changing bases changes the matrix representation: $[T]_{\beta'} = P^{-1}[T]_{\beta}P$
Eigenvalue theory is central to matrix analysis and appears heavily in GATE. Topics include characteristic polynomials, diagonalizability criteria, and the Cayley-Hamilton theorem.
A scalar $\lambda$ is an eigenvalue of a linear operator $T$ (or matrix $A$) if there exists a nonzero vector $v$ such that $Av = \lambda v$. The vector $v$ is the corresponding eigenvector.
- Algebraic multiplicity: the multiplicity of $\lambda$ as a root of $p(\lambda)$
- Geometric multiplicity: $\dim(\ker(A - \lambda I))$ — the dimension of the eigenspace
- For every eigenvalue: $1 \leq \text{geometric mult.} \leq \text{algebraic mult.}$
A matrix $A$ is diagonalizable if there exists an invertible matrix $P$ such that $P^{-1}AP = D$ is diagonal. This happens if and only if the geometric multiplicity equals the algebraic multiplicity for every eigenvalue.
Step 1: Characteristic polynomial: $\det(A - \lambda I) = (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda - 1)(\lambda - 3)$.
Step 2: Eigenvalues are $\lambda_1 = 1$ and $\lambda_2 = 3$.
Step 3 ($\lambda = 1$): Solve $(A - I)v = 0$: $\begin{pmatrix}1&1\\1&1\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix} = \begin{pmatrix}0\\0\end{pmatrix}$. So $x = -y$, giving eigenvector $v_1 = \begin{pmatrix}-1\\1\end{pmatrix}$.
Step 4 ($\lambda = 3$): Solve $(A - 3I)v = 0$: $\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix} = \begin{pmatrix}0\\0\end{pmatrix}$. So $x = y$, giving eigenvector $v_2 = \begin{pmatrix}1\\1\end{pmatrix}$.
Since $A$ has 2 distinct eigenvalues in a $2 \times 2$ matrix, $A$ is diagonalizable: $P^{-1}AP = \begin{pmatrix}1&0\\0&3\end{pmatrix}$ where $P = \begin{pmatrix}-1&1\\1&1\end{pmatrix}$.
- $\text{trace}(A) = \sum \lambda_i$ and $\det(A) = \prod \lambda_i$
- Real symmetric matrices are always diagonalizable with real eigenvalues
- Cayley-Hamilton is useful for computing $A^{-1}$ and powers of $A$
Adding geometric structure to vector spaces: inner products, orthogonality, and the Gram-Schmidt process.
The norm induced by an inner product is $\|v\| = \sqrt{\langle v, v \rangle}$.
Vectors $u, v$ are orthogonal if $\langle u, v \rangle = 0$. An orthonormal basis is a basis of mutually orthogonal unit vectors. The Gram-Schmidt process converts any basis into an orthogonal one.
Step 1: Set $u_1 = v_1 = (1, 1, 0)$.
Step 2: Compute $u_2 = v_2 - \dfrac{\langle v_2, u_1 \rangle}{\langle u_1, u_1 \rangle} u_1$.
$\langle v_2, u_1 \rangle = (1)(1) + (0)(1) + (1)(0) = 1$ and $\langle u_1, u_1 \rangle = 1 + 1 + 0 = 2$.
$u_2 = (1,0,1) - \frac{1}{2}(1,1,0) = \left(\frac{1}{2}, -\frac{1}{2}, 1\right)$.
Step 3 (Normalize): $e_1 = \frac{u_1}{\|u_1\|} = \frac{1}{\sqrt{2}}(1,1,0)$ and $e_2 = \frac{u_2}{\|u_2\|} = \frac{1}{\sqrt{3/2}}\left(\frac{1}{2}, -\frac{1}{2}, 1\right) = \frac{1}{\sqrt{6}}(1, -1, 2)$.
The orthonormal set is $\left\{\frac{1}{\sqrt{2}}(1,1,0),\; \frac{1}{\sqrt{6}}(1,-1,2)\right\}$.
Orthogonal projection: The projection of $v$ onto subspace $W$ is $\text{proj}_W(v) = \sum_{i} \langle v, e_i \rangle e_i$ where $\{e_i\}$ is an orthonormal basis for $W$. This is the foundation of the least squares method.
- Gram-Schmidt is a standard procedure tested in GATE — practice the computation
- Orthogonal matrices satisfy $Q^T Q = I$ and preserve norms and inner products
- The least squares solution to $Ax = b$ is $x = (A^T A)^{-1} A^T b$
The structure theorem for linear operators: Jordan and rational canonical forms provide the deepest insight into the behaviour of matrices.
When a matrix is not diagonalizable, the next best structure is the Jordan canonical form. Every complex matrix is similar to a block-diagonal matrix of Jordan blocks.
- The minimal polynomial is the monic polynomial of least degree such that $m(A) = O$
- The minimal polynomial divides the characteristic polynomial, and they share the same roots
- $A$ is diagonalizable if and only if the minimal polynomial has no repeated roots
The rational canonical form (companion matrix form) works over any field and depends only on the invariant factors of $A - \lambda I$. It is unique and does not require knowledge of eigenvalues.
Applications to differential equations: For a system $\mathbf{x}' = A\mathbf{x}$, the Jordan form of $A$ determines the solution structure. Each Jordan block $J_k(\lambda)$ contributes terms of the form $t^j e^{\lambda t}$ for $j = 0, 1, \ldots, k-1$.
- Jordan form exists for every complex matrix; it is unique up to ordering of blocks
- The minimal polynomial determines the size of the largest Jordan block for each eigenvalue
- Rational canonical form is field-independent and uniquely determined by invariant factors