3.3 Inner Products, Special Matrices and Quadratic Forms
Inner products, orthonormal bases and Gram–Schmidt; the special classes of matrices (symmetric, skew-symmetric, orthogonal, Hermitian, skew-Hermitian, unitary, normal) and what each says about eigenvalues; unitary diagonalisation of normal matrices; and bilinear and quadratic forms with their rank, signature and definiteness.
Sections 3.1 and 3.2 studied vector spaces with nothing but addition and scaling. That is enough to talk about rank, eigenvalues and diagonalisation, but not about length, angle or perpendicularity. An inner product supplies all three, and with them comes a second, stronger kind of diagonalisation: not just "some basis of eigenvectors", but an orthonormal one, reached by a rigid change of coordinates (a rotation or reflection, or its complex analogue, a unitary matrix). Which matrices allow this? Exactly the normal ones, those that commute with their conjugate transpose. This is the spectral theorem, and it explains a whole list of facts GATE treats as routine: symmetric and Hermitian matrices have real eigenvalues and perpendicular eigenvectors, unitary matrices have eigenvalues of modulus one, skew-Hermitian matrices have purely imaginary ones. The same theorem, applied to a symmetric matrix, rotates a quadratic form such as $5x^2+6xy+5y^2$ into a sum of squares $8X^2+2Y^2$. The signs of those squares give the rank, signature and definiteness of the form, the type of a conic or quadric, and (through the Hessian of §1.2) the nature of a critical point.
GATE MA asks about this material every year. MCQs ask for the eigenvalues of a rotation or the signature of a form; MSQs list statements about Hermitian, unitary and normal matrices ("if all eigenvalues are real then the matrix is Hermitian", "a matrix whose eigenvalues all have modulus $1$ is unitary") and you have to decide each one, which needs a ready supply of counterexamples; NATs ask for a distance computed by Gram–Schmidt, a range of a parameter for positive definiteness, or the area of an ellipse. This section gives the definitions, the theorems with short proofs, the GATE routines and the standard traps.
Inner Products, Orthogonality and Gram–Schmidt
Throughout, $\mathbb{F}$ is $\mathbb{R}$ or $\mathbb{C}$, $V$ is a finite-dimensional vector space over $\mathbb{F}$, vectors in $\mathbb{F}^n$ are columns, $A^{\mathsf T}$ is the transpose, $\bar A$ the entrywise conjugate and $A^*=\bar A^{\mathsf T}$ the conjugate transpose (for real matrices $A^*=A^{\mathsf T}$). We use the same conventions as the CSIR course: the inner product is linear in its first slot.
An inner product on $V$ is a map $V\times V\to\mathbb{F}$, $(x,y)\mapsto\langle x,y\rangle$, such that for all $x,y,z\in V$ and $a,b\in\mathbb{F}$:
- (linearity in the first slot) $\langle ax+by,z\rangle$ $=a\langle x,z\rangle+b\langle y,z\rangle$;
- (conjugate symmetry) $\langle y,x\rangle$ $=\overline{\langle x,y\rangle}$ (over $\mathbb{R}$: symmetry);
- (positivity) $\langle x,x\rangle\gt0$ for every $x\ne0$.
Consequently $\langle x,ay+bz\rangle$ $=\bar a\langle x,y\rangle+\bar b\langle x,z\rangle$ (conjugate-linear in the second slot) and $\langle 0,x\rangle=0$. The norm is $\|x\|$ $=\sqrt{\langle x,x\rangle}$. Standard examples: on $\mathbb{F}^n$, $\langle x,y\rangle$ $=y^*x$ $=\sum_ix_i\bar y_i$ (the dot product when $\mathbb{F}=\mathbb{R}$); on $M_n(\mathbb{F})$, $\langle A,B\rangle$ $=\operatorname{tr}(B^*A)$ $=\sum_{i,j}a_{ij}\bar b_{ij}$; on polynomials or continuous functions, $\langle f,g\rangle$ $=\int_a^bf(x)\overline{g(x)}\,dx$; and on $\mathbb{R}^n$, $\langle x,y\rangle_M$ $=y^{\mathsf T}Mx$ for a positive definite matrix $M$ (Theorem 3.44 shows these are all the inner products on $\mathbb{R}^n$).
Proof. If $y=0$ both sides are $0$. Otherwise put $c$ $=\langle x,y\rangle/\|y\|^2$ and $w=x-cy$. Then $\langle w,y\rangle$ $=\langle x,y\rangle-c\langle y,y\rangle$ $=0$, so the cross terms in $\|w+cy\|^2$ vanish and
$$\|x\|^2=\|w\|^2+|c|^2\|y\|^2\ge\frac{|\langle x,y\rangle|^2}{\|y\|^2},$$with equality exactly when $w=0$, i.e. $x=cy$. For the triangle inequality, expand and use $\operatorname{Re}\langle x,y\rangle$ $\le|\langle x,y\rangle|$ $\le\|x\|\|y\|$:
$$\begin{aligned}\|x+y\|^2&=\|x\|^2+2\operatorname{Re}\langle x,y\rangle+\|y\|^2\\&\le(\|x\|+\|y\|)^2.\qquad\blacksquare\end{aligned}$$Proof. (a) If $\sum_ia_iu_i=0$, take the inner product with $u_j$: every term with $i\ne j$ vanishes, leaving $a_j\|u_j\|^2=0$, so $a_j=0$. (b) Write $x=\sum_ia_ie_i$; the inner product with $e_j$ gives $a_j=\langle x,e_j\rangle$. Expanding $\langle x,y\rangle$ with $y=\sum_jb_je_j$ gives $\sum_{i,j}a_i\bar b_j\langle e_i,e_j\rangle$ $=\sum_ia_i\bar b_i$, and $y=x$ gives the norm formula. ∎
So orthonormal bases make coordinates free: no linear system has to be solved, a coefficient is one inner product. The Gram–Schmidt process manufactures such bases from any basis.
Let $v_1,\dots,v_k$ be linearly independent. Define recursively
$$u_1=v_1,\qquad u_j=v_j-\sum_{i=1}^{j-1}\frac{\langle v_j,u_i\rangle}{\|u_i\|^2}\,u_i,\qquad e_j=\frac{u_j}{\|u_j\|}.$$Then $e_1,\dots,e_k$ is orthonormal and $\operatorname{span}\{e_1,\dots,e_j\}$ $=\operatorname{span}\{v_1,\dots,v_j\}$ for every $j$. If the $v$'s are dependent, the process produces $u_j=0$ exactly at the first $v_j$ lying in the span of $v_1,\dots,v_{j-1}$. Consequences: every finite-dimensional inner product space has an ONB; every orthonormal set extends to an ONB; and a matrix $A\in M_{m\times k}(\mathbb{F})$ with independent columns factors as $A=QR$, where $Q$ has orthonormal columns $e_j$ and $R$ is upper triangular with $r_{ij}$ $=\langle v_j,e_i\rangle$ and positive diagonal $r_{jj}=\|u_j\|$.
Proof. Induction on $j$. Suppose $u_1,\dots,u_{j-1}$ are nonzero, pairwise orthogonal and span $S_{j-1}$ $=\operatorname{span}\{v_1,\dots,v_{j-1}\}$. For $l\lt j$, orthogonality kills every term of the sum except $i=l$:
$$\langle u_j,u_l\rangle=\langle v_j,u_l\rangle-\frac{\langle v_j,u_l\rangle}{\|u_l\|^2}\|u_l\|^2=0.$$Also $u_j\in v_j+S_{j-1}$, so $u_j=0$ would put $v_j$ in $S_{j-1}$, contradicting independence; and $\operatorname{span}\{u_1,\dots,u_j\}$ $=S_j$ because each of $u_j$, $v_j$ is the other plus an element of $S_{j-1}$. Normalising changes nothing. For the extension statement, apply the process to the orthonormal set followed by a basis of $V$ and discard the zero vectors that appear; orthonormal vectors at the start are left unchanged. Finally $v_j$ $=u_j+\sum_{i\lt j}(\cdots)u_i$ expresses $v_j$ in terms of $e_1,\dots,e_j$ only, with coefficients $\langle v_j,e_i\rangle$ (Theorem 3.30) and $\langle v_j,e_j\rangle$ $=\langle u_j,e_j\rangle$ $=\|u_j\|$ $\gt0$: this is $A=QR$. ∎
Let $W$ be a subspace of $V$ with ONB $e_1,\dots,e_k$, and define the orthogonal projection $P_Wx$ $=\sum_{i=1}^k\langle x,e_i\rangle e_i$. Then:
- $x-P_Wx\in W^\perp$; hence $V=W\oplus W^\perp$, $\dim W^\perp=\dim V-\dim W$ and $(W^\perp)^\perp=W$.
- (Best approximation) $\|x-P_Wx\|\lt\|x-w\|$ for every $w\in W$ with $w\ne P_Wx$, and $\operatorname{dist}(x,W)^2$ $=\|x\|^2-\sum_{i=1}^k|\langle x,e_i\rangle|^2$.
- In $\mathbb{F}^n$ with the standard inner product: if the columns of $A$ form a basis of $W$, the matrix of $P_W$ is $A(A^*A)^{-1}A^*$; if they are orthonormal, it is $AA^*$. It satisfies $P^2=P=P^*$.
Proof. (1) $\langle x-P_Wx,e_j\rangle$ $=\langle x,e_j\rangle-\langle x,e_j\rangle$ $=0$ for each $j$, so $x-P_Wx$ is orthogonal to $W$. Thus $x=P_Wx+(x-P_Wx)$ lies in $W+W^\perp$, and the sum is direct because $w\in W\cap W^\perp$ gives $\langle w,w\rangle=0$. Dimensions add, and $W\subseteq(W^\perp)^\perp$ with equal dimensions. (2) For $w\in W$, $x-w=(x-P_Wx)+(P_Wx-w)$ with the first term in $W^\perp$ and the second in $W$, so by Pythagoras $\|x-w\|^2$ $=\|x-P_Wx\|^2+\|P_Wx-w\|^2$. The distance formula is Pythagoras again: $\|x\|^2$ $=\|P_Wx\|^2+\|x-P_Wx\|^2$. (3) $P_Wx=Ac$ where $x-Ac\perp$ every column, i.e. $A^*(x-Ac)=0$, so $c=(A^*A)^{-1}A^*x$ ($A^*A$ is invertible because $A^*Ac=0$ gives $\|Ac\|^2=0$). ∎
Part (3) is the method of least squares: the best solution of an inconsistent system $Ac\approx b$ solves the normal equations $A^*Ac=A^*b$, and the residual $b-Ac$ is orthogonal to the column space (Exercise I.9).
The Adjoint and the Special Classes of Matrices
For the standard inner product on $\mathbb{F}^n$, $\langle Ax,y\rangle$ $=y^*Ax$ $=(A^*y)^*x$ $=\langle x,A^*y\rangle$. So $A^*$ is the matrix that "moves across" the inner product; it is called the adjoint of $A$. (On an abstract inner product space the adjoint $T^*$ of an operator is defined by $\langle Tx,y\rangle$ $=\langle x,T^*y\rangle$; its matrix in an orthonormal basis is the conjugate transpose, but in a non-orthonormal basis it is not.) The rules $(AB)^*=B^*A^*$, $(A^*)^*=A$, $(A^{-1})^*=(A^*)^{-1}$, $\det A^*=\overline{\det A}$ follow from those for the transpose.
| real matrices | complex matrices | condition |
|---|---|---|
| symmetric | Hermitian | $A^{\mathsf T}=A$, resp. $A^*=A$ |
| skew-symmetric | skew-Hermitian | $A^{\mathsf T}=-A$, resp. $A^*=-A$ |
| orthogonal | unitary | $A^{\mathsf T}A=I$, resp. $A^*A=I$ |
| (real) normal | normal | $A^*A=AA^*$ |
A real matrix is Hermitian iff it is symmetric and unitary iff it is orthogonal. Hermitian, skew-Hermitian and unitary matrices are normal ($A^*$ is $A$, $-A$ or $A^{-1}$, each of which commutes with $A$); so are all diagonal matrices and all matrices $\begin{pmatrix}a&-b\\b&a\end{pmatrix}$. In a Hermitian matrix the diagonal is real and $a_{ji}=\overline{a_{ij}}$; in a skew-Hermitian matrix the diagonal is purely imaginary (or $0$) and $a_{ji}=-\overline{a_{ij}}$. A complex symmetric matrix ($A^{\mathsf T}=A$) is in general none of these.
A useful analogy: $A\mapsto A^*$ behaves like $z\mapsto\bar z$. Hermitian matrices play the part of real numbers, skew-Hermitian ones of imaginary numbers, unitary ones of numbers of modulus $1$; every matrix splits uniquely as $A=H+iK$ with $H=\tfrac12(A+A^*)$ and $K=\tfrac1{2i}(A-A^*)$ Hermitian (Exercise II.2), like $z=x+iy$. The analogy becomes exact for normal matrices (Theorem 3.39).
Proof. (a)$\Leftrightarrow$(b): for square matrices a one-sided inverse is a two-sided inverse. (a)$\Leftrightarrow$(c): the $(i,j)$ entry of $U^*U$ is $u_i^*u_j$ $=\langle u_j,u_i\rangle$ for the columns $u_i$; (b)$\Leftrightarrow$(d) in the same way with rows. (a)$\Rightarrow$(e): $\langle Ux,Uy\rangle$ $=y^*U^*Ux$ $=y^*x$. (e)$\Rightarrow$(f): take $y=x$. (f)$\Rightarrow$(e): norms determine inner products by the polarisation identity $\langle x,y\rangle$ $=\tfrac14\sum_{k=0}^{3}i^k\|x+i^ky\|^2$ (over $\mathbb{R}$: $\tfrac14(\|x+y\|^2-\|x-y\|^2)$). (e)$\Rightarrow$(a): $\langle(U^*U-I)x,y\rangle$ $=\langle Ux,Uy\rangle-\langle x,y\rangle$ $=0$ for all $y$, so $(U^*U-I)x=0$ for all $x$. Finally $(UV)^*UV=V^*U^*UV=I$, $U^{-1}=U^*$ is unitary, and $|\det U|^2$ $=\overline{\det U}\det U$ $=\det(U^*U)$ $=1$. ∎
- Hermitian ($A^*=A$): $x^*Ax$ is real for every $x$; every eigenvalue is real; eigenvectors for distinct eigenvalues are orthogonal; $\det A$ and $\operatorname{tr}A$ are real. A real symmetric matrix has real eigenvalues and real eigenvectors.
- Skew-Hermitian ($A^*=-A$): $iA$ is Hermitian, so every eigenvalue is purely imaginary or $0$. A real skew-symmetric matrix has eigenvalues $0$ or $\pm i\beta$ in conjugate pairs; if $n$ is odd, $\det A=0$; and $I+A$ is always invertible.
- Unitary: every eigenvalue has $|\lambda|=1$, $|\det U|=1$, and eigenvectors for distinct eigenvalues are orthogonal. A real orthogonal matrix has $\det=\pm1$ and its non-real eigenvalues come in pairs $e^{\pm i\theta}$. If $Q$ is $3\times3$ real orthogonal with $\det Q=1$, then $1$ is an eigenvalue, $Q$ is a rotation through an angle $\theta$ about the line of that eigenvector, and $\operatorname{tr}Q$ $=1+2\cos\theta$.
Proof. Let $Ax=\lambda x$ with $x\ne0$, so $\langle Ax,x\rangle$ $=\lambda\|x\|^2$ and $\langle x,Ax\rangle$ $=\bar\lambda\|x\|^2$. (1) If $A=A^*$ then $\overline{x^*Ax}$ $=(x^*Ax)^*$ $=x^*A^*x$ $=x^*Ax$; and $\langle Ax,x\rangle$ $=\langle x,Ax\rangle$ gives $\lambda=\bar\lambda$. If also $Ay=\mu y$ with $\mu\ne\lambda$ (both real), then $\lambda\langle x,y\rangle$ $=\langle Ax,y\rangle$ $=\langle x,Ay\rangle$ $=\mu\langle x,y\rangle$, so $\langle x,y\rangle=0$. The determinant and trace are the product and sum of the (real) eigenvalues. For real symmetric $A$, $A-\lambda I$ is a real singular matrix, so it has a real null vector. (2) $(iA)^*=-iA^*=iA$, so $i\lambda$ is real. For real skew-symmetric $A$ the characteristic polynomial is real, so non-real roots pair up; $\det A$ $=\det A^{\mathsf T}$ $=\det(-A)$ $=(-1)^n\det A$ forces $\det A=0$ for odd $n$; and $-1$ is not purely imaginary, so it is not an eigenvalue and $I+A$ is invertible. (3) $\|x\|=\|Ux\|=|\lambda|\|x\|$. If $Uy=\mu y$ with $\mu\ne\lambda$, then $\langle x,y\rangle$ $=\langle Ux,Uy\rangle$ $=\lambda\bar\mu\langle x,y\rangle$, and $\lambda\bar\mu$ $=\lambda/\mu$ $\ne1$, so $\langle x,y\rangle=0$. For $3\times3$ real orthogonal $Q$ with $\det Q=1$: a real cubic has a real root, so the eigenvalues are either $1,e^{i\theta},e^{-i\theta}$ or three real numbers $\pm1$ with product $1$, i.e. $\{1,1,1\}$ or $\{1,-1,-1\}$; in every case $1$ occurs. In an ONB whose first vector spans that eigenline, $Q$ fixes the axis and acts on the perpendicular plane as a $2\times2$ orthogonal matrix of determinant $1$, a rotation by $\theta$, so $\operatorname{tr}Q$ $=1+2\cos\theta$. ∎
Unitary Diagonalisation: Schur and the Spectral Theorem
A matrix $A$ is unitarily diagonalisable if $U^*AU$ is diagonal for some unitary $U$ — equivalently, if $\mathbb{C}^n$ has an orthonormal basis of eigenvectors of $A$ (the columns of $U$). It is orthogonally diagonalisable if this can be done with a real orthogonal $Q$. Every matrix can at least be triangularised unitarily.
Proof. Induction on $n$; $n=1$ is trivial. Pick an eigenvalue $\lambda_1$ (one exists by the fundamental theorem of algebra) and a unit eigenvector $u_1$, and extend it to an ONB $u_1,\dots,u_n$ of $\mathbb{C}^n$ (Theorem 3.31). With $U_1=[u_1\ \cdots\ u_n]$, the first column of $U_1^*AU_1$ is $U_1^*Au_1$ $=\lambda_1U_1^*u_1$ $=\lambda_1e_1$, so
$$U_1^*AU_1=\begin{pmatrix}\lambda_1&*\\0&A_1\end{pmatrix},\qquad A_1\in M_{n-1}(\mathbb{C}).$$By induction $A_1=VT_1V^*$ with $V$ unitary and $T_1$ upper triangular; then $U$ $=U_1\begin{pmatrix}1&0\\0&V\end{pmatrix}$ is unitary and $U^*AU$ $=\begin{pmatrix}\lambda_1&*\\0&T_1\end{pmatrix}$ is upper triangular. Similar matrices have the same characteristic polynomial, which for $T$ is $\prod(t-t_{ii})$. In the real case choose a real eigenvector for the real eigenvalue $\lambda_1$, run Gram–Schmidt in $\mathbb{R}^n$, and note that $A_1$ again has only real eigenvalues (the characteristic polynomial of $A$ is $(t-\lambda_1)$ times that of $A_1$). ∎
Proof. (a) $\|Ax\|^2$ $=\langle A^*Ax,x\rangle$ $=\langle AA^*x,x\rangle$ $=\|A^*x\|^2$. (b) $A-\lambda I$ is normal too (both products equal $A^*A-\lambda A^*-\bar\lambda A+|\lambda|^2I$), and its adjoint is $A^*-\bar\lambda I$; apply (a) to it. (c) If $Ax=\lambda x$ and $Ay=\mu y$, then by (b) $\lambda\langle x,y\rangle$ $=\langle Ax,y\rangle$ $=\langle x,A^*y\rangle$ $=\langle x,\bar\mu y\rangle$ $=\mu\langle x,y\rangle$. (d) Let $T$ be upper triangular and normal. Compare the $(1,1)$ entries of $T^*T$ and $TT^*$: they are $|t_{11}|^2$ and $\sum_j|t_{1j}|^2$. Hence $t_{1j}=0$ for $j\ge2$, so $T$ $=\operatorname{diag}(t_{11},T')$ with $T'$ upper triangular and normal, and induction finishes. ∎
A matrix $A\in M_n(\mathbb{C})$ is unitarily diagonalisable if and only if it is normal. In that case
$$A=UDU^*=\sum_{j}\lambda_jP_j,$$where $U$ is unitary, $D$ $=\operatorname{diag}(\lambda_1,\dots,\lambda_n)$, and in the second form $\lambda_j$ runs over the distinct eigenvalues and $P_j$ is the orthogonal projection onto the eigenspace of $\lambda_j$ ($P_j^2=P_j=P_j^*$, $P_jP_k=0$ for $j\ne k$, $\sum_jP_j=I$). For every polynomial $f$ (and, by definition, every function on the spectrum), $f(A)$ $=Uf(D)U^*$ $=\sum_jf(\lambda_j)P_j$.
Proof. If $A=UDU^*$, then $A^*A=UD^*DU^*=UDD^*U^*=AA^*$ because diagonal matrices commute. Conversely, let $A$ be normal and write $A=UTU^*$ by Schur's theorem. Then $T=U^*AU$ is normal ($T^*T=U^*A^*AU=U^*AA^*U=TT^*$) and upper triangular, hence diagonal by Theorem 3.36(d). The columns of $U$ are orthonormal eigenvectors; grouping those for each distinct eigenvalue $\lambda_j$ into $U_j$ gives $P_j=U_jU_j^*$ and $A=\sum\lambda_jP_j$. Finally $A^k=UD^kU^*$, and linear combinations give $f(A)$. ∎
Proof. If $A=QDQ^{\mathsf T}$ then $A^{\mathsf T}$ $=QD^{\mathsf T}Q^{\mathsf T}$ $=A$. Conversely, a real symmetric matrix has only real eigenvalues (Theorem 3.34), so the real form of Schur's theorem gives $Q^{\mathsf T}AQ=T$ real upper triangular; but $T^{\mathsf T}$ $=Q^{\mathsf T}A^{\mathsf T}Q$ $=T$, and a symmetric triangular matrix is diagonal. ∎
A real normal matrix that is not symmetric, such as the rotation $\begin{pmatrix}0&-1\\1&0\end{pmatrix}$, is unitarily diagonalisable (eigenvalues $\pm i$, eigenvectors $(1,\mp i)$) but not orthogonally: it has no real eigenvectors at all. The best one can do over $\mathbb{R}$ is block-diagonal form with $2\times2$ blocks $\begin{pmatrix}a&-b\\b&a\end{pmatrix}$ (a statement we do not prove here).
Let $A$ be normal with eigenvalues $\lambda_1,\dots,\lambda_n$. Then $A$ is Hermitian $\Leftrightarrow$ every $\lambda_j\in\mathbb{R}$; skew-Hermitian $\Leftrightarrow$ every $\lambda_j\in i\mathbb{R}$; unitary $\Leftrightarrow$ every $|\lambda_j|=1$; an orthogonal projection ($A^2=A$) $\Leftrightarrow$ every $\lambda_j\in\{0,1\}$; nilpotent $\Leftrightarrow$ $A=0$. For every $A\in M_n(\mathbb{C})$ (Schur's inequality)
$$\sum_{j=1}^n|\lambda_j|^2\le\sum_{i,j}|a_{ij}|^2=\operatorname{tr}(A^*A),$$with equality if and only if $A$ is normal.
Proof. With $A=UDU^*$ we have $A^*=U\bar DU^*$, $A^*A=U|D|^2U^*$ and $A^2=UD^2U^*$; so $A^*=A$ iff $\bar D=D$, $A^*=-A$ iff $\bar D=-D$, $A^*A=I$ iff $|D|^2=I$, $A^2=A$ iff $D^2=D$, and $A^k=0$ iff $D^k=0$ iff $D=0$. For the inequality write $A=UTU^*$ (Schur). The trace is a similarity invariant and $A^*A=U(T^*T)U^*$, so
$$\begin{aligned}\operatorname{tr}(A^*A)&=\operatorname{tr}(T^*T)=\sum_{i\le j}|t_{ij}|^2\\&\ge\sum_i|t_{ii}|^2=\sum_j|\lambda_j|^2,\end{aligned}$$with equality iff $T$ is diagonal; and $T$ diagonal implies $A$ normal, while $A$ normal implies $T$ normal, hence diagonal. ∎
Without normality every one of these equivalences fails: $\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has only the eigenvalue $1$ but is not unitary; $\begin{pmatrix}0&1\\0&0\end{pmatrix}$ is nilpotent and nonzero; $J$ of Example 3 has real eigenvalues and is not Hermitian. Schur's inequality is a quick normality test: for $J$, $\sum|a_{ij}|^2=14\gt1+9$.
- Check the hypothesis: real symmetric (orthogonal $Q$), or normal (unitary $U$). If the matrix is not normal, no unitary $U$ exists, however nice the eigenvalues.
- Find the eigenvalues. Use $\operatorname{tr}A$ $=\sum\lambda$, $\det A=\prod\lambda$, and obvious eigenvectors such as $(1,\dots,1)$ for constant row sums.
- For each eigenvalue find a basis of the eigenspace. Eigenspaces of different eigenvalues are automatically orthogonal (Theorem 3.36): you only need Gram–Schmidt inside a repeated eigenspace. In $\mathbb{R}^3$, a $2$-dimensional eigenspace is the plane orthogonal to the third eigenvector.
- Normalise and place the vectors as the columns of $Q$ or $U$, in the order of the eigenvalues in $D$. Then $A=QDQ^{\mathsf T}$, $A^k=QD^kQ^{\mathsf T}$, $A^{-1}=QD^{-1}Q^{\mathsf T}$; or use projections: $A=\sum\lambda_jP_j$, with $P=uu^*$ for a unit eigenvector $u$ of a simple eigenvalue and $I-P$ for the complementary eigenspace.
Bilinear and Quadratic Forms; Sylvester's Law of Inertia
From here on the scalars are real unless we say otherwise. A quadratic form is a homogeneous polynomial of degree $2$, such as the second-order part of a Taylor expansion; it is the restriction to the diagonal $x=y$ of a symmetric bilinear form, and both are encoded by a symmetric matrix.
A bilinear form on a real vector space $V$ is a map $B:V\times V\to\mathbb{R}$ linear in each variable; it is symmetric if $B(x,y)=B(y,x)$ and skew-symmetric if $B(x,y)=-B(y,x)$. Its matrix in a basis $v_1,\dots,v_n$ is $M=\big(B(v_i,v_j)\big)$, so that $B(x,y)=[x]^{\mathsf T}M[y]$ in coordinates. A quadratic form is $q(x)=B(x,x)$ for a symmetric bilinear $B$. On $\mathbb{R}^n$:
$$q(x)=x^{\mathsf T}Ax=\sum_ia_{ii}x_i^2+2\sum_{i\lt j}a_{ij}x_ix_j,\qquad A=A^{\mathsf T}.$$So the coefficient of $x_ix_j$ ($i\ne j$) is twice the matrix entry: cross terms are halved when they go into $A$. The rank of $q$ is $\operatorname{rank}A$. Two symmetric matrices are congruent if $A'=P^{\mathsf T}AP$ for some invertible $P$.
- (Change of variables) If $x=Px'$ with $P$ invertible (a change of basis), the matrix of $q$ becomes $P^{\mathsf T}AP$. Congruent matrices have the same rank, and $\det(P^{\mathsf T}AP)$ $=(\det P)^2\det A$ has the same sign as $\det A$.
- (Polarisation) $B(x,y)$ $=\tfrac12\big(q(x+y)-q(x)-q(y)\big)$: a quadratic form determines its symmetric bilinear form, and $q\leftrightarrow B\leftrightarrow A$ are one-to-one correspondences.
- Every bilinear form is uniquely the sum of a symmetric and a skew-symmetric one, $M$ $=\tfrac12(M+M^{\mathsf T})+\tfrac12(M-M^{\mathsf T})$, and $x^{\mathsf T}Mx$ $=x^{\mathsf T}\tfrac12(M+M^{\mathsf T})x$. On an $n$-dimensional space the bilinear forms form a space of dimension $n^2$, the symmetric ones $\tfrac{n(n+1)}2$, the skew-symmetric ones $\tfrac{n(n-1)}2$.
Proof. (1) $q(Px')$ $=x'^{\mathsf T}(P^{\mathsf T}AP)x'$; multiplying by invertible matrices does not change rank. (2) Expand $B(x+y,x+y)$ $=q(x)+2B(x,y)+q(y)$ using symmetry. (3) The two parts are symmetric and skew-symmetric, and uniqueness holds because a matrix that is both is $0$; a skew-symmetric $S$ has $x^{\mathsf T}Sx$ $=(x^{\mathsf T}Sx)^{\mathsf T}$ $=-x^{\mathsf T}Sx$, so $x^{\mathsf T}Sx=0$. The dimensions count the free entries: all $n^2$; those on or above the diagonal; those strictly above it. ∎
Proof. (a) is the real spectral theorem. (b) Induction on the number of variables. If some $a_{ii}\ne0$, say $a_{11}$, collect the terms containing $x_1$: $a_{11}x_1^2+2x_1L$ $=a_{11}\left(x_1+L/a_{11}\right)^2-L^2/a_{11}$, where $L$ is linear in $x_2,\dots,x_n$; put $z_1=x_1+L/a_{11}$ and repeat on the rest, a form in $x_2,\dots,x_n$. If every $a_{ii}=0$ but some $a_{ij}\ne0$, substitute $x_i=u+v$, $x_j=u-v$: then $2a_{ij}x_ix_j$ $=2a_{ij}(u^2-v^2)$ creates square terms. If $q=0$ there is nothing to do. (c) is immediate. ∎
Proof. The rank is invariant (Theorem 3.40), so $n_++n_-$ and $n_0$ are fixed; it remains to show $n_+$ is. Suppose $q$ $=\sum_{i\le p}y_i^2-\sum_{p\lt i\le r}y_i^2$ and also $q$ $=\sum_{i\le p'}z_i^2-\sum_{p'\lt i\le r}z_i^2$ (after rescaling), with $p\gt p'$. Let $U$ $=\{x:y_{p+1}=\cdots=y_n=0\}$, of dimension $p$, on which $q(x)=\sum_{i\le p}y_i^2\gt0$ for $x\ne0$; and $W=\{x:z_1=\cdots=z_{p'}=0\}$, of dimension $n-p'$, on which $q\le0$. Since $p+(n-p')\gt n$, $U\cap W$ contains some $x\ne0$, and then $q(x)\gt0$ and $q(x)\le0$ at once — a contradiction. So $p\le p'$, and by symmetry $p=p'$. Taking the principal-axes diagonalisation identifies $n_+$, $n_-$ with the eigenvalue counts. ∎
Proof. Same inertia: both are congruent to the same $\operatorname{diag}(I_{n_+},-I_{n_-},0)$ (Theorem 3.41(c)), and congruence is an equivalence relation. Congruent: the same form in two coordinate systems, so the same inertia by Theorem 3.42. The classes correspond to triples of non-negative integers with $n_++n_-+n_0=n$; there are $\binom{n+2}{2}$ of them, and $n+1$ with $n_0=0$. ∎
Congruence is not similarity: $\operatorname{diag}(1,4)$ and $I_2$ are congruent (both have inertia $(2,0,0)$) but not similar (different eigenvalues). Congruence remembers only the signs of the eigenvalues. For real symmetric matrices, similar implies orthogonally similar (both are $QDQ^{\mathsf T}$ with the same $D$), which implies congruent.
- Write the symmetric matrix, halving cross terms.
- If the eigenvalues are easy (block structure, constant row sums, $J$-type matrices), count their signs.
- Otherwise complete squares (Lagrange); only the signs of the coefficients matter. A product of two independent linear forms is $\ell_1\ell_2$ $=\left(\tfrac{\ell_1+\ell_2}2\right)^2-\left(\tfrac{\ell_1-\ell_2}2\right)^2$: inertia $(1,1,n-2)$.
- Sign-counting shortcuts: $\det A\lt0$ means an odd number of negative eigenvalues; $\det A\gt0$ an even number; combine with the trace (e.g. $3\times3$ with $\det\lt0$ and $\operatorname{tr}\gt0$ has exactly one negative eigenvalue, since three negative ones would make the trace negative). A positive and a negative diagonal entry already force indefiniteness.
Definiteness, Sylvester's Criterion and the Rayleigh Quotient
For a Hermitian (in particular, real symmetric) $n\times n$ matrix $A$ the following are equivalent:
- $A$ is positive definite;
- every eigenvalue of $A$ is positive;
- $A=B^*B$ for some invertible $B$;
- (Sylvester's criterion) $\Delta_1$ $\gt0,\ \Delta_2$ $\gt0,\ \dots,\ \Delta_n$ $\gt0$;
- $\langle x,y\rangle_A=y^*Ax$ is an inner product on $\mathbb{F}^n$.
Consequences: a positive definite $A$ has $a_{ii}\gt0$, $|a_{ij}|^2\lt a_{ii}a_{jj}$, $\det A\gt0$, every principal submatrix positive definite, $A^{-1}$ and $A^k$ positive definite, and a unique positive definite square root $A^{1/2}=UD^{1/2}U^*$. $A$ is negative definite iff $(-1)^k\Delta_k\gt0$ for all $k$ (signs $-,+,-,\dots$).
Proof. (1)$\Leftrightarrow$(2): write $A=UDU^*$ and $y=U^*x$ (so $y\ne0$ iff $x\ne0$); then $x^*Ax$ $=y^*Dy$ $=\sum_i\lambda_i|y_i|^2$, which is positive for all $y\ne0$ iff every $\lambda_i\gt0$. (2)$\Rightarrow$(3): $B=D^{1/2}U^*$. (3)$\Rightarrow$(1): $x^*B^*Bx=\|Bx\|^2\gt0$ for $x\ne0$. (1)$\Leftrightarrow$(5): linearity in the first slot is automatic, conjugate symmetry is $A^*=A$, and positivity is (1).
(1)$\Rightarrow$(4): for $x=(x',0)$ with $x'\in\mathbb{F}^k$ we get $x^*Ax=x'^*A_kx'$, so each $A_k$ is positive definite and $\Delta_k=\det A_k$, the product of its positive eigenvalues, is positive. (4)$\Rightarrow$(1): induction on $n$, the case $n=1$ being clear. By the induction hypothesis $A_{n-1}$ is positive definite. Suppose $A$ were not. Since $\det A=\Delta_n\gt0$ is the product of the eigenvalues, none is $0$ and an even number are negative — so at least two, with orthonormal eigenvectors $u,w$ and eigenvalues $\mu,\nu\lt0$. On $S$ $=\operatorname{span}\{u,w\}$, $q(au+bw)$ $=\mu|a|^2+\nu|b|^2$ $\lt0$ unless $a=b=0$. But $S$ (dimension $2$) and $H=\{x:x_n=0\}$ (dimension $n-1$) have $2+(n-1)\gt n$, so they share some $x\ne0$; for it $q(x)\lt0$ (as $x\in S$) while $q(x)=x'^*A_{n-1}x'\gt0$ (as $x\in H$). Contradiction. ∎
The consequences follow by restricting to coordinate subspaces ($a_{ii}=e_i^*Ae_i$; a $2\times2$ principal submatrix has positive determinant $a_{ii}a_{jj}-|a_{ij}|^2$), from the eigenvalues ($\lambda^{-1},\lambda^k\gt0$), and from the spectral theorem for the square root (uniqueness: a positive definite square root $S$ of $A$ commutes with $A=S^2$, so it preserves the eigenspaces of $A$, and on each of them it must act as $\sqrt\lambda$).
Proof. (a)$\Leftrightarrow$(b)$\Leftrightarrow$(c) as in Theorem 3.44, with $\ge$ in place of $\gt$ and $B=D^{1/2}U^*$, now possibly singular. (a)$\Rightarrow$(d): a principal submatrix is the matrix of $q$ restricted to a coordinate subspace, hence positive semidefinite, and its determinant is a product of non-negative eigenvalues. (d)$\Rightarrow$(a): expanding the determinant, $\det(A_k+\varepsilon I_k)$ $=\varepsilon^k+c_1\varepsilon^{k-1}+\cdots+c_k$, where $c_j$ is the sum of the $j\times j$ principal minors of $A_k$, all $\ge0$ by hypothesis. So for every $\varepsilon\gt0$ all leading minors of $A+\varepsilon I$ are positive, $A+\varepsilon I$ is positive definite by Sylvester's criterion, and $x^*Ax$ $=\lim_{\varepsilon\to0^+}x^*(A+\varepsilon I)x$ $\ge0$. ∎
Leading minors are not enough for semidefiniteness: $\operatorname{diag}(0,-1)$ has $\Delta_1=\Delta_2=0$ yet is negative semidefinite (the principal minor $-1$ gives it away). And in Sylvester's criterion all the leading minors must be checked — Example 7 shows the last one can fail after the others pass.
Proof. With $A=UDU^*$ and $y=U^*x$ (so $\|y\|=\|x\|$), $R_A(x)$ $=\sum_i\lambda_i|y_i|^2\big/\sum_i|y_i|^2$ is a weighted average of the eigenvalues with weights $|y_i|^2\ge0$. It equals $\lambda_{\max}$ iff all the weight sits on indices with $\lambda_i=\lambda_{\max}$, i.e. iff $x$ is in that eigenspace; similarly for $\lambda_{\min}$. Take $x=e_i$ for the diagonal entries, and apply the result to the positive semidefinite matrix $B^*B$, using $\|Bx\|^2=x^*B^*Bx$. ∎
(The real symmetric case was obtained in §1.2 by Lagrange multipliers: at a constrained extremum of $x^{\mathsf T}Ax$ on the sphere, $\nabla(x^{\mathsf T}Ax)=2Ax$ is parallel to $\nabla(x^{\mathsf T}x)=2x$, so $x$ is an eigenvector.) Geometrically, the principal axes theorem turns the level set $q=1$ into a standard conic or quadric.
Let $A\ne0$ be real symmetric. In $\mathbb{R}^2$, with eigenvalues $\lambda_1\ge\lambda_2$, the curve $x^{\mathsf T}Ax=1$ is: an ellipse if $\lambda_2\gt0$, with semi-axes $1/\sqrt{\lambda_1}$ and $1/\sqrt{\lambda_2}$ along the eigenvectors and area $\pi/\sqrt{\det A}$; a hyperbola if $\lambda_1\gt0\gt\lambda_2$; a pair of parallel lines if $\lambda_1\gt0=\lambda_2$; empty if $\lambda_1\le0$. In $\mathbb{R}^3$ the surface $x^{\mathsf T}Ax=1$ is determined by the inertia:
| $(3,0,0)$ ellipsoid | $(2,1,0)$ hyperboloid of one sheet | $(1,2,0)$ hyperboloid of two sheets |
| $(2,0,1)$ elliptic cylinder | $(1,1,1)$ hyperbolic cylinder | $(1,0,2)$ two parallel planes |
and it is empty when $n_+=0$.
Proof. In the principal axes $y=Q^{\mathsf T}x$ (a rotation or reflection, which preserves shapes, lengths and areas) the equation is $\sum\lambda_iy_i^2=1$; read off each case. For the ellipse $\lambda_1y_1^2+\lambda_2y_2^2$ $=1$ the semi-axes are $a=1/\sqrt{\lambda_1}$, $b=1/\sqrt{\lambda_2}$ and the area is $\pi ab$ $=\pi/\sqrt{\lambda_1\lambda_2}$ $=\pi/\sqrt{\det A}$. ∎ (For $x^{\mathsf T}Ax=c$ with $c\gt0$, replace $A$ by $A/c$: the area becomes $\pi c/\sqrt{\det A}$.)
- Complex symmetric is not Hermitian. $\tfrac15\begin{pmatrix}3&4i\\4i&3\end{pmatrix}$ is symmetric with non-real eigenvalues; $\begin{pmatrix}1&i\\i&-1\end{pmatrix}$ is symmetric, nonzero, and squares to $0$. The Hermitian facts need $A^*=A$, with the conjugate.
- No conjugate, no inner product on $\mathbb{C}^n$. $z_1w_1+z_2w_2$ gives $\langle(i,0),(i,0)\rangle$ $=-1$.
- Real eigenvalues $\not\Rightarrow$ symmetric; diagonalisable $\not\Rightarrow$ unitarily diagonalisable. $\begin{pmatrix}1&2\\0&3\end{pmatrix}$: real distinct eigenvalues, eigenvectors at $45^\circ$, not normal.
- All $|\lambda|=1$ $\not\Rightarrow$ unitary. $\begin{pmatrix}1&1\\0&1\end{pmatrix}$; even a diagonalisable one: $\begin{pmatrix}1&2\\0&-1\end{pmatrix}$ has eigenvalues $\pm1$ and a column of length $\sqrt5$.
- Sums and products leave the classes. Symmetric $+$ skew-symmetric $=\begin{pmatrix}0&2\\0&0\end{pmatrix}$, not normal; $\operatorname{diag}(1,2)\begin{pmatrix}0&1\\1&0\end{pmatrix}$ $=\begin{pmatrix}0&1\\2&0\end{pmatrix}$, a product of Hermitian matrices that is not Hermitian ($AB$ is Hermitian iff $AB=BA$). Unitary $+$ unitary: $I+I=2I$ is not unitary.
- Real normal $\not\Rightarrow$ orthogonally diagonalisable. The rotation $\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ is orthogonal (so normal) with eigenvalues $\pm i$ and no real eigenvector.
- Use the symmetric part. For $A$ $=\begin{pmatrix}1&4\\0&1\end{pmatrix}$ the eigenvalues are $1,1$, yet $x^{\mathsf T}Ax$ $=x_1^2+4x_1x_2+x_2^2$ is indefinite (its matrix is $\begin{pmatrix}1&2\\2&1\end{pmatrix}$, eigenvalues $3,-1$). Halve the cross terms before testing anything.
- Positive diagonal and positive determinant $\not\Rightarrow$ positive definite ($n\ge3$): $\begin{pmatrix}1&2&2\\2&1&2\\2&2&1\end{pmatrix}$ has determinant $5$ and eigenvalues $5,-1,-1$. Check every leading minor ($\Delta_2=-3$ here).
- Leading minors $\ge0$ $\not\Rightarrow$ positive semidefinite. $\operatorname{diag}(0,-1)$. For semidefiniteness test all principal minors (Theorem 3.45) or the eigenvalues.
- Congruent $\not\Rightarrow$ similar. $\operatorname{diag}(1,4)$ and $I_2$. And equal rank and determinant $\not\Rightarrow$ congruent: $I_2$ and $-I_2$.
- Positive definite $A,B$ $\not\Rightarrow$ $AB$ positive definite. $AB$ need not be symmetric, and $x^{\mathsf T}ABx$ can be negative (Exercise V.3), although the eigenvalues of $AB$ are positive.
- Gram–Schmidt depends on the order and on the inner product. Vectors orthogonal for $\langle x,y\rangle_M$ need not look perpendicular (Interactive 1).
- "Signature" has two meanings in the literature ($n_+-n_-$ or the pair $(n_+,n_-)$). Here it is $n_+-n_-$.
- CSIR Unit 1 · §10.1 Inner Product Spaces and Orthonormal Bases — the Jordan–von Neumann theorem (which norms come from inner products), Gram determinants, Bessel's inequality and Parseval's identity, the adjoint of an operator on an abstract inner product space, and least squares in detail.
- CSIR Unit 1 · §10.2 Self-Adjoint, Normal and Unitary Operators — the spectral theorem for operators, real normal matrices, the functional calculus, positive square roots, the singular value and polar decompositions, Courant–Fischer, and Gershgorin's discs.
- CSIR Unit 1 · §10.3 Quadratic Forms: Reduction and Classification — Lagrange's reduction in full, congruence classes over $\mathbb{R}$ and $\mathbb{C}$, Jacobi's theorem (inertia from the signs of leading minors), and the complete classification of conics and quadrics with linear terms.
Two Labs
Exercises
- 1 Show that $\langle x,y\rangle$ $=3x_1y_1-x_1y_2-x_2y_1+x_2y_2$ is an inner product on $\mathbb{R}^2$, and find the angle between $e_1$ and $e_2$ for it.
- 2 For which real $c$ is $\langle x,y\rangle$ $=x_1y_1+c(x_1y_2+x_2y_1)+4x_2y_2$ an inner product on $\mathbb{R}^2$?
- 3 Apply Gram–Schmidt to $(1,1,1,1)$, $(1,1,0,0)$, $(1,0,1,0)$ in $\mathbb{R}^4$.
- 4 On $P_2(\mathbb{R})$ with $\langle p,q\rangle$ $=\int_0^1p(x)q(x)\,dx$, orthonormalise $1,x,x^2$. Find the polynomial of degree at most $1$ closest to $x^2$, and the distance.
- 5 Find the matrix of the orthogonal projection of $\mathbb{R}^3$ onto the plane $x-2y+2z=0$, and the distance from $(1,1,1)$ to the plane.
- 6 In an inner product space, show that $\|x+y\|=\|x\|+\|y\|$ with $y\ne0$ implies $x=ty$ for some $t\ge0$.
- 7 Prove the real polarisation identity $\langle x,y\rangle$ $=\tfrac14\left(\|x+y\|^2-\|x-y\|^2\right)$, and show that the norm $|x_1|+|x_2|$ on $\mathbb{R}^2$ does not come from any inner product.
- 8 On $M_3(\mathbb{R})$ with $\langle A,B\rangle$ $=\operatorname{tr}(B^{\mathsf T}A)$, let $W$ be the subspace of matrices of trace $0$. Find $\dim W^\perp$ and describe $W^\perp$.
- 9 Modelling. A cart starts at the origin and moves at constant speed $c$. Its measured positions at $t=1,2,3$ s are $2.1$, $3.9$, $6.2$ m. Find the least-squares estimate of $c$ as an orthogonal projection, the residual vector, and check that the residual is orthogonal to $(1,2,3)$.
- 1 Show that $\begin{pmatrix}0&1+i\\-1+i&0\end{pmatrix}$ is skew-Hermitian and find its eigenvalues.
- 2 Show that every $A\in M_n(\mathbb{C})$ is uniquely $H+iK$ with $H,K$ Hermitian, and find $H$, $K$ for $A$ $=\begin{pmatrix}1&2\\0&i\end{pmatrix}$.
- 3 (Cayley transform) Let $S$ be real skew-symmetric. Show that $I+S$ is invertible and $Q=(I-S)(I+S)^{-1}$ is orthogonal. Compute $Q$ for $S$ $=\begin{pmatrix}0&1\\-1&0\end{pmatrix}$.
- 4 Prove that a real skew-symmetric matrix of odd order is singular, and give an invertible $2\times2$ real skew-symmetric matrix.
- 5 Let $Q$ be a $3\times3$ real orthogonal matrix with $\det Q=-1$. Prove that $-1$ is an eigenvalue of $Q$.
- 6 Show that $R$ $=\tfrac13\begin{pmatrix}2&-1&2\\2&2&-1\\-1&2&2\end{pmatrix}$ is a rotation, and find its axis and angle.
- 7 Show that a matrix that is both unitary and Hermitian satisfies $U^2=I$ and has eigenvalues in $\{1,-1\}$; give one that is not $\pm I$.
- 8 Prove that a Hermitian $A$ with $A^2=0$ is $0$. Is the same true for complex symmetric matrices?
- 1 Orthogonally diagonalise $\begin{pmatrix}7&-2\\-2&4\end{pmatrix}$.
- 2 Orthogonally diagonalise $B$ $=\begin{pmatrix}1&0&2\\0&-1&0\\2&0&1\end{pmatrix}$ and compute $B^{10}$.
- 3 Unitarily diagonalise the Hermitian matrix $\begin{pmatrix}1&i\\-i&1\end{pmatrix}$.
- 4 Show that $\begin{pmatrix}3&1\\-1&3\end{pmatrix}$ is normal and unitarily diagonalise it. Is it orthogonally diagonalisable?
- 5 For which real $a,b$ is $\begin{pmatrix}1&a\\b&2\end{pmatrix}$ normal?
- 6 Prove that a normal matrix all of whose eigenvalues have modulus $1$ is unitary, and that a normal matrix is unitarily similar to a real diagonal matrix iff it is Hermitian.
- 7 A $3\times3$ normal matrix has eigenvalues $1$, $2i$, $-2$. Find $\sum_{i,j}|a_{ij}|^2$. Compare $\sum|a_{ij}|^2$ with $\sum|\lambda_j|^2$ for $\begin{pmatrix}1&1\\0&1\end{pmatrix}$.
- 1 Write the matrix of $q=3x^2-2xy+4xz+y^2-6yz+5z^2$ and find its rank and inertia without computing eigenvalues.
- 2 Reduce $x^2+6xy+5y^2$ to a sum of signed squares; find its rank, index and signature.
- 3 Show that $x^2+2y^2+2z^2+2xy+2xz+2yz$ is positive definite in two ways: by completing squares and by Sylvester's criterion.
- 4 Find the inertia and signature of $x_1x_2+x_2x_3+x_3x_4$ on $\mathbb{R}^4$.
- 5 For which $k$ is $x^2+2y^2+z^2+2kxy+2yz$ positive definite?
- 6 Let $\ell_1,\ell_2$ be linearly independent linear forms on $\mathbb{R}^n$. Show that $\ell_1^2$ is positive semidefinite of rank $1$ and that $\ell_1\ell_2$ has inertia $(1,1,n-2)$.
- 7 How many congruence classes of real symmetric $3\times3$ matrices are there? How many consist of invertible matrices? List the classes of $2\times2$ matrices.
- 8 Find the dimensions of the spaces of all, of symmetric, and of skew-symmetric bilinear forms on $\mathbb{R}^3$.
- 9 Identify the conic $2x^2+4xy-y^2=6$ and find its principal axes.
- 1 Every real square matrix all of whose eigenvalues are real is orthogonally similar to an upper triangular matrix.
- 2 If $A$ is real symmetric and $A^3=I$, then $A=I$.
- 3 If $A$ and $B$ are real symmetric positive definite, then $x^{\mathsf T}ABx\gt0$ for all $x\ne0$.
- 4 If a real symmetric matrix has all its leading principal minors $\ge0$, it is positive semidefinite.
- 5 Two real symmetric matrices with the same eigenvalues (with multiplicity) are orthogonally similar.
- 6 Two real symmetric $n\times n$ matrices with the same rank and the same determinant are congruent.
E1. GATE MA · MCQ Let $Q$ be a $3\times3$ real orthogonal matrix with $\det Q=1$ and $\operatorname{tr}Q=0$. The eigenvalues of $Q$ are (A) $1,\ i,\ -i$; (B) $1,\ -1,\ -1$; (C) $1,\ e^{2\pi i/3},\ e^{-2\pi i/3}$; (D) $1,\ e^{\pi i/3},\ e^{-\pi i/3}$.
E2. GATE MA · MCQ The rank and signature of the quadratic form $q(x_1,x_2,x_3,x_4)$ $=x_1x_2+x_3x_4$ are (A) rank $2$, signature $0$; (B) rank $4$, signature $0$; (C) rank $4$, signature $2$; (D) rank $2$, signature $2$.
E3. GATE MA · MSQ Let $A\in M_n(\mathbb{C})$ be normal. Which of the following is/are necessarily true? (One or more options may be correct.) (A) If every eigenvalue of $A$ is real, then $A$ is Hermitian. (B) $\|Ax\|=\|A^*x\|$ for every $x\in\mathbb{C}^n$. (C) If $A^3=0$, then $A=0$. (D) If $A$ has real entries, then $A$ is diagonalisable over $\mathbb{R}$.
E4. GATE MA · MSQ For $a\in\mathbb{R}$ let $M_a$ $=\begin{pmatrix}4&2&0\\2&a&1\\0&1&1\end{pmatrix}$. Which of the following is/are true? (One or more options may be correct.) (A) $M_3$ is positive definite. (B) $M_a$ is positive definite for every $a\gt1$. (C) $M_2$ is positive semidefinite and singular. (D) $M_0$ has exactly one negative eigenvalue.
E5. GATE MA · NAT In $\mathbb{R}^4$ with the dot product, let $W$ $=\operatorname{span}\{(1,1,0,0),(0,1,1,1)\}$ and $b=(2,0,1,3)$. The square of the distance from $b$ to $W$ is ______ (correct to two decimal places).
E6. GATE MA · NAT The area of the region enclosed by the ellipse $5x^2+6xy+5y^2=8$ is ______ (correct to two decimal places).
Suggested plan — 3 lectures × 60 min.
- Inner products. Definition 3.14 with the four standard examples; Cauchy–Schwarz with the projection proof (draw $w=x-cy$). Theorems 3.30–3.32, stressing "coefficients are inner products". Example 1 on the board (the rational ONB is a pleasant surprise). Project Interactive 1 with the dot product, then switch to $M$ $=\begin{pmatrix}2&1\\1&1\end{pmatrix}$ and ask why $e_1,e_2$ no longer look perpendicular. Example 2 as a class exercise.
- Special matrices and the spectral theorem. The table of Definition 3.16 and the $z\mapsto\bar z$ analogy. Theorem 3.34 with the one-line proofs; Example 3. Schur's theorem (sketch the block step), Theorem 3.36(d) on the board for a $2\times2$ triangular matrix, then Theorems 3.37–3.39. Example 4 using projections; Example 5(b) as a quick-fire true/false round.
- Forms and definiteness. Definition 3.17 (insist on halving cross terms), Lagrange's reduction on Example 6, the law of inertia with the dimension-count proof. Sylvester's criterion with its proof; Example 7 and the $\sqrt2\lt|k|\lt2$ trap. Interactive 2: drive $(a,b,c)$ from an ellipse to a hyperbola, and stop $\theta$ on the eigen-axes. Finish with GATE corner E4 and E6.
Discussion questions. (1) Every real symmetric matrix is orthogonally diagonalisable, but the rotation by $90^\circ$ — which is orthogonal and normal — is not. What exactly does symmetry add in the proof? (2) Why does the law of inertia say nothing about the sizes of the coefficients $d_i$, and why is that the right answer for congruence? (3) The Gram–Schmidt vectors in Interactive 1 for $\langle\cdot,\cdot\rangle_M$ are "conjugate diameters" of the ellipse $x^{\mathsf T}Mx=1$. What does that mean, and how does it relate to $M=R^{\mathsf T}R$?
Misconceptions to watch for. Forgetting the conjugate (complex symmetric treated as Hermitian); believing real eigenvalues, or eigenvalues of modulus $1$, decide the class without normality; using the eigenvalues of a non-symmetric $A$ to judge $x^{\mathsf T}Ax$; not halving cross terms; checking only some leading minors, or only leading minors for semidefiniteness; confusing congruence with similarity; using Gram–Schmidt across different eigenvalues (unnecessary) but forgetting it inside a repeated eigenspace (necessary).
Suggested homework. I(1, 3, 4, 5, 9), II(1, 3, 5, 6, 8), III(2, 3, 5, 7), IV(1, 4, 5, 7, 9), V(2, 3, 6); GATE corner E1, E3, E5.
Research question. At what frequencies does a three-storey building sway in an earthquake, what do its vibration modes look like, and how well does a single guessed shape predict the lowest frequency?
Why it matters. Earthquake engineers in Japan, Chile, Nepal, Turkey and New Zealand design buildings so that their natural frequencies avoid the frequencies at which the ground shakes hardest. In the simplest "shear building" model, with floor masses $m$ and storey stiffness $k$, the floor displacements satisfy $m\,\ddot x+kTx=0$ with the symmetric matrix $T$ $=\begin{pmatrix}2&-1&0\\-1&2&-1\\0&-1&1\end{pmatrix}$. Each eigenvalue $\lambda$ of $T$ gives a natural angular frequency $\omega=\sqrt{\lambda k/m}$, and the spectral theorem guarantees three perpendicular mode shapes. When the floors have different masses the problem becomes $Tx=\lambda Mx$ with a positive definite mass matrix $M$, and the modes are orthogonal for the inner product $\langle x,y\rangle_M$ $=y^{\mathsf T}Mx$ instead.
What to do.
- Check with Sylvester's criterion that $T$ is positive definite (why must it be, physically?).
- Find the eigenvalues and orthonormal eigenvectors of $T$ numerically. With $k/m=1000\ \text{s}^{-2}$, convert to frequencies $f=\omega/2\pi$ in Hz and sketch the three mode shapes (floor displacement against floor number).
- The Rayleigh quotient $R(x)$ $=x^{\mathsf T}Tx/x^{\mathsf T}x$ is an upper bound for the smallest eigenvalue (Theorem 3.46). Evaluate it for the guess $x=(1,2,3)$ and compare the resulting frequency with the exact one.
- Now let the two lower floors be twice as heavy: $M$ $=\operatorname{diag}(2,2,1)$. Solve $Tx=\lambda Mx$ (for instance by symmetrising with $y=M^{1/2}x$), and verify that the modes are $M$-orthogonal but not orthogonal for the dot product.
Tools: Python with numpy/scipy (numpy.linalg.eigh, scipy.linalg.eigh(T, M)), or any matrix calculator.
Hand in: the three frequencies and mode-shape sketches; the Rayleigh estimate and its percentage error; the $M$-orthogonality check; a short paragraph on why a building whose fundamental frequency is near $2$–$3$ Hz is vulnerable to shaking of that frequency (resonance).
Going further. Show that the eigenvalues of $T$ are $2-2\cos\frac{(2j-1)\pi}{7}$, $j=1,2,3$. Read about base isolation, which lowers the fundamental frequency of a building on purpose, and about tuned mass dampers in tall towers.
🧪 Self-check quiz
Q1. The eigenvalues of a skew-Hermitian matrix are always
Q2. Which matrix is normal but not Hermitian?
Q3. Gram–Schmidt with the dot product applied to $v_1=(1,1)$, $v_2=(1,0)$ (in this order) gives $e_2=$
Q4. On $\mathbb{R}^3$, the quadratic form $q(x,y,z)=x^2+2xy$ has
Q5. The matrix $\begin{pmatrix}1&k\\k&4\end{pmatrix}$ is positive definite exactly when