Chapter 7 · Introduction

Chapter 7 — Multivariate Analysis

Many variables measured on the same unit are related, and treating them one at a time throws that information away. This chapter builds the multivariate normal law, the distributions of quadratic forms and sample covariance matrices, correlation with other variables held fixed, joint tests of mean vectors, and the classical tools for reducing and classifying multivariate data.

A paediatric clinic records five numbers for every child it sees: height, weight, head circumference, haemoglobin and blood pressure. None of these numbers means much on its own. A tall child is expected to be heavy, so a weight that would be alarming for a small child is quite ordinary for a tall one. The informative question is whether the combination is unusual. Asking it one variable at a time fails in two ways. A child can pass every single check and still be very unusual jointly, and with five separate checks at the $5\%$ level the chance of a false alarm is far more than $5\%$. In the 1920s and 1930s the same difficulty arose wherever several measurements were taken together: in John Wishart's work on agricultural field trials, in Harold Hotelling's tests for several means at once, and in P. C. Mahalanobis's anthropometric surveys in India, which led him to a distance that respects the correlations between body measurements. Their answers rest on one idea: replace the variance of a single variable by the covariance matrix $\Sigma$ of the whole vector, and measure distances in the metric that $\Sigma$ defines.

The multivariate normal distribution is the natural home of this idea. Every linear combination of its components is normal, its contours are ellipsoids drawn by $\Sigma$, its conditional laws are again normal with linear regressions, and for it "uncorrelated" really does mean "independent". Section 7.1 develops this law, the chi-square laws of quadratic forms (Cochran's theorem), and the Wishart law of the sample covariance matrix. Section 7.2 uses them to measure association with other variables held fixed (partial correlation), to measure how well one variable is predicted by several (multiple correlation), and to test a whole mean vector at once with Hotelling's $T^2$. Section 7.3 turns to data reduction and classification: principal components, discriminant functions, clusters and canonical correlations.

Mean vector & Σ→Nₚ(μ, Σ)→Marginals & conditionals→Mahalanobis distance→Quadratic forms & Cochran→Wishart, X̄ and S→Partial & multiple correlation→Hotelling's T²→PCA & discriminants→Clusters & canonical correlation
🎯 By the end of this chapter you can…
  • compute means, variances and covariances of linear combinations of a random vector, and decide whether a given matrix can be a covariance or correlation matrix;
  • define the multivariate normal law through linear combinations, write down its density and characteristic function, and find the distributions of linear transformations, marginals and conditionals;
  • prove that jointly normal variables are independent exactly when they are uncorrelated, and give the counterexamples that show why "jointly" cannot be dropped;
  • use the Mahalanobis distance and its $\chi^2_p$ law to build probability ellipsoids and to judge how unusual an observation is;
  • decide when a quadratic form in normal variables is chi-square, when two forms are independent, and apply Cochran's theorem to split a sum of squares;
  • state and use the properties of the Wishart distribution and the joint law of the sample mean vector and the sample covariance matrix;
  • compute partial and multiple correlation coefficients, relate them to residuals, regression and the precision matrix, and test them;
  • carry out one- and two-sample Hotelling $T^2$ tests, build simultaneous confidence intervals, and recognise $T^2$ as a likelihood ratio test;
  • compute principal components, Fisher's linear discriminant, hierarchical and $k$-means clusters and canonical correlations, and interpret the results.
Teaser: Two Checks One at a Time, or One Check Together?

Points are drawn from a bivariate normal law with standard normal coordinates and correlation $\rho$ (seeded generator). The dashed box is what two separate checks accept: $|x_1|\le1.96$ and $|x_2|\le1.96$, each a $95\%$ range. The ellipse is the joint $95\%$ region $D^2\le\chi^2_{2,0.05}=5.991$, where $D^2=\frac{x_1^2-2\rho x_1x_2+x_2^2}{1-\rho^2}$ is the squared Mahalanobis distance. Red points pass both one-variable checks yet lie outside the ellipse; blue points fail a single check although they are jointly ordinary. Move $\rho$ and watch the two regions disagree.

0.80
points drawn—
inside the box (theory)—
inside the ellipse (theory 0.95)—
red: in box, outside ellipse—

How this is examined

CSIR NET draws on this chapter for both parts of the paper. Part B questions usually ask for one number or one law: the distribution of a linear combination or of a subvector of a normal vector, a conditional mean or variance, the value of a constant that makes a quadratic form chi-square, the degrees of freedom of a Wishart matrix or of an $F$ statistic built from Hotelling's $T^2$, a partial or multiple correlation computed from three simple correlations, or the proportion of variance explained by the first principal component. Part C questions, where several options may be correct, test the structure of the theory. Are uncorrelated normal variables independent if they are only marginally normal? Is a quadratic form chi-square when its matrix is not idempotent? Are two forms independent? Can a partial correlation have the opposite sign to the simple one? Can $T^2$ reject when every univariate $t$-test accepts? Which statements about eigenvalues, loadings, discriminant directions or canonical correlations are invariant under rescaling? The wrong options come from the ⚠️ boxes of the three sections. GATE ST examines the same material, often as numerical-answer questions.

Sections in this chapter

✅ Before you start

This chapter extends ideas you have met one variable or two variables at a time. You need joint, marginal and conditional distributions, covariance, the Jacobian formula and the bivariate normal law (§1.4); characteristic functions and their uniqueness (§2.2); the $\chi^2$, $t$ and $F$ laws and the independence of $\bar X$ and $S^2$ for normal samples (§4.1); likelihood ratio tests and Wilks' theorem (§5.1); and the normal linear model $Y=X\beta+\varepsilon$, the hat matrix $H$, the $F$-test of a linear hypothesis and the statement of Cochran's theorem (§6.1), together with $R^2$ (§6.3). From linear algebra you need the spectral theorem for symmetric matrices, positive (semi)definite matrices, rank, trace, determinants and orthogonal projections; the sister course CSIR NET Unit 1: Analysis and Linear Algebra covers them. Throughout, $A^{\mathsf T}$ is the transpose, $|A|$ the determinant, $\operatorname{tr}A$ the trace, and $A\gt0$ ($A\ge0$) means that a symmetric $A$ is positive definite (semidefinite).