6.1 Partial, Directional and Total Derivatives
Limits and continuity in $\mathbb{R}^n$, partial and directional derivatives, and the total derivative $Df(a)$ as the best linear approximation — with the chain rule, Schwarz's theorem on mixed partials and Taylor's formula in several variables.
In one variable a function is differentiable at $a$ when its graph, seen under a stronger and stronger microscope, becomes a straight line; the slope of that line is $f'(a)$. In several variables the same idea survives, but the "line" becomes a linear map: $f:\mathbb{R}^n\to\mathbb{R}^m$ is differentiable at $a$ when $f(a+h)-f(a)$ is, up to an error much smaller than $\|h\|$, a linear function of $h$. This linear map — the total derivative $Df(a)$, recorded by the Jacobian matrix — is what the chain rule multiplies, what the inverse and implicit function theorems of §6.2 invert, and what optimisation, mechanics and machine learning compute every day.
The new feature of several variables is that a point can be approached from infinitely many directions and along curves of every shape. As a result, the familiar one-variable implications break: a function can have both partial derivatives at a point and still be discontinuous there; it can have a directional derivative in every direction and still fail to be differentiable; its mixed partial derivatives $f_{xy}$ and $f_{yx}$ can differ. CSIR NET examines exactly these gaps — "which of the following is true at the origin?" — and this section collects the standard examples, together with the positive theorems ($C^1\Rightarrow$ differentiable, Schwarz's theorem) that close the gaps.
Limits and Continuity in $\mathbb{R}^n$
Throughout, $\mathbb{R}^n$ carries the Euclidean norm $\|x\|=\left(x_1^2+\cdots+x_n^2\right)^{1/2}$, and $U\subseteq\mathbb{R}^n$ denotes an open set. Since $\max_i|x_i|\le\|x\|\le\sqrt n\,\max_i|x_i|$, "$x\to a$" means the same thing whether we measure with $\|\cdot\|$ or coordinate by coordinate (Chapter 7 shows that all norms on $\mathbb{R}^n$ give the same convergence).
The open ball of radius $r\gt0$ about $a$ is $B(a,r)=\{x\in\mathbb{R}^n:\|x-a\|\lt r\}$; a set $U$ is open if every point of $U$ is the centre of a ball contained in $U$. Let $E\subseteq\mathbb{R}^n$, let $f:E\to\mathbb{R}^m$, and let $a$ be a limit point of $E$.
- $\displaystyle\lim_{x\to a}f(x)=L$ means: for every $\varepsilon\gt0$ there is $\delta\gt0$ such that $\|f(x)-L\|\lt\varepsilon$ for all $x\in E$ with $0\lt\|x-a\|\lt\delta$.
- $f$ is continuous at $a\in E$ if for every $\varepsilon\gt0$ there is $\delta\gt0$ with $\|f(x)-f(a)\|\lt\varepsilon$ whenever $x\in E$, $\|x-a\|\lt\delta$.
For $n=2$ we write $\lim_{(x,y)\to(a,b)}f(x,y)$ and call it the double limit, to distinguish it from the iterated limits $\lim_{x\to a}\lim_{y\to b}f(x,y)$ and $\lim_{y\to b}\lim_{x\to a}f(x,y)$.
Let $f=(f_1,\dots,f_m):E\to\mathbb{R}^m$ and let $a$ be a limit point of $E$.
- $\lim_{x\to a}f(x)=L$ if and only if $\lim_{x\to a}f_i(x)=L_i$ for every $i=1,\dots,m$.
- $\lim_{x\to a}f(x)=L$ if and only if $f(x_k)\to L$ for every sequence $x_k\in E\setminus\{a\}$ with $x_k\to a$.
- Limits of sums, scalar multiples, products and quotients (nonzero denominator limit) of real-valued functions are the sums, multiples, products and quotients of the limits. Consequently polynomials and rational functions in $x_1,\dots,x_n$ are continuous wherever they are defined, and compositions of continuous maps are continuous.
Proof. (1) follows from $|f_i(x)-L_i|\le\|f(x)-L\|\le\sqrt m\,\max_i|f_i(x)-L_i|$. (2) If the limit is $L$ and $x_k\to a$ with $x_k\ne a$, given $\varepsilon$ take $\delta$ from the definition and $K$ with $\|x_k-a\|\lt\delta$ for $k\ge K$; then $\|f(x_k)-L\|\lt\varepsilon$. Conversely, if the limit is not $L$, there is $\varepsilon\gt0$ such that for every $k$ some $x_k\in E$ has $0\lt\|x_k-a\|\lt1/k$ but $\|f(x_k)-L\|\ge\varepsilon$: this sequence violates the sequential condition. (3) is proved exactly as in one variable (§3.1), using (2) and the algebra of limits of sequences; each coordinate function $x\mapsto x_j$ is continuous because $|x_j-a_j|\le\|x-a\|$. ∎
Let $f$ be real-valued near $a\in\mathbb{R}^n$ (except possibly at $a$).
- Paths. If $\gamma_1,\gamma_2:(0,1]\to\mathbb{R}^n\setminus\{a\}$ satisfy $\gamma_j(t)\to a$ as $t\to0^+$, and $f(\gamma_1(t))\to L_1$, $f(\gamma_2(t))\to L_2$ with $L_1\ne L_2$ (or if one of these limits fails to exist), then $\lim_{x\to a}f(x)$ does not exist.
- Squeeze. If $|f(x)-L|\le g(\|x-a\|)$ for $0\lt\|x-a\|\lt r_0$, where $g(r)\to0$ as $r\to0^+$, then $\lim_{x\to a}f(x)=L$. In the plane this is often used in polar form: $|f(a+r\cos\theta,\,b+r\sin\theta)-L|\le g(r)$ with $g$ independent of $\theta$.
Proof. (1) If $\lim_{x\to a}f(x)=L$ existed, then by Theorem 6.1(2) (applied to the sequences $\gamma_j(1/k)$, and more generally to $\gamma_j(t_k)$ for any $t_k\to0^+$) we would have $f(\gamma_j(t))\to L$ for both $j$, forcing $L_1=L=L_2$. (2) Given $\varepsilon$, choose $\delta\le r_0$ with $g(r)\lt\varepsilon$ for $0\lt r\lt\delta$. ∎
Paths can only disprove a limit: agreement along every line, or even along every line and every parabola, proves nothing (Example 1(b)). A limit is proved by an estimate valid uniformly in all directions, as in part (2). The polar bound must not depend on $\theta$; "for each fixed $\theta$ the expression tends to $0$" is only the statement that the limit along each ray is $0$.
For functions of two variables one can also take the limits one variable at a time. The two procedures are genuinely different, and neither iterated limit determines the double limit.
Proof. Given $\varepsilon\gt0$, choose $\delta$ with $|f(x,y)-L|\lt\varepsilon$ whenever $0\lt\|(x,y)-(a,b)\|\lt\delta$. Fix $y$ with $0\lt|y-b|\lt\delta/2$. For $0\lt|x-a|\lt\delta/2$ the point $(x,y)$ is within $\delta$ of $(a,b)$ and differs from it, so $|f(x,y)-L|\lt\varepsilon$; letting $x\to a$ gives $|\varphi(y)-L|\le\varepsilon$. Thus $\varphi(y)\to L$ as $y\to b$. ∎
Partial and Directional Derivatives
The simplest way to differentiate a function of several variables is to freeze all variables but one. More generally we can move from $a$ along any straight line and differentiate the resulting function of one variable.
Let $f:U\to\mathbb{R}$, $a\in U$, and let $e_1,\dots,e_n$ be the standard basis of $\mathbb{R}^n$. For $v\in\mathbb{R}^n$, the derivative of $f$ at $a$ along $v$ is
$$D_vf(a)=\lim_{t\to0}\frac{f(a+tv)-f(a)}{t},$$when this limit exists. For a unit vector $u$, $D_uf(a)$ is the directional derivative in the direction $u$ (CSIR questions use unit vectors unless they say otherwise). The partial derivatives are $\dfrac{\partial f}{\partial x_j}(a)=D_jf(a)=D_{e_j}f(a)$; in the plane we write $f_x,f_y$. If all $n$ partial derivatives exist, the gradient is $\nabla f(a)=\left(D_1f(a),\dots,D_nf(a)\right)$. For $f=(f_1,\dots,f_m):U\to\mathbb{R}^m$ everything is taken componentwise.
Two remarks. First, $D_vf(a)$ is the ordinary derivative at $t=0$ of the one-variable function $t\mapsto f(a+tv)$, so every one-variable rule applies; and $D_{cv}f(a)=c\,D_vf(a)$ for every scalar $c$. Second, and this is the trap, partial and directional derivatives only look at $f$ on lines through $a$. They cannot see what happens along curves, so they do not control continuity: for $f(x,y)=\dfrac{xy}{x^2+y^2}$, $f(0,0)=0$, we have $f(x,0)=0=f(0,y)$, hence $f_x(0,0)=f_y(0,0)=0$, although $f$ is not even continuous at the origin (Example 1(a)).
The Total Derivative as a Linear Map
The right generalisation of differentiability asks for a single linear map that approximates $f(a+h)-f(a)$ for all small $h$ simultaneously, with an error that is small compared with $\|h\|$.
Let $f:U\to\mathbb{R}^m$ and $a\in U$. $f$ is differentiable at $a$ if there is a linear map $T:\mathbb{R}^n\to\mathbb{R}^m$ such that
$$\lim_{h\to0}\frac{\|f(a+h)-f(a)-Th\|}{\|h\|}=0.$$Equivalently, $f(a+h)=f(a)+Th+\|h\|\,\varepsilon(h)$ with $\varepsilon(h)\to0=\varepsilon(0)$. The map $T$ is the total derivative (or Fréchet derivative) $Df(a)$. Its matrix with respect to the standard bases is the $m\times n$ Jacobian matrix $J_f(a)$; when $m=n$, $\det J_f(a)$ is the Jacobian determinant. $f$ is differentiable on $U$ if it is differentiable at every point of $U$.
Let $f=(f_1,\dots,f_m):U\to\mathbb{R}^m$ be differentiable at $a$, with $T=Df(a)$. Then:
- $T$ is unique, and $f$ is continuous at $a$.
- For every $v\in\mathbb{R}^n$ the derivative along $v$ exists and $D_vf(a)=Df(a)v$; in particular $v\mapsto D_vf(a)$ is linear.
- All partial derivatives exist and $J_f(a)=\left[\dfrac{\partial f_i}{\partial x_j}(a)\right]_{i,j}$ (row $i$ is $\nabla f_i(a)$).
- For real-valued $f$: $D_uf(a)=\nabla f(a)\cdot u$ for every unit vector $u$. Hence $|D_uf(a)|\le\|\nabla f(a)\|$; if $\nabla f(a)\ne0$, the largest directional derivative is $\|\nabla f(a)\|$, attained only for $u=\nabla f(a)/\|\nabla f(a)\|$, and $D_uf(a)=0$ exactly for $u\perp\nabla f(a)$.
Proof. (2) Fix $v\ne0$ and put $h=tv$ with $t\to0$, $t\ne0$. Then
$$\left\|\frac{f(a+tv)-f(a)}{t}-Tv\right\|=\frac{\|f(a+tv)-f(a)-T(tv)\|}{\|tv\|}\,\|v\|\to0,$$so $D_vf(a)$ exists and equals $Tv$ (for $v=0$ both sides are $0$). (1) If $T'$ also satisfies the definition, then $Tv=D_vf(a)=T'v$ for every $v$ by (2), so $T=T'$. Continuity: $\|f(a+h)-f(a)\|\le\|Th\|+\|h\|\,\|\varepsilon(h)\|$, which is at most $\left(\|T\|+\|\varepsilon(h)\|\right)\|h\|\to0$, where $\|T\|=\max_{\|v\|=1}\|Tv\|$ is finite because a linear map on $\mathbb{R}^n$ is continuous and the unit sphere is compact. (3) The $(i,j)$ entry of the matrix of $T$ is the $i$-th component of $Te_j=D_{e_j}f(a)=\left(D_jf_1(a),\dots,D_jf_m(a)\right)$. (4) For real $f$, $J_f(a)$ is the row vector $\nabla f(a)$, so $D_uf(a)=\nabla f(a)\cdot u$. By the Cauchy–Schwarz inequality $|\nabla f(a)\cdot u|\le\|\nabla f(a)\|\,\|u\|=\|\nabla f(a)\|$, with equality (and positive sign) iff $u$ is a positive multiple of $\nabla f(a)$. ∎
Geometrically, for $f:U\to\mathbb{R}$ with $U\subseteq\mathbb{R}^2$ the graph of $h\mapsto f(a)+\nabla f(a)\cdot h$ is the tangent plane $z=f(a)+f_x(a)(x-a_1)+f_y(a)(y-a_2)$, and the gradient is perpendicular to the level curve of $f$ through $a$ (the derivative of $f$ along the curve is $0$).
- Is $f$ continuous at $a$? If not, stop: $f$ is not differentiable (Theorem 6.4(1)).
- Compute the partial derivatives at $a$ from the definition (do not differentiate a formula that is valid only away from $a$). If one fails to exist, $f$ is not differentiable. Otherwise the only candidate is $Th=\nabla f(a)\cdot h$ (componentwise for vector $f$).
- Quick necessary test: if some directional derivative is missing, or $D_uf(a)\ne\nabla f(a)\cdot u$ for some $u$, then $f$ is not differentiable.
- Decisive test: examine $\rho(h)=\dfrac{f(a+h)-f(a)-\nabla f(a)\cdot h}{\|h\|}$. Prove $\rho\to0$ by a bound uniform in direction (polar form), or disprove it with a path along which $\rho\not\to0$.
- Shortcut: if the partials exist near $a$ and are continuous at $a$, $f$ is differentiable (Theorem 6.5). The converse fails (Example 4(d)).
Proof. By Theorem 6.1(1) applied to the quotient in Definition 6.3, it suffices to treat a real-valued $f$. Let $h=(h_1,\dots,h_n)$ with $\|h\|\lt r$, and walk from $a$ to $a+h$ along the edges of a box: put $p_0=a$ and $p_j=a+h_1e_1+\cdots+h_je_j$, so $p_n=a+h$, and every $p_j$ lies in $B(a,r)$ (indeed $\|p_j-a\|\le\|h\|$). Then
$$f(a+h)-f(a)=\sum_{j=1}^n\left[f(p_{j-1}+h_je_j)-f(p_{j-1})\right].$$On the $j$-th edge only the $j$-th coordinate changes, so $s\mapsto f(p_{j-1}+se_j)$ is a function of one variable with derivative $D_jf(p_{j-1}+se_j)$; by the one-variable mean value theorem (§3.3) the $j$-th bracket equals $h_j\,D_jf(q_j)$ for some point $q_j$ on that edge, so $\|q_j-a\|\le\|h\|$. Hence
$$\left|f(a+h)-f(a)-\sum_jD_jf(a)h_j\right|=\left|\sum_j\left(D_jf(q_j)-D_jf(a)\right)h_j\right|\le\|h\|\sum_j\left|D_jf(q_j)-D_jf(a)\right|.$$As $h\to0$ we have $q_j\to a$, and continuity of $D_jf$ at $a$ makes the last sum tend to $0$. So $f$ is differentiable at $a$ with $Df(a)h=\nabla f(a)\cdot h$. ∎
The proof uses continuity at $a$ of only $n-1$ of the partials: on the first edge one can use the definition of $D_1f(a)$ directly, because $p_0=a$. So if $f_y$ is continuous at $a$ and $f_x(a)$ merely exists, $f$ is differentiable at $a$ — a refinement that occasionally decides a Part C option.
The Chain Rule, Euler's Theorem and the Mean Value Theorem
The great advantage of the total derivative over partial derivatives is that it composes: the derivative of a composite is the composite of the derivatives, and in coordinates Jacobian matrices multiply.
Proof. Write $A=Dg(a)$, $B=Df(b)$ and
$$g(a+h)=b+Ah+\|h\|\,\varepsilon_1(h),\qquad f(b+k)=f(b)+Bk+\|k\|\,\varepsilon_2(k),$$with $\varepsilon_1(h)\to0$ as $h\to0$, $\varepsilon_2(k)\to0$ as $k\to0$, and $\varepsilon_1(0)=0$, $\varepsilon_2(0)=0$. Put $k(h)=g(a+h)-b=Ah+\|h\|\varepsilon_1(h)$. Then $\|k(h)\|\le\left(\|A\|+\|\varepsilon_1(h)\|\right)\|h\|$, so $k(h)\to0$, and
$$f(g(a+h))-f(b)-BAh=\|h\|\,B\varepsilon_1(h)+\|k(h)\|\,\varepsilon_2(k(h)).$$Divide by $\|h\|$: the first term is at most $\|B\|\,\|\varepsilon_1(h)\|\to0$, and the second at most $\left(\|A\|+\|\varepsilon_1(h)\|\right)\|\varepsilon_2(k(h))\|\to0$, since $k(h)\to0$ and $\varepsilon_2$ is continuous at $0$ with value $0$. (Setting $\varepsilon_2(0)=0$ is what handles the points where $k(h)=0$ — the step at which the naive "multiply and divide by $\Delta g$" proof breaks down, exactly as in one variable.) Hence $D(f\circ g)(a)=BA$; the matrix of a composite is the product of the matrices. ∎
The most used special case: if $x(t),y(t)$ are differentiable and $f$ is differentiable, then $\dfrac{d}{dt}f(x(t),y(t))=f_x\,x'(t)+f_y\,y'(t)=\nabla f\cdot\gamma'(t)$ with $\gamma=(x,y)$. Note that $f$ must be differentiable, not merely have partial derivatives: for $f_2=\frac{x^2y}{x^2+y^2}$ and $\gamma(t)=(t,t)$, $f_2(\gamma(t))=t/2$ has derivative $\tfrac12$, whereas $\nabla f_2(0)\cdot\gamma'(0)=0$.
Proof. Fix $x\in C$ and let $\varphi(t)=f(tx)$, $t\gt0$. By the chain rule $\varphi'(t)=\nabla f(tx)\cdot x$. If $f$ is homogeneous, $\varphi(t)=t^kf(x)$, so $\varphi'(1)=kf(x)$, which is the identity. Conversely, if the identity holds everywhere on $C$, then $t\varphi'(t)=tx\cdot\nabla f(tx)=k\varphi(t)$, so $\dfrac{d}{dt}\left(t^{-k}\varphi(t)\right)=t^{-k-1}\left(t\varphi'(t)-k\varphi(t)\right)=0$ on $(0,\infty)$; hence $t^{-k}\varphi(t)=\varphi(1)$, i.e. $f(tx)=t^kf(x)$. For the second identity, each $\partial f/\partial x_j$ is homogeneous of degree $k-1$ (differentiate $f(tx)=t^kf(x)$ with respect to $x_j$), so by the first part $\sum_ix_i\,\partial_i\partial_jf=(k-1)\partial_jf$; multiply by $x_j$ and sum: $\sum_{i,j}x_ix_j\,\partial_i\partial_jf=(k-1)\,x\cdot\nabla f=k(k-1)f$. ∎
A typical use: for $u=\log\dfrac{x^4+y^4}{x+y}$ on $x+y\gt0$, the function $e^u$ is homogeneous of degree $3$, and $x(e^u)_x+y(e^u)_y=e^u(xu_x+yu_y)=3e^u$ gives $xu_x+yu_y=3$ without computing a single derivative.
Let $U\subseteq\mathbb{R}^n$ be open and let the segment $[a,b]=\{a+t(b-a):0\le t\le1\}$ lie in $U$.
- If $f:U\to\mathbb{R}$ is differentiable on $U$, then $f(b)-f(a)=\nabla f(c)\cdot(b-a)$ for some $c$ on the open segment between $a$ and $b$.
- If $f:U\to\mathbb{R}^m$ is differentiable on $U$ and $\|Df(x)\|\le M$ for all $x\in[a,b]$, then $\|f(b)-f(a)\|\le M\|b-a\|$. For $m\ge2$ there need not be any $c$ with $f(b)-f(a)=Df(c)(b-a)$.
- If $U$ is open and connected and $Df(x)=0$ for every $x\in U$, then $f$ is constant on $U$.
Proof. (1) $\varphi(t)=f(a+t(b-a))$ is differentiable on a neighbourhood of $[0,1]$ with $\varphi'(t)=\nabla f(a+t(b-a))\cdot(b-a)$ (chain rule). The one-variable mean value theorem gives $\varphi(1)-\varphi(0)=\varphi'(\tau)$ for some $\tau\in(0,1)$; take $c=a+\tau(b-a)$.
(2) If $f(b)=f(a)$ there is nothing to prove. Otherwise let $w=\dfrac{f(b)-f(a)}{\|f(b)-f(a)\|}$, a unit vector, and apply (1) to the real function $g(x)=w\cdot f(x)$, whose gradient satisfies $\nabla g(x)\cdot v=w\cdot Df(x)v$. We get $c$ with $\|f(b)-f(a)\|=g(b)-g(a)$ $=w\cdot Df(c)(b-a)$, and by Cauchy–Schwarz this is at most $\|Df(c)\|\,\|b-a\|\le M\|b-a\|$. For the failure of equality take $f(t)=(\cos t,\sin t)$ on $[0,2\pi]$: $f(2\pi)-f(0)=0$, while $f'(c)=(-\sin c,\cos c)$ is never $0$.
(3) Fix $a\in U$ and let $S=\{x\in U:f(x)=f(a)\}$. $S$ is nonempty, and closed in $U$ because $f$ is continuous. It is open: if $x\in S$ and $B(x,r)\subseteq U$, every $y\in B(x,r)$ is joined to $x$ by a segment inside the ball, and (2) with $M=0$ gives $f(y)=f(x)=f(a)$. A nonempty subset of a connected set that is both open and closed in it is the whole set (§7.3), so $S=U$. ∎
Connectedness cannot be dropped in (3): on $U=(0,1)\cup(2,3)$ the function equal to $0$ on the first interval and $1$ on the second has derivative $0$. And a single vanishing partial is weaker than one might think — $f_x\equiv0$ on a connected $U$ means that $f$ is constant on each horizontal segment in $U$, not that $f$ is independent of $x$ (Exercise III.7).
Higher Derivatives: Schwarz's Theorem and Taylor's Formula
If $D_jf$ exists near $a$ and has a partial derivative with respect to $x_i$ at $a$, we write $D_iD_jf(a)=\dfrac{\partial^2f}{\partial x_i\,\partial x_j}(a)$. In the plane we use the subscript convention $f_{xy}=(f_x)_y=\dfrac{\partial^2f}{\partial y\,\partial x}$ — differentiate first in $x$, then in $y$. $f$ is of class $C^k$ on $U$ if all partial derivatives of order $\le k$ exist and are continuous on $U$; $C^\infty$ means $C^k$ for every $k$. For $f\in C^2$, the Hessian matrix at $a$ is $H_f(a)=\left[D_iD_jf(a)\right]_{i,j=1}^n$.
Proof. For small $h,k\ne0$ consider the second difference
$$\Delta(h,k)=f(a+h,b+k)-f(a+h,b)-f(a,b+k)+f(a,b).$$Step 1. Let $\varphi(s)=f(s,b+k)-f(s,b)$. Then $\Delta=\varphi(a+h)-\varphi(a)$, and $\varphi'(s)=f_x(s,b+k)-f_x(s,b)$ exists near $a$. By the mean value theorem, for some $\theta\in(0,1)$,
$$\Delta=h\,\varphi'(a+\theta h)=h\left[f_x(a+\theta h,b+k)-f_x(a+\theta h,b)\right].$$The function $t\mapsto f_x(a+\theta h,t)$ has derivative $f_{xy}(a+\theta h,t)$, so a second application gives $\Delta=hk\,f_{xy}(a+\theta h,b+\theta'k)$ with $\theta'\in(0,1)$. Since $f_{xy}$ is continuous at $(a,b)$: for every $\varepsilon\gt0$ there is $\delta\gt0$ with
$$\left|\frac{\Delta(h,k)}{hk}-f_{xy}(a,b)\right|\lt\varepsilon\qquad\text{whenever }0\lt|h|,|k|\lt\delta.$$Step 2. Fix $h$ with $0\lt|h|\lt\delta$ and let $k\to0$. Because $f_y$ exists near $(a,b)$,
$$\frac{\Delta(h,k)}{k}=\frac{f(a+h,b+k)-f(a+h,b)}{k}-\frac{f(a,b+k)-f(a,b)}{k}\to f_y(a+h,b)-f_y(a,b).$$Passing to the limit in the inequality of Step 1 gives $\left|\dfrac{f_y(a+h,b)-f_y(a,b)}{h}-f_{xy}(a,b)\right|\le\varepsilon$ for $0\lt|h|\lt\delta$. This says precisely that $f_{yx}(a,b)=\lim_{h\to0}\dfrac{f_y(a+h,b)-f_y(a,b)}{h}$ exists and equals $f_{xy}(a,b)$. The statements for $C^k$ and $n$ variables follow by applying the two-variable result to pairs of variables, the others being frozen. ∎
In Example 6, $f_{xy}$ exists near $0$ but is discontinuous at $0$, so Theorem 6.9 does not apply — and the conclusion fails. (Young's theorem gives a different sufficient condition: if $f_x$ and $f_y$ are differentiable at $(a,b)$, then $f_{xy}(a,b)=f_{yx}(a,b)$.)
Proof. Let $\varphi(t)=f(a+th)$ on an open interval containing $[0,1]$. By the chain rule $\varphi'(t)=\sum_ih_iD_if(a+th)=(h\cdot\nabla)f(a+th)$; applying the same rule to each $D_if$ (a $C^k$ function) and inducting, $\varphi^{(j)}(t)=(h\cdot\nabla)^jf(a+th)$ for $j\le k+1$, and $\varphi\in C^{k+1}$. The one-variable Taylor theorem with Lagrange remainder (§3.3) gives $\varphi(1)=\sum_{j=0}^k\frac{\varphi^{(j)}(0)}{j!}+\frac{\varphi^{(k+1)}(\tau)}{(k+1)!}$ for some $\tau\in(0,1)$; put $c=a+\tau h$. For the $o(\|h\|^2)$ form with $k=1$: $\left|h^{\mathsf T}(H_f(c)-H_f(a))h\right|\le\|H_f(c)-H_f(a)\|\,\|h\|^2$, and $\|H_f(c)-H_f(a)\|\to0$ as $h\to0$ because the second partials are continuous and $\|c-a\|\le\|h\|$. ∎
- Lines do not decide limits. $\frac{x^2y}{x^4+y^2}\to0$ along every line through $0$ but equals $\tfrac12$ on $y=x^2$; $\frac{xy^2}{x^2+y^4}$ likewise on $x=y^2$.
- Equal iterated limits prove nothing. $\frac{xy}{x^2+y^2}$ has both iterated limits $0$ and no double limit. Conversely $x\sin\frac1y+y\sin\frac1x$ has double limit $0$ and no iterated limits.
- Partials exist $\not\Rightarrow$ continuous. $\frac{xy}{x^2+y^2}$ at $0$: $f_x=f_y=0$, yet $f$ is discontinuous.
- All directional derivatives exist $\not\Rightarrow$ continuous. $\frac{x^2y}{x^4+y^2}$ at $0$: $D_uf(0)=u_1^2/u_2$ for $u_2\ne0$, $0$ for $u_2=0$; discontinuous.
- Continuous with all $D_uf$ $\not\Rightarrow$ differentiable. $\frac{x^2y}{x^2+y^2}$: $D_uf(0)=u_1^2u_2$, not linear in $u$.
- Even linear $u\mapsto D_uf(a)$ is not enough. $\frac{x^3y}{x^4+y^2}$: continuous, all $D_uf(0)=0$, not differentiable (remainder ratio $\tfrac12$ on $y=x^2$).
- Differentiable $\not\Rightarrow C^1$. $(x^2+y^2)\sin\frac{1}{\sqrt{x^2+y^2}}$ is differentiable at $0$ with discontinuous partials.
- $\sqrt{|xy|}$ is continuous, has $f_x(0)=f_y(0)=0$, and is not differentiable at $0$ (on $y=x$ the ratio is $1/\sqrt2$).
- Mixed partials can differ. $\frac{xy(x^2-y^2)}{x^2+y^2}$ is $C^1$ with $f_{xy}(0)=-1\ne1=f_{yx}(0)$. Existence of both mixed partials is not enough; continuity of one of them at the point is.
- The chain rule needs differentiability. For $f=\frac{x^2y}{x^2+y^2}$ and $\gamma(t)=(t,t)$: $(f\circ\gamma)'(0)=\tfrac12$ but $\nabla f(0)\cdot\gamma'(0)=0$.
- No mean value equality for vector functions. $(\cos t,\sin t)$ on $[0,2\pi]$; only the inequality $\|f(b)-f(a)\|\le\sup\|Df\|\,\|b-a\|$ survives.
- Normalise the direction. "The directional derivative along $(2,-1,2)$" means along $u=\tfrac13(2,-1,2)$; $\nabla f\cdot v$ with $\|v\|\ne1$ is $\|v\|$ times too big.
Exercises
- 1 $\dfrac{x^2-y^2}{x^2+y^2}$.
- 2 $\dfrac{xy^2}{x^2+y^4}$.
- 3 $\dfrac{\sin(x^2+y^2)}{x^2+y^2}$.
- 4 $\dfrac{xy}{\sqrt{x^2+y^2}}$.
- 5 $\dfrac{x^3+y^3}{x^2+y^2}$.
- 6 Let $f(x,y)=\dfrac{x^2y^2}{x^2y^2+(x-y)^2}$ for $(x,y)\ne(0,0)$. Show that both iterated limits at $(0,0)$ exist and are equal, but the double limit does not exist.
- 7 Let $f(x,y)=x\sin\dfrac1y$ for $y\ne0$ and $f(x,0)=0$. Show that $\lim_{(x,y)\to(0,0)}f=0$ and $\lim_{y\to0}\lim_{x\to0}f=0$, but $\lim_{x\to0}\lim_{y\to0}f$ does not exist.
- 8 For $\alpha\gt0$ let $f(x,y)=\dfrac{|x|^\alpha\,|y|}{x^2+y^2}$ for $(x,y)\ne(0,0)$ and $f(0,0)=0$. Find all $\alpha$ for which $f$ is continuous at the origin.
- 1 For $f(x,y,z)=xe^{yz}+y^2$ and $P=(1,0,2)$, find $\nabla f(P)$, the directional derivative at $P$ in the direction of $(1,2,-2)$, and the maximal directional derivative at $P$.
- 2 Find the Jacobian matrix of $F(x,y,z)=\left(xy^2,\ x+\sin(yz)\right)$ at $(1,1,0)$, and use it to approximate $F(1.01,\,0.98,\,0.02)$.
- 3 Show that $f(x,y)=\sqrt{|xy|}$ is continuous at $(0,0)$ with $f_x(0,0)=f_y(0,0)=0$, but not differentiable there. In which directions $u$ does $D_uf(0,0)$ exist?
- 4 For which $\alpha\gt0$ is $f(x,y)=|xy|^\alpha$ differentiable at $(0,0)$?
- 5 Let $f(x,y)=\sqrt[3]{x^3+y^3}$. Compute $D_uf(0,0)$ for every unit vector $u$ and decide whether $f$ is differentiable at $(0,0)$.
- 6 Prove: if $f:\mathbb{R}^n\to\mathbb{R}$ satisfies $|f(x)|\le\|x\|^2$ for all $x$, then $f$ is differentiable at $0$ and $Df(0)=0$. Must $f$ be continuous at any point other than $0$?
- 7 Let $f$ be differentiable at $a$ with $\nabla f(a)\ne0$, and let $f:\mathbb{R}^2\to\mathbb{R}$. Show that there are exactly two unit vectors $u$ with $D_uf(a)=0$, and that if $D_uf(a)=3$ and $D_vf(a)=4$ for orthonormal $u,v$, then $\|\nabla f(a)\|=5$.
- 8 Show that every linear map $L$ is differentiable with $DL(a)=L$ for all $a$, and that $B(x,y)=x\cdot y$ on $\mathbb{R}^n\times\mathbb{R}^n$ is differentiable with $DB(a,b)(h,k)=a\cdot k+h\cdot b$.
- 1 Let $f$ be differentiable and $u=f(x-y,\ y-z,\ z-x)$. Show that $u_x+u_y+u_z=0$.
- 2 Let $w=x^2+yz$ with $x=t^2$, $y=\cos t$, $z=\sin t$. Find $\dfrac{dw}{dt}$ at $t=\dfrac\pi2$ by the chain rule, and check by substituting first.
- 3 Let $u=\log\dfrac{x^4+y^4}{x+y}$ on $\{x+y\gt0\}$. Find $xu_x+yu_y$.
- 4 For $f(x,y)=x^2y+3y$, find the point $c$ on the segment from $(0,0)$ to $(1,1)$ given by the mean value theorem.
- 5 Let $f:\mathbb{R}^2\to\mathbb{R}$ be differentiable with $\|\nabla f(x)\|\le2$ everywhere and $f(0,0)=1$. Give the best possible upper bound for $f(3,4)$, and a function attaining it.
- 6 Let $g:\mathbb{R}\to\mathbb{R}$ be differentiable and $f(x,y)=g(x^2+y^2)$. Show that $yf_x-xf_y=0$, and interpret this geometrically. Conversely, show that a differentiable $f$ on $\mathbb{R}^2\setminus\{0\}$ with $yf_x-xf_y\equiv0$ is constant on every circle centred at $0$.
- 7 Find an open connected set $U\subseteq\mathbb{R}^2$ and a $C^1$ function $f$ on $U$ with $f_x\equiv0$ on $U$, but $f(x_1,y)\ne f(x_2,y)$ for some points $(x_1,y),(x_2,y)\in U$.
- 1 Verify $f_{xy}=f_{yx}$ for $f(x,y)=x^2y^3+e^{xy}$.
- 2 Let $f(x,y)=\dfrac{xy(x^2-2y^2)}{x^2+y^2}$, $f(0,0)=0$. Compute $f_{xy}(0,0)$ and $f_{yx}(0,0)$.
- 3 Find the second-order Taylor polynomial of $f(x,y)=\log(1+x+2y)$ about $(0,0)$, and use it to approximate $\log1.07$ with $(x,y)=(0.03,0.02)$.
- 4 Find the second-order Taylor polynomial of $f(x,y)=\sin x\sin y$ about $\left(\frac\pi4,\frac\pi4\right)$.
- 5 Show that $\log(x^2+y^2)$ is harmonic ($u_{xx}+u_{yy}=0$) on $\mathbb{R}^2\setminus\{0\}$, and find all real $n$ for which $\|x\|^n$ is harmonic on $\mathbb{R}^3\setminus\{0\}$.
- 6 Prove that if $f\in C^2(\mathbb{R}^2)$ and $f_{xy}\equiv0$, then $f(x,y)=g(x)+h(y)$ for some $C^2$ functions $g,h$.
- 7 Let $f\in C^2(\mathbb{R}^2\setminus\{0\})$ be homogeneous of degree $k$. Show that $x^2f_{xx}+2xyf_{xy}+y^2f_{yy}=k(k-1)f$, and check it for $f=x^3-5xy^2$.
E1. CSIR Part B Let $f(x,y)=\dfrac{xy^2}{x^2+y^4}$ for $(x,y)\ne(0,0)$ and $f(0,0)=0$. Which of the following is true? (A) $f$ is continuous at $(0,0)$; (B) $D_uf(0,0)$ exists for every unit vector $u$; (C) $f$ is differentiable at $(0,0)$; (D) $f$ is unbounded on every neighbourhood of $(0,0)$.
E2. CSIR Part B GATE MA For $f(x,y,z)=x^2y+yz^3$, the maximum of the directional derivative $D_uf(1,1,1)$ over all unit vectors $u$ is (A) $7$; (B) $\sqrt{17}$; (C) $17$; (D) $\sqrt7$.
E3. CSIR Part B Let $f\in C^2(\mathbb{R}^2\setminus\{0\})$ be homogeneous of degree $3$ with $f(1,2)=5$. Then $\left(x^2f_{xx}+2xyf_{xy}+y^2f_{yy}\right)(1,2)$ equals (A) $15$; (B) $30$; (C) $45$; (D) it cannot be determined from the data.
E4. CSIR Part C Let $f(x,y)=\dfrac{x^2y^2}{x^2+y^2}$ for $(x,y)\ne(0,0)$ and $f(0,0)=0$. Which of the following are true? (One or more options may be correct.) (A) $f$ is differentiable at $(0,0)$. (B) $f_x$ is continuous at $(0,0)$. (C) $f_{xy}(0,0)=f_{yx}(0,0)$. (D) $f$ is of class $C^2$ on $\mathbb{R}^2$.
E5. CSIR Part C Let $f:\mathbb{R}^2\to\mathbb{R}$. Which of the following statements are true? (A) If $f_x$ and $f_y$ exist at every point of $\mathbb{R}^2$, then $f$ is continuous. (B) If $f$ is differentiable at $a$, then $u\mapsto D_uf(a)$ is a linear function of $u$. (C) If $D_uf(a)$ exists for every $u$ and $u\mapsto D_uf(a)$ is linear, then $f$ is differentiable at $a$. (D) If $f_x$ and $f_y$ exist everywhere and are bounded on $\mathbb{R}^2$, then $f$ is continuous on $\mathbb{R}^2$.
E6. CSIR Part C GATE MA Let $f\in C^2(\mathbb{R}^2)$, written $f(u,v)$, and $g(x,y)=f(x^2-y^2,\ 2xy)$. Which of the following are true? (A) $g_x=2xf_u+2yf_v$. (B) $g_{xx}+g_{yy}=4(x^2+y^2)(f_{uu}+f_{vv})$. (C) If $f(u,v)=u$, then $g$ is harmonic. (D) The Jacobian determinant of $(x,y)\mapsto(x^2-y^2,2xy)$ is $4xy$.
Suggested plan — 3 lectures × 60 min.
- Limits in $\mathbb{R}^n$. Definition 6.1 and Theorems 6.1–6.2. Project Interactive 1 with $\frac{x^2y}{x^4+y^2}$ on lines first; let the class vote "limit $0$?" before switching to a parabola. Example 1 on the board, then Theorem 6.3 and Example 2. Close with partial derivatives of $\frac{xy}{x^2+y^2}$ at $0$: "derivatives without continuity".
- The total derivative. Definition 6.3, Theorem 6.4 with full proof, Example 3. Interactive 2: run the four functions in order; ask students to predict the solid curve before pressing Play. Example 4 and the strategy box; Theorem 6.5 with the box-path proof drawn on the board.
- Calculus with derivatives. Chain rule (Theorem 6.6) — stress the $\varepsilon_2(0)=0$ trick; Example 5; Euler's theorem; the mean value theorem and why equality fails for curves. Example 6 (unequal mixed partials) worked carefully; Schwarz's theorem; Taylor's formula and Example 7. Finish with Exam-corner E4 done live.
Discussion questions. (1) Why is "differentiable" defined by one linear map for all directions, rather than by derivatives along lines? What would go wrong with the chain rule otherwise? (2) $f_3=\frac{x^3y}{x^4+y^2}$ has every directional derivative $0$. In what sense is it "flat" at the origin, and in what sense not? (3) Example 6 is $\tfrac14r^2\sin4\theta$ in polar form. Which feature of $\sin4\theta$ makes the mixed partials differ, and what happens for $r^2\sin2\theta$?
Misconceptions to watch for. Proving a limit by checking lines (or lines and parabolas); concluding the double limit from equal iterated limits; differentiating a formula valid only for $(x,y)\ne0$ and "plugging in $0$"; believing that existence of all partials (or all directional derivatives) gives differentiability or continuity; forgetting to normalise the direction vector; treating $f_{xy}=f_{yx}$ as automatic.
Suggested homework. I(2, 4, 6, 8), II(1, 3, 4, 5, 7), III(1, 3, 5, 7), IV(2, 3, 5, 6); Exam corner E1, E4, E5.
Research question. How do machine-learning libraries compute exact gradients of programs with millions of inputs — and why do they not simply use the definition of the derivative with a small step $h$?
Why it matters. Training a neural network, fitting a climate model to satellite observations (the "4D-Var" data assimilation used by weather centres in Europe and Asia), and optimising the shape of an aircraft wing all need gradients of functions of very many variables. The technique behind them, automatic differentiation, is nothing but the chain rule (Theorem 6.6) applied mechanically to the elementary operations of a program.
What to do.
- Take $f(x,y)=e^{xy}\sin(x+y^2)$ and compute $\nabla f(1,\tfrac12)$ exactly with a computer algebra system.
- Approximate $f_x(1,\tfrac12)$ by the forward difference $\frac{f(1+h,\frac12)-f(1,\frac12)}{h}$ and the central difference $\frac{f(1+h,\frac12)-f(1-h,\frac12)}{2h}$ for $h=10^{-1},10^{-2},\dots,10^{-14}$. Plot the errors on a log–log scale and explain the V-shape: Taylor's theorem gives a truncation error proportional to $h$ (or $h^2$), while rounding contributes about $10^{-16}/h$.
- Implement dual numbers $a+b\varepsilon$ with $\varepsilon^2=0$: define $+$, $\times$, $\exp$ and $\sin$ (e.g. $\sin(a+b\varepsilon)=\sin a+b\cos a\,\varepsilon$). Show that evaluating $f$ at $(1+\varepsilon,\tfrac12)$ returns $f(1,\tfrac12)+f_x(1,\tfrac12)\,\varepsilon$ exactly, and explain why this is the chain rule.
- Forward mode needs one pass per input variable. Read about reverse mode (backpropagation): it computes the whole gradient of $f:\mathbb{R}^n\to\mathbb{R}$ in a few passes, by multiplying the Jacobians of the chain rule in the opposite order. Why is the order of a matrix product so important when the first factor is a single row?
Tools: Python with sympy (step 1) and plain floating point (steps 2–3); any plotting tool.
Hand in: The error plot with the two optimal step sizes marked, your dual-number code with its output, and a paragraph comparing the costs of forward and reverse mode for $n=10^6$ inputs.
Going further. For $F:\mathbb{R}^n\to\mathbb{R}^m$ built from $L$ layers, $J_F=J_L\cdots J_2J_1$. Find the cheapest way to multiply out such a product when the matrices have very different shapes (the "matrix chain" problem), and relate it to the forward/reverse choice.
🧪 Self-check quiz
Q1. Let $f(x,y)=\dfrac{xy}{x^2+y^2}$ for $(x,y)\ne(0,0)$ and $f(0,0)=0$. At the origin,
Q2. If $f:\mathbb{R}^n\to\mathbb{R}$ is differentiable at $a$ and $u$ is a unit vector, then $D_uf(a)$ equals
Q3. Which condition guarantees that $f:\mathbb{R}^2\to\mathbb{R}$ is differentiable at $a$?
Q4. For $f(x,y)=\dfrac{xy(x^2-y^2)}{x^2+y^2}$, $f(0,0)=0$, the values $\left(f_{xy}(0,0),f_{yx}(0,0)\right)$ are
Q5. If $f$ is differentiable and homogeneous of degree $4$ on $\mathbb{R}^2\setminus\{0\}$ with $f(1,1)=2$, then $f_x(1,1)+f_y(1,1)$ equals