Definition 4.5.1.
The matrix of the linear approximation to a scalar function \(f: \RR^n \rightarrow \RR\) is the \(1 \times n\) matrix of partial derivatives.
Section 4.5 Linear Approximation
Subsection 4.5.1 Single Variable Interpretation
Consider a single variable function \(f: \RR \rightarrow
\RR\text{.}\) In Calculus I, I defined the linear approximation of \(f\) at the point \((a,f(a)\text{.}\)
\begin{equation*}
f(x) \approx f(a) + f^\prime(a) (x-a)
\end{equation*}
From Calculus II, you might recognize this as simply the first-order Taylor approximation for \(f\text{:}\) the Taylor series truncated after the linear term. The linear approximation is the line that best approximates \(f\) at this point. Its graph is simply the tangent line to \(f\) at \((a,f(a))\text{.}\)
I can rearrange the linear approximaiton to help extent the idea to multivariable functions.
\begin{equation}
(f(x) - f(a)) \approx f^\prime(a) (x-a)\tag{4.5.1}
\end{equation}
In this course, I have encouraged you to think of points in \(\RR^n\) along with local direction vectors: each point can be thought of as the origin for a system of local directions. I can do the same here: let \((a,f(a))\) be a local origin for a system of local directions. Then switching from \(f(x)\) to \(f(x) - f(a)\) and from \(x\) to \(x-a\) is just moving from the usual origin to this new, local, origin.
In this new origin, the function in Equation (4.5.1) is approximated by multiplication by \(f^\prime(a)\text{.}\) This gives a new interpretation for the single-variable derivative: the derivative is the multiplicative factor for the local linear approximation of \(f\text{.}\) Locally, (in coordinates pretending that \((a,f(a))\) is the origin), the function is approximated by multiplication by \(f^\prime(a)\text{.}\)
Now think of a function \(f: \RR^2 \rightarrow \RR\text{.}\) To approximate by linear functions, I need to understand linear functions: \(\RR^2 \rightarrow \RR\text{.}\) I look to matrices. Matrices completely describe linear functions. A \(1 \times 1\) matrix is just a number, and the linear function \(\RR \rightarrow \RR\) is just multiplication by that number. But for \(\RR^2 \rightarrow
\RR\text{,}\) a \(1 \times 2\) matrix describes the linear function.
Subsection 4.5.2 Linear Approximation in \(\RR^2\)
So I ask: what is a linear approximation to the function \(f:
\RR^2 \rightarrow \RR\text{.}\) It must be a \(1 \times 2\) matrix \(M\) and fit into a equation mirroring Equation (4.5.1).
\begin{equation*}
f(x,y) - f(a,b) \approx M \begin{pmatrix}x-a\\y-b\end{pmatrix}
\end{equation*}
In local coordinates at \((a,b,f(a,b))\text{,}\) this is just matrix multiplication by \(M\text{.}\) So a linear approximation is a matrix multiplication in local coordintaes. What is the matrix \(M\text{?}\) Well, if I work with partial derivatives, the linear approximation should be formed of linear approximation in \(x\) and linear approximation in \(y\text{.}\)
\begin{equation*}
f(x,y) \approx f(a,b) + \frac{\del f}{\del x} (a,b) (x-a) +
\frac{\del f}{\del x} (a,b) (y-b)
\end{equation*}
I put this in matrix form.
\begin{equation*}
f(x,y) - f(a,b) \approx \left( \begin{matrix} \frac{\del
f}{\del x} \amp \frac{\del f}{\del y} \end{matrix} \right)
\begin{pmatrix}x-a\\y-b\end{pmatrix}
\end{equation*}
The graph of the linear approximation is the tangent plane at the point \((a,b,f(a,b))\text{.}\) This matches the equation in Proposition 4.3.1.
Subsection 4.5.3 Examples
Example 4.5.2.
I’ll return to the function in Example Example 4.3.2, \(f(x,y) = \frac{1}{1 + x^2
+ y^2}\text{.}\)
\begin{equation*}
M = \left( \begin{matrix} \frac{-2x}{(1+x^2+y^2)^2} \amp
\frac{-2y}{(1+x^2+y^2)^2} \end{matrix} \right)
\end{equation*}
Look at the point \((0,0)\text{.}\)
\begin{equation*}
f(x,y) \approx f(0,0) + \left( \begin{matrix} 0 \amp 0
\end{matrix} \right) \begin{pmatrix}x\\y\end{pmatrix} = 1
\end{equation*}
This linear approximation is a constant \(1\text{,}\) which makes sense at the top of the small hill. Momentarily, at the peak, nothing is changing and the function doesn’t do anything. The linear approximation to doing nothing is appropriately a constant.
Look at the point \((1,1)\text{.}\)
\begin{equation*}
f(x,y) \approx f(1,1) + \left( \begin{matrix} \frac{-2}{9}
\amp \frac{-2}{9} \end{matrix} \right)
\begin{pmatrix}x-1\\y-1\end{pmatrix} = \frac{1}{3} -
\frac{2(x-1)}{9} - \frac{2(y-1)}{9}
\end{equation*}
Look at the point \((-2,2)\text{.}\)
\begin{equation*}
f(x,y) \approx f(-2,2) + \left( \begin{matrix}
\frac{4}{81} \amp \frac{-4}{81} \end{matrix} \right)
\begin{pmatrix}x+2\\y-2\end{pmatrix} = \frac{1}{9} +
\frac{4(x+2)}{81} - \frac{4(y-2)}{81}
\end{equation*}
Example 4.5.3.
\(f(x,y) = x^2 e^{x-y}\text{.}\)
\begin{equation*}
M = \left( \begin{matrix} 2xe^{x-y} + x^2e^{x-y}, -x^2
e^{x-y} \end{matrix} \right)
\end{equation*}
Look at the point \((2,2)\text{.}\)
\begin{equation*}
f(x,y) \approx f(2,2) + \left( \begin{matrix} 4+4 \amp
-4 \end{matrix} \right)
\begin{pmatrix}x-2\\y-2\end{pmatrix} = 4 + (4+4e^2)(x-2) -
4(y-2)
\end{equation*}
Look at the point \((-1,-1)\text{.}\)
\begin{equation*}
f(x,y) \approx f(-1,-1) + \left( \begin{matrix} (2+e) \amp
-1 \end{matrix} \right)
\begin{pmatrix}x+1\\y+1\end{pmatrix} = 1 + (2+e)(x+1) -
(y+1)
\end{equation*}
Subsection 4.5.4 What Is A Derivative?
I extended the idea of a derivative with partial derivatives and directional derivatives. These are both useful, but neither are a universal extension. Both are only pieces of derivatives along certain directions. Gradients were a more universal extension, but they only identify the direction of greatest change. Tangent planes were a good geometric extension, but without an algebraic analogue.
The discussion of linear approximation leads to a much more universal idea to generalize the derivative. The big idea is this: derivatives, in any dimension, are linear approximations to functions. This idea works in single variables, where multiplication by \(f^\prime(a)\) was the linear approximation. The derivatives calculates that factor. For higher dimensions, I can use matrix multiplication instead of just multiplication by a number.
Therefore, I could realistically say that the derivative of a function \(f: \RR^n \rightarrow \RR^m\) is the matrix of partial derivatives, which serves as a linear approximation of the function at any point in its domain. It will be useful to give this matrix a name, for future reference. I’ll state the definition so that it works for scalar fields (\(m=1\)) for this course, as well as for vector fields (\(m \gt 1\)), which is important in Calculus IV.
Definition 4.5.4.
Let \(f: \RR^n \rightarrow \RR^m\) be a differentiable function. Then the \(m \times n\) matrix of partial derivatives of \(f\) is called the Jacobian Matrix of the function.
