Error Analysis of Series

Section 2.4 Error Analysis of Series

When we truncate a series to get a Taylor polynomial, we obviously change the function. But how much have we changed it? This change is called the error of the Taylor polynomial, so we ask: what is the error introduced by truncating an infinite series? The study of approximations in mathematics is all about understanding and controlling error.

For Taylor series, there is a theorem that controls the error of Taylor polynomial approximation. This is sometimes called Taylor's Theorem or the Lagrange Error Bound.

Theorem 2.4.1.

Let \(f\) be analytic at \(\alpha\) with a positive radius of convergence \(R\) and let \(d \in \RR\) with \(0\lt d\lt R\text{.}\) Let \(M \in \RR_{> 0}\) such that on the interval \((\alpha-d, \alpha+d)\) we have

\begin{equation*} |f^{k+1}(x)| \leq M\text{.} \end{equation*}

The error of the \(k\)th Taylor polynomial approximation can be written as

\begin{equation*} R_k = f - \sum_{n=0}^k c_n (x-\alpha)^n = \sum_{n=k+1}^\infty c_n (x-\alpha)^n\text{.} \end{equation*}

Under all these assumptions, the error satisfies an inequality.

\begin{equation*} |R_k(x)| \leq \frac{M}{(k+1)!} |x-\alpha|^{k+1} \end{equation*}

This is a difficult theorem to understand. The basic idea is that there exists a balance between three pieces: the order of the approximation, the width of the interval, and the size of the error. Higher order approximations are more accurate, but they are also computationally more difficult; to work on computers, we want to use a higher enough order to be accurate, but a low enough order to allow the computer to finish the calculation quickly. Likewise, a small interval minimizes the error, since error tends to be larger the farther we move away from the centre of the series. We would like to work with a large interval, since it means that one series works for many difficult calculations, but we may not have the accuracy we desire on a large interval. To summarize: we want a low order, a large interval and a small error, but it is difficult to achieve all three. Sometimes, one is sacrificed for the others.

Example 2.4.2.

Consider \(f(x) = \sin x\text{.}\) We can take \(M=1\text{,}\) since the derivatives of \(\sin x\) are \(\pm \sin x\) and \(\pm \cos x\text{,}\) all of which are bounded in absolutely value by \(1\text{.}\) Then we can ask: what order do we need in order to have \(\frac{1}{10000}\) precision on \((-10,10)\text{?}\) The error calculation looks like:

\begin{align*} \frac{M}{(n+1)!} |x|^{n+1} \amp \leq \frac{1}{10^4}\\ \frac{1}{(n+1)!} |10|^{n+1} \amp \leq \frac{1}{10^4} \end{align*}

This is actually quite difficult to solve for \(n\text{.}\) However, we can try some \(n\text{.}\) Here are some values for \(n\) and the error bound:

\begin{equation*} \begin{array}{*4{>{\displaystyle}l}} n=5 \amp |R_5(x)| \leq \frac{12500}{9} \amp n=10 \amp |R_{10}(x)| \leq \frac{15625000}{6237} \\ n=15 \amp |R_{15}(x)| \leq 477 \amp n=18 \amp |R_{18}(x)| \leq 82.3 \\ n=20 \amp |R_{20}(x)| \leq 19.6 \amp n=22 \amp |R_{22}(x)| \leq 3.87 \\ n=24 \amp |R_{24}(x)| \leq 0.645 \amp n=26 \amp |R_{26}(x)| \leq 0.0919 \\ n=28 \amp |R_{28}(x)| \leq 0.0114 \amp n=30 \amp |R_{30}(x)| \leq 0.00122 \\ n=31 \amp |R_{31}(x)| \leq 0.000381 \amp n=32 \amp |R_{32}(x)| \leq 0.000116 \\ n=33 \amp |R_{33}(x)| \leq 0.0000339 \amp \amp \end{array} \end{equation*}

So, finally, when \(n=33\) we have the desired accuracy. We need an order \(33\) polynomial to achieve the desired precision on the desired interval.

What about smaller intervals? Say we only need the interval \(\left( \frac{-1}{10}, \frac{1}{10} \right)\text{.}\) Let's see how accurate various orders are on this interval.

\begin{align*} \amp n=5 \amp \amp |R_5(x)| \leq 1.39 \times 10^{-9}\\ \amp n=10 \amp \amp |R_{10}(x)| \leq 2.51 \times 10^{-19}\\ \amp n=15 \amp \amp |R_{15}(x)| \leq 4.78 \times 10^{-30} \end{align*}

It is much easier to be precise on this smaller interval.

Example 2.4.3.

Consider the function \(f(x) = \frac{1}{1-x}\) on \((\frac{-1}{2}, \frac{1}{2})\text{.}\) Look at the pattern of derivatives.

\begin{equation*} f^{(n)}(x) = \frac{n!}{(1-x)^{n+1}} \end{equation*}

\(M\) bounds the \(n+1\)st derivative, and the derivatives are increasing functions, so they are maximized at \(x = \frac{1}{2}\text{.}\) This lets us calculate \(M\text{.}\)

\begin{equation*} M = \frac{(n+1)!}{(1-\frac{1}{2})^{n+2}} = 2^{n+2}(n+1)! \end{equation*}

We apply the inequality in the Lagrange Error Bound.

\begin{equation*} R_n \leq \frac{M}{(n+1)!}d^{n+1} = \frac{2^{n+2}(n+1)!}{(n+1)!} \left( \frac{1}{2} \right)^{n+1} = 2 \end{equation*}

This is entirely inconclusive, for any order, showing the potential limitations of the Lagrange Error Bound; sometimes, other methdos are required for special situation. For this example, we must work directly with the series to determine its accuracy. Here is a direct calculation of a bound for the error.

\begin{align*} R_n \amp = \sum_{k=n+1}^\infty x^k \leq \sum_{k=n+1}^\infty \left( \frac{1}{2} \right)^k = \sum_{k=n+1}^\infty \left( \frac{1}{2} \right)^{k-n-1} \left( \frac{1}{2} \right)^{n+1}\\ \amp = \left( \frac{1}{2} \right)^{n+1} \sum_{k=n+1}^\infty \left( \frac{1}{2} \right)^{k-n-1} = \left( \frac{1}{2} \right)^{n+1} \sum_{k=0}^\infty \left( \frac{1}{2} \right)^{k} = \left( \frac{1}{2} \right)^{n+1} \frac{1}{1 - \frac{1}{2}}\\ \amp = \left( \frac{1}{2} \right)^{n+1} 2 = \frac{1}{2^n} \end{align*}

So the error of the \(n\)th order can be bounded by \(\frac{1}{2^n}\text{.}\) For \(n=10\text{,}\) that is less than a thousandth. For \(n=20\text{,}\) less than a millionth, and so on.