1.1 Review of Calculus#

There are several concepts and facts from Calculus that we need in Numerical Analysis. In this section we will list some definitions and theorems that will be needed later. For the most part functions in this book refer to real valued functions defined on real numbers \(\mathbf{R}\), or an interval \((a,b)\subset\mathbf{R}\).

Definition 1

  1. A function \(f\) has the limit \(L\) at \(x_{0}\), written as \(\lim_{x\rightarrow x_{0}}f(x)=L,\) if for any \(\epsilon>0,\) there exists \(\delta>0\) such that \(|f(x)-L|<\epsilon\) whenever \(0<|x-x_{0}|<\delta\).

  2. A function \(f\) is continuous at \(x_{0}\) if \(\lim_{x\rightarrow x_{0}}f(x)=f(x_{0}),\) and \(f\) is continuous on a set \(A\) if it is continuous at each \(x_{0}\in A\).

  3. Let \(\{x_{n}\}_{n=1}^{\infty}\) be an infinite sequence of real numbers. The sequence has the limit \(x,\) i.e., \(\lim_{n\rightarrow\infty}x_{n}=x\) (or, written as \(x_{n}\rightarrow x\) as \(n\rightarrow\infty)\) if for any \(\epsilon>0,\) there exists an integer \(N>0\) such that \(|x_{n}-x|<\epsilon\) whenever \(n>N\).

Theorem 1

The following are equivalent for a real valued function \(f:\)

  1. \(f\) is continuous at \(x_{0}\)

  2. If \(\{x_{n}\}_{n=1}^{\infty}\) is any sequence converging to \(x_{0},\) then \(\lim_{n\rightarrow\infty}f(x_{n})=f(x_{0})\).

Definition 2

We say \(f(x)\) is differentiable at \(x_{0}\) if

\[\begin{equation*} f'(x_{0})=\lim_{x\rightarrow x_{0}}\frac{f(x)-f(x_{0})}{x-x_{0}}=\lim_{h\rightarrow0}\frac{f(x_{0}+h)-f(x_{0})}{h} \end{equation*}\]

exists.

Notation:

\(C^{n}(A)\) denotes the set of all functions \(f\) such that \(f\) and its first \(n\) derivatives are continuous on \(A.\) If \(f\) is only continuous on \(A\), then we write \(f\in C^{0}(A).\) \(C^{\infty}(A)\) consists of functions that have derivatives of all orders, for example, \(f(x)=\sin x\) or \(f(x)=e^{x}\).

The following well-known theorems of Calculus will often be used in the remainder of the book.

Theorem 2 (Mean value theorem)

If \(f\in C^{0}[a,b]\) and \(f\) is differentiable on \((a,b)\), then there exists \(c\in(a,b)\) such that \(f'(c)=\frac{f(b)-f(a)}{b-a}\).

Theorem 3 (Extreme value theorem)

If \(f\in C^{0}[a,b]\) then the function attains a minimum and maximum value over \([a,b].\) If \(f\) is differentiable on \((a,b),\) then the extreme values occur either at the endpoints \(a,b\) or where \(f'\) is zero.

Theorem 4 (Intermediate value theorem)

If \(f\in C^{0}[a,b]\) and \(K\) is any number between \(f(a)\) and \(f(b)\), then there exists \(c\in(a,b)\) with \(f(c)=K\).

Theorem 5 (Taylor’s theorem)

Suppose \(f\in C^{n}[a,b]\) and \(f^{(n+1)}\) exists on \((a,b),\) and \(x_{0}\in(a,b).\) Then, for \(x\in (a,b)\)

\[\begin{equation*} f(x)=P_{n}(x)+R_{n}(x) \end{equation*}\]

where \(P_{n}\) is the \(n\)th order Taylor polynomial

\[\begin{equation*} P_{n}(x)=f(x_{0})+f'(x_{0})(x-x_{0})+f''(x_{0})\frac{(x-x_{0})^{2}}{2!}+...+f^{(n)}(x_{0})\frac{(x-x_{0})^{n}}{n!} \end{equation*}\]

and \(R_{n}\) is the remainder term

\[\begin{equation*} R_{n}(x)=f^{(n+1)}(\xi)\frac{(x-x_{0})^{n+1}}{(n+1)!} \end{equation*}\]

for some \(\xi\) between \(x\) and \(x_{0}\).

Example 1

Let \(f(x)=x\cos x -x.\)

  1. Find \(P_{3}(x)\) about \(x_{0}=\pi/2\) and use it to approximate \(f(0.8).\)

  2. Compute the exact value for \(f(0.8)\), and the error \(|f(0.8)-P_{3}(0.8)|.\)

  3. Use the remainder term \(R_{3}(x)\) to find an upper bound for the error \(|f(0.8)-P_{3}(0.8)|.\) Compare the upper bound with the actual error found in part 2.

Solution.

  1. First note that \(f(\pi/2)=-\pi/2\). Differentiating \(f\) we get:

    \[\begin{align*} f'(x) & =\cos x - x \sin x -1 \Rightarrow f'(\pi/2)=-\pi/2-1\\ f''(x) & =-2\sin x -x \cos x \Rightarrow f''(\pi/2)=-2\\ f'''(x) & =-3 \cos x+x \sin x\Rightarrow f'''(\pi/2)=\pi/2. \end{align*}\]

    Therefore

    \[\begin{equation*} P_{3}(x)=-\pi/2-(\pi/2+1)(x-\pi/2)-(x-\pi/2)^2+\frac{\pi}{12} (x-\pi/2)^3. \end{equation*}\]

    Then we approximate \(f(0.8)\) by \(P_{3}(0.8)=-0.3033\) (using 4-digits with rounding).

  2. The exact value is \(f(0.8)=-0.2426\) and the absolute error is \(|f(0.8)-P_{3}(0.8)|=0.06062\).

  3. To find an upper bound for the error, write

    \[\begin{equation*} |f(0.8)-P_{3}(0.8)|=|R_{3}(0.8)| \end{equation*}\]

    where

    \[\begin{equation*} R_{3}(0.8)=f^{(4)}(\xi)\frac{(0.8-\pi/2)^{4}}{4!} \end{equation*}\]

    and \(\xi\) is between \(0.8\) and \(\pi/2\). We need to differentiate \(f\) one more time: \(f^{(4)}(x)=4\sin x +x \cos x.\) Since \(0.8<\xi<\pi/2\), we can find an upper bound for \(f^{(4)}(x),\) and thus an upper bound for \(R_{3}(0.8),\) using triangle inequality:

    \[\begin{align*} \left|R_{3}(0.8)\right| & =\left|f^{(4)}(\xi)\frac{(0.8-\pi/2)^{4}}{4!}\right|=\left|4 \sin\xi + \xi \cos \xi\right|(0.01471)\\ & \leq 0.05883 | \sin \xi | + 0.01471 |\xi| |\cos \xi|. \end{align*}\]

    Note that on \(0.8<\xi<\pi/2\), \(\sin\xi\) is a positive increasing function, and \(|\sin(\xi)|<\sin(\pi/2)=1\). For the second term, we can find an upper bound by observing \(|\xi|\) attains a maximum value of \(\pi/2\) on \(0.8<\xi<\pi/2\), and \(\cos\xi\), which is a decreasing positive function on \(0.8<\xi<\pi/2\), has a maximum value of \(\cos{(0.8)}=0.6967\). Putting these together, we get

    \[\begin{equation*} \left|R_{3}(0.8)\right| < 0.05883 (1) + (0.01471) (\pi/2) (0.6967) \approx 0.07493. \end{equation*}\]

    Therefore, our estimate for the actual error (which is 0.06062 from part 2) is less than 0.07493.

Excercise 1.1-1

Find the second order Taylor polynomial for \(f(x)=e^{x}\sin x\) about \(x_{0}=0\).

a. Compute \(P_{2}(0.4)\) to approximate \(f(0.4)\). Use the remainder term \(R_{2}(0.4)\) to find an upper bound for the error \(|P_{2}(0.4)-f(0.4)|\). Compare the upper bound with the actual error.

b. Compute \(\int_{0}^{1}P_{2}(x)dx\) to approximate \(\int_{0}^{1}f(x)dx\). Find an upper bound for the error using \(\int_{0}^{1}R_{2}(x)dx\), and compare it to the actual error.