Proof of Fundamental Theorem of Calculus

  • The quadrature of all figures follow from the inverse method of tangents, and thus the whole science of sums and quadratures can be reduced to analysis, a thing nobody even had any hopes of before. (Leibniz)
  • Knowing thus the Algorithm of this calculus, which I call Differential Calculus, all differential equations can be solved by a common method. (Leibniz

Let us now study the effect of the time step in solution of the basic IVP

  • \dot u(t)=f(t),\quad\mbox{for }t>0,\quad u(0)=u^0,

by Forward Euler time stepping

  • u(ndt+dt)=u(ndt)+f(ndt)dt,\quad n=0,1,2,...

We compare taking one step with time step dt with two steps of time step \frac{dt}{2}, for a given n:

  • u(ndt+dt)-\bar u({ndt+dt)}=f(ndt)dt - (f(ndt)+f(ndt+\frac{dt}{2}))\frac{dt}{2}
  • =(f(ndt)-f(ndt+\frac{dt}{2}))\frac{dt}{2},

where \bar u is computed with time step \frac{dt}{2}, and we assume that the same intial value for t=ndt is used so that \bar u(ndt)=u(ndt). Assuming that f(t) is Lipschitz continuous with Lipschitz constant L, we then find that

  • \vert u(ndt +dt)-\bar u(ndt +dt)\vert \le \frac{L}{4} dt^2.

Summing now the contributions from all time steps with n=0,1,2,...,N, where T=(N+1)dt is a final time, we get using that \sum_{n=0}^Ndt=T,

  • \vert u(T)-\bar u(T)\vert \le \frac{LT}{4}\, dt,

where thus u(T) is computed with time step dt and \bar u(T) with time step \frac{dt}{2}.

Repeating the argument with successively refined times step \frac{dt}{4},\frac{dt}{8},..., we get

  • \vert u(T)-\bar u(T)\vert \le \frac{LT}{2}\, dt

for the difference between u(T) computed with time step dt and \bar u(T) computes with vanishingly small time step, since

  • \frac{1}{4}+\frac{1}{8}+\frac{1}{16}+...< \frac{1}{2}.

The fundamental step in the proof of the Fundamental Theorem.

We have now proved the Fundamental Theorem of Calculus:

Theorem If f:[0,T]\rightarrow R is Lipschitz continuous, then the function u(t)=\int_0^tf(s)\, ds defined by Forward Euler time-stepping with vanishing time step, solves the IVP: \dot u(t)=f(t) for t\in (0,T), u(0)=0.

Understanding the Fundamental Theorem

The proof shows what it means to understand the Fundamental Theorem of Calculus:  This is to realize that (letting k denote a finite time step and dt a vanishingly small step)

  • u(T)=\int_0^Tf(t)\, dt\approx \sum_{n=0}^Nf(nk)k\quad\mbox{if }T=(N+1)k,

as a consequence of

  • u((n+1)k)\approx u(nk)+f(nk)k,\quad\mbox{or}\quad\frac{u((nk+k)-u(nk)}{k}\approx f(nk),

where the sum is referred to as a Riemann sum, with the following bound for the difference

  • \vert \int_0^Tf(t)\, dt-\sum_{n=0}^Nf(nk)k\vert \le \frac{LTk}{2}\quad\mbox{with}T=(N+1)k,

assuming f:[0,T]\rightarrow R is Lipschitz continuous with Lipschitz constant L.

In other words, understanding the integral u(t)=\int_0^tf(s)\, ds of a function f:[0,T]\rightarrow R, means to understand that:

  1. u(t)=\int_0^tf(s)\, ds is determined by Riemann sums with vanishingly small step size, as the solution to the IVP \dot u(t)=f(t), u(0)=0,
  2. the difference between two Riemann sums with mesh size k and \frac{k}{2}, is bounded by Lk (or more precisely by \frac{L}{4}k).

Even Better Understanding

As a serious student, you now probably ask: In precisely what sense the differential equation \dot u(t)=f(t) is satisfied by an Euler Forward solution u(t) with time step k? It certainly is so constructed, but can we get a direct verification? One way to do this is to associate a continuous piecewise linear function determined by the values u(nk) at the discrete time levels nk,again denoted by u(t). We then have on each interval (nk,(n+1)k), by the definition of u(t):

  • \dot u(t)=\frac{u((n+1)k)-u(nk)}{k}=f(nk),

from which we conclude that

  • \vert \dot u(t)-f(t)\vert\le \vert f(nk)-f(t)\vert\le Lk\quad\mbox{for }t\in ((n+1)k,nk).

We can thus say that u(t) satisfies the differential equation \dot u(t)=f(t) for all t with a precision of Lk. In other words, the residual \dot u(t)-f(t) is smaller than Lk.

We have now understood the Fundamental Theorem even better, right?

We shall see below that extending a function defined on a discrete set of points to a continuous piecewise linear function, is a central aspect of approximation in general and of the Finite Element Method FEM in particular.

To Think About

  • What is fundamental about the Fundamental Theorem?
  • Why is \frac{d}{dt}\int_0^tf(s)\, ds=f(t)? (compare with last argument)
  • What is the effect of finite precision computation according to Constructive Calculus in Finite Precision? (see below)
  • What is the Riemann sum error using the Trapezoidal Rule ?

Hint:

  • \int_0^{t+dt}f(s)\, ds-\int_0^tf(s)\, ds=\int_t^{t+dt}f(s)\, ds = f(t)dt\pm \frac{L}{2}dt^2.

The sad result of not understanding the mathematics of Archimedes.

Babbage’s Difference Engine No. 2 1847.

Fundamental Theorem in Finite Precision

Lipschitz continuity in the presence of finite precision can be defined as follows:  A real-valued function f(t) of a real variable t is Lipschitz continuous with Lipschitz constant L in finite precision \epsilon > 0, if for all t and \Delta t

  • \vert f(t+\Delta t) - f(t)\vert\le L\vert \Delta t+ \epsilon\vert.

We see that here dt will effectively be bounded below by \epsilon. With this extension of the concept of Lipschitz continuity to finite precision, the first step of the above proof takes the form

  • \vert u(T)-\bar u(T)\vert \le \frac{LT}{4}\, (dt+\epsilon).

In the second step, the repetition with successively refined times step \frac{dt}{4},\frac{dt}{8},..., is performed until \frac{dt}{2^N} <\epsilon for some natural number N, which gives

  • \vert u(T)-\bar u(T)\vert \le \frac{LT}{2}dt+N\frac{LT}{4}\epsilon

for the difference between u(T) computed with time step dt and \bar u(T) computed with time step dt \ge \epsilon. Here N is the 2-logarithm of \frac{\epsilon}{dt} and thus is a constant of moderate size (not large). In essence, we thus obtain the previous estimate with dt replaced by dt +\epsilon and \epsilon appears as a lower bound of the time step dt

Next: Rules of Integration    Previous: Rules of Differentiation

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: