- The whole of Calculus is reducible to quadrature (computing integrals by summation). (Leibniz)
- I want to find all possible universal methods for solving all problems straight away. (Leibniz)
DigiMat presents Digital Calculus as the Calculus of the computer age as time stepping of dx = f(t)*dt, releasing the power of Symbolic Calculus as classical Calculus. Digital Calculus is easy to learn and powerful, while Symbolic Calculus is difficult to learn (tricky), beautiful but powerless.
The Fundamental Theorem of Calculus
The Calculus of Newton/Leibniz (late 17th century) opened to the Industrial forever changing human conditions in the form of mathematics of change as the initial value problem of finding a function x(t) of the (time) variable t, which solves (compare home page head picture):
- dx= f(t)*dt (or dx/dt = f(t) formally dividing by dt) for t>0,
where f(t) is a given function of t and x0 a given initial value.
Computing x(t) by time stepping dx = f(t)*dt in the form
- x(t+dt) = x(t) + f(t)*dt (1)
- (same as x(t+dt) – x(t) = f(t)*dt or dx = f(t)*dt with dx = x(t+dt) – x(t))
with dt > 0 a given time step, successively computes x(t+dt) at next time t+dt from x(t) at present time t plus the change f(t)*dt.
Repeating this process corresponds to summation over all time steps to a final time T=N*dt after N steps (in each time step adding f(t)*dt with t = n*dt)
To capture this summation formula, Leibniz introduced (with a stroke of genius) the notation
where the integral sign ∫ in the form of an extended S represents the summation with t as integration variable and dt as time step.
We can now express the Fundamental Theorem of Calculus as:
- Time stepping of dx= f(t)*dt constructs the integral or primitive function of the integrand f(t) by summation or quadrature.
- The derivate dx/dt of x(t) is defined as dx/dt = f(t) as another way of writing dx = f(t)*dt, that is, the derivative of the integral is the integrand.
The essence of Calculus (following Leibniz) is thus to construct the integral or primitive function x(t) by time stepping dx = f(t)*dt with f(t) given time, thus by a process of successive summation. As a result of the construction, the derivate dx/dt appears by dividing (1) by dt
- dx/dt = (x(t+dt) – x(t))/dt = f(t) (or ), (2)
as the difference quotient
- dx/dt = (x(t+dt) – x(t))/dt (or ).
Leibniz vs Newton
Note that the time step dt is small and that division by a small number is sensitive to perturbations, which means that direct computation of dx/dt as a difference quotient from the values of x(t+dt) and x(t) is sensitive to perturbations in the values of x. In the setting of time stepping dx = f(t)*dt the division by dt is not performed, since by construction dx/dt = f(t).
This directly connects to Newton’s version of Calculus (forming the “dot-age”) as the differential equation
with first priority to define as the limit of as dt tends to zero. The mystery of Calculus under the name of infinitesimal calculus or calculus of the infinitesimals with dt a quantity which is smaller than any positive number yet not zero, that is infinitesimally small, connects Newton’s version of Calculus. There is no mystery in Leibniz version. Only time stepping with smaller and smaller time step.
Leibniz: Start computing integrals (easy). Newton: Start by computing derivatives (hard)
It is instructive to consider the following analog mechanical integrator (very popular before the computer) which computes the integral ∫ y(x) dx of a function y(x). Can you explain how it works?
Central Question: Accuracy
The dependence of the integral x(T) on the size of the time step dt is a central question. The goal is to see that taking smaller and smaller time steps will compute the integral with increasing accuracy. If the accuracy scales with dt then the accuracy is said to be first order, with dt*dt second order et cetera. For first order the accuracy improves by a factor 2 upon decreasing dt with a factor 2. For second order the accuracy improves by a factor 4 = 2*2, et cetera.
Mystery of Calculus
Calculus is surrounded by mystery concerning the definition of derivative through a difference quotient involving division by a time step dt supposed to be so small that it is zero, yet it is not zero.
- To those who ask what the infinitely small quantity in mathematics is, we answer that it is actually zero. Hence there are not so many mysteries hidden in this concept as they are usually believed to be. (Euler)
- Laplace carried the spirit of the infinitesimal into administration. (Napoleon, when Laplace hade served for six weeks as minister of the interior)
Extension to f(x,t) depending also on x is direct, see 5. and 8. below.
A basic case is f(x,t)=x for which the integral is x =exp(t) with the remarkable property of being equal to its derivative as shown here.
More generally, the elementary functions sin(t), cos(t), log(t) and exp(t), and special functions such as Bessel functions, all arise as solutions to initial value problems of basic form, and their values are computed by time stepping as shown here and in BesselFunction V2 V3 V4 V5 V6 V7.
Things To do
1. Study experimentally by computing the dependence of on the time step dt. Compare with Midpoint Euler vs Forward/Endpoint Euler. Divide time step by 2 and see effect.
2. Solve formally dx/dt = t for t > 0 with x(0)=0. (Hint: show that x(t)=t*t/2 satisfies x(t+dt) – x(t) = t + quantity of size dt*dt)
3. Solve formally dx/dt = t*t for t > 0 with x(0)=0.
4. Solve formally dx/dt = t*t….*t (n times) for t > 0 with x(0)=0. (Hint)
5. Compare the effect of taking one step with time step 2*dt to taking two steps with time step dt when time stepping dx = f(t)*dt. See that the difference is (f(t)-f(t+dt))*dt, which is bounded by L*dt*dt if f(t) is Lipschitz continuous with Lipschitz constant L. Conclude that the variation of the integral x(T) with the time step dt is bounded by T*L*dt, Conclude that the integral of a Lipschitz continuous function can be computed to arbitrary accuracy by time stepping. In short: if the variation over one time step scales with dt*dt, then the variation after 1/dt time steps scales with dt.
6. Compare accuracy (see Midpoint Euler vs Forward/Endpoint Euler ) of
- Forward Euler: x(t+dt) = x(t) + f(t)*dt
- Backward Euler: x(t+dt) = x(t) + f(t+dt)*dt
- Midpoint Euler: x(t+dt) = x(t) + f(t+dt/2)*dt
Find that Forward and Backward Euler are first order accurate in the sense that the variation of x(T) with the time step is bounded by T*L*dt. Find that Midpoint Euler is second order accurate with variation bounded by T*L*dt*dt.
7. Note that if x(t) solves dx = f(t)*dt and |f(t)| < L, then |x(t+dt) – x(t)| = |dx| < L*|dt|, which means that x(t) is Lipschitz continuous with Lipschitz constant L.
8. Compare the effect when time stepping dx = f(x)*dt where f(x) is Lipschitz continuous with Lipschitz constant L of taking one step with time step 2*dt starting from time t to taking two steps with time step dt , assuming the same initial data at t. See that the difference is (f(x) – f(x+f(x)*dt))*dt and that therefore that the difference is bounded by L*|f(x)|*dt*dt < L*M*dt*dt if |f(x)|<M. See that the effect at t+dt of using different initial data at t can be bounded by the factor (1+L*dt). See that this effect after N=T/dt steps is bounded by (1+L*dt)^N ≈ exp(L*T). Combine to conclude that the variation of the solution x(T) of dx/dt = f(x), where f(x) is bounded and Lipschitz continuous, with the time step dt, is bounded by exp(L*T)*T*L*M*dt. Conclude that x(T) can be computed to arbitrary accuracy by time stepping. In short: if the variation over one time step scales with dt*dt, then the variation after 1/dt time steps scales with dt. If f(x) is Lipschitz continuous. Extend to f(x,t) depending on both x and t.
9. Complete the argument in 5. and 8. by noting that if the effect of decreasing the time step from dt to dt/2, is bounded by dt/2, then the total effect of continued decrease of the time step with a factor of 2, is (1/2 + 1/4 + 1/8 + ….)*dt< dt. The error when using time step dt as compared to using a vanishingly small time step, thus is of size dt as an expression of first order convergence.
10. Let x(t) solve dx/dt = f(x) for 0<t<T with x(0)=0, and let X(t) be a computed (approximate) solution satisfying dX/dt = f(X) + R(X) for 0<t<T with X(0)=0 and R(X) the residual, as a measure of how well X satisfies the given equation. See that e = x – X satisfies de/dt = Df*e + R(X), where Df(t) =df/dx evaluated between x(t) and X(t). Let phi(t) be a dual solution satisfying -dphi/dt = Df*phi for T>t>0 (backwards in time) with phi(T)=1. Then dphi/dt*e=- Df*phi*e=-phi*Df*e= -phi*de/dt -R(X)*phi, that is d/dt (phi*e) = R(X)*phi, so that integrating from 0 to T, using that phi(T)=1 and e(0)=0,
- e(T)=int_0^T R(X)*phi dt.
This gives a representation of the error e = x-X at T in terms of an integral over time of the residual R(X) with the dual solution phi(t) as weight. Computing an approximation of the dual solution then gives an a estimate of the error in terms of the residual R(X) which is computable from X, thus an a posteriori error estimate as an estimate of the error which is computable in terms of the computed solution X. This to be compared with an a priori error estimate in terms of the (unknown) exact solution x(t), as in 8. and 9. Compare with Error Estimation by Duality.
Essence 1 of Calculus:
If f(t) is Lipschitz continuous, then time stepping with Forward Euler solves dx = f(t)*dt with global error bounded by constant*dt and local variation
- x(t+dt) = x(t) + f(t)*dt + E, with |E| < constant*dt*dt,
expressing that x(t) is locally close to a linear function with slope dx/dt = f(t). Illustrate in a picture! Example f(t) = cos(t), x(t)=sin(t), x(0)=0.
If |f(t)| is bounded, then a solution x(t) of dx = f(t)*dt is Lipschitz continuous and so is locally close to a constant function with local variation bounded by constant*dt. Illustrate in a picture!
Essence 2 of Calculus:
If f(x) is Lipschitz continuous, then time stepping with Forward Euler solves dx = f(x)*dt with global error bounded by constant*dt and local variation
- x(t+dt) = x(t) + f(x(t))*dt + E, with |E| < constant*dt*dt,
expressing that x(t) is locally close to a linear function with slope dx/dt = f(x). Example f(x)=x, x(t)=exp(t), x(0)=1.