# Euler's methods

The slope of the secant through and can be approximated by , , or, more accurately, the average of the two: . Correspondingly, we have the following methods:

• Forward Euler's method:

This method uses the derivative at the beginning of the interval to approximate the increment :

 (185)

Comparing this method with the Taylor series expansion of :

 (186)

we see that this method has a second order truncation error .

If the step size is not small enough, this truncation error will accumulate as the iterative process progresses, and so generated may deviate from the true solution quickly in a few steps. Consequently, the iteration may become divergent , i.e., the method may not be stable. Therefore Euler's method is useful only if the step size is sufficiently small, so that the error does not accumulate too quickly.

• Backward Euler's method:

This method uses the derivative at the end of the interval to approximate the increment :

 (187)

Replacing in the expression by its Taylor expansion:

 (188)

we get

 (189)

Comparing this with the Taylor expansion in Eq. (186), we see that the backward Euler's method also has a second order error . As appears on both sides of Eq. (187), this method is implicit, instead of explicit. The desired function value at can be found by solving the following equation for treated as the unknown:

 (190)

The Newton-Raphson method can be used to solve this equation by the following iteration from some initial guess :

 where (191)

Obviously this method is more computationally expensive, however, as an implicit method, it is always stable compared to the forward method, as we will see below.

• Trapezoidal method:

This method uses the average of the derivatives at the beginning and end points of the interval to approximate the increment :

 (192)

To find the order of the truncation error, we rewrite the above as
 (193)

Comparing this with the Taylor expansion in Eq. (186), we see that the trapezoidal method has a third order error .

Same as the backward method, this is also an implicit method. To find , we need to solve the following equation by Newton-Raphson method.:

 (194)

In summary, here is how the three methods find the increment of function :

 (195)

which can be implemented iteratively from the known initial condition

 (196)

Example: Consider a simple first order constant coefficient DE:

 i.e. (197)

with and initial condition . The closed form solution of this equation is known to be , which decays exponentially to zero when . This DE can be solved numerically by each of the three methods.
• The forward Euler's method:
 (198)

The iteration will converge to the solution, only if i.e., or . If the step size is greater than , iteration will diverge.
• The backward Euler's method:

 (199)

Solving the equation for we get

 (200)

• The trapezoidal method:

 (201)

Solving the equation for we get

 (202)

The results of all three methods are shown together with the ground truth solution with and . The four plots correspond to five different step sizes .

We make the following observations:

• The solution of the forward method is always below the true solution while that of the backward method is always above. This is because the negative slop of the solution, whose magnitude is continuously reduced, is over estimated by used in the forward method, but under estimate by used by the backward method. As the average of the two, the result of the trapezoidal method coincide with the true solution more closely.
• The backward method always produces a stable approximation of the true solution, while the performance of the forward method is very sensitive to the step size . In particular, when , the iteration becomes divergent.

As can be seen from the above, there are in general two approaches to estimate the future, such as the next value of a function :

• Explicit method: it is estimated based on the present and past information, such as and . This method can be divergent and therefore unstable (the iteration grows out of bound);
• implicit method: it is estimated based on the future information, such as , as well as the present and past information such as and . This method is in general stable.