To solve equation , we first consider the Taylor series
expansion of at any point :
If is not linear, the sum of the first two terms of the Taylor
expansion is only an approximation of , and the resulting
we found above is only an estimate of the root. However, it can still
be used as a step of an iterative process by which the estimate is
continuously improved and the solution can be eventually reached:
The Newton-Raphson method can be considered as the fixed point
. The root
at which is also the fixed point of , i.e., .
For the iteration to converge, needs to be a contraction with
We make the following notes:
The order of convergence of the Newton-Raphson iteration can be found
based on the Taylor expansion of at the neighborhood of the root
We see that, if , then the order of convergence of the Newton-Raphson method is , and the rate of convergence is . However, if , the convergence is linear rather than quadratic, as shown in the example below.
Example: Consider solving the equation
We consider in general a function with a repeated root at of
In such case, we can accelerate the iteration by using a step size
The difficulty, however, is that the multiplicity of a root is unknown ahead of time. If is used blindly some root may be skipped, and the iteration may oscillate around the real root.
Example: Consider solving , with a double root and a single root . In the following, we compare the performance of both and .