Solving equations using Newton-Raphson method

Martin McBride
2022-01-15

The Newton Raphson method is a numerical methods to solve equations of the form $f(x) = 0$.

This method uses the first differential of the function.

Example problem

As a simple example, we will solve the equation:

$$x^2 - 2 = 0$$

This equation has a solution $x = \sqrt 2$ (and an second solution $x = -\sqrt 2$). We will only look for the first solution. Although we already know the answer to the problem, it is still useful to work through the numerical solution to see how it works (and to gain an approximate value for the square root of 2).

To use the technique we need to have a rough idea of where the solution is. It is useful to sketch a graph of the function:

The Newton-Raphson method starts with an initial guess at the solution. The guess doesn't need to be particularly accurate, we can just use the value 2.

The Newton-Raphson method proceeds as follows:

  1. Start with an initial guess $x$ (2 in this case).
  2. Draw a tangent to the curve at the point $x$.
  3. Find the value of $x$ where the tangent crosses the x-axis. This will be the next value for $x$.
  4. Repeat step two with the new value.

Steps 2 to 4 are repeated until a sufficiently accurate result is obtained, as described in the solving equations article. On each pass, the interval width is halved, so the approximation becomes more accurate. When the result is accurate enough, the process ends.

A graphical explanation of interval bisection

To gain an intuitive understanding of the process, we will go through a couple of iterations and show the results on a graph. This is for illustration only, you don't need to draw an accurate graph to use this method.

First iteration

The first iteration starts with a guess of 2. Here is the graph where we have zoomed in to the region of interest. a shows the line $x = 2$:

We now draw a line that forms a tangent to the curve at $x = 2$:

This tangent line hits the x-axis at $x = 1.5$. This forms our second approximation.

The formula for calculating the next value of $x$ is shown later in this article,

Second iteration

The second iteration starts with the new value 1.5. On the graph below, we have zoomed in on the range 0.5 to 2.5:

Again, we draw a tangent. Notice that at this stage the curve itself is very close to being a straight line, so the tangent is very close to curve when it hits the x-axis.

The tangent hits the x-axis at $0.4\overline{6}$.

Deriving the formula

This diagram shows a curve $y = f(x)$, with its tangent at point A. We need to calculate the position of point B, where the tangent crosses the x-axis.

If point A has an x-value of $x_0$, then its y-value will be $f(x_0)$ because the point is on the curve.

Since the line AB is a tangent to the curve, we also know that the slope is equal to the slope of the function curve at point A, which is the first derivative of the curve at that point:

$$ slope = f'(x_0) $$

Next we can draw a right-angled triangle ABC where C is on the x-axis directly below A (ie at point $x_0$):

This triangle tells us that the slope of the line AB is:

$$ slope = \frac{AC}{BC} $$

Since the length of AC is $f(x_0)$ and the length of BC is $x_1 - x_0$, we have

$$ slope = \frac{f(x_0)}{x_0 - x_1} $$

We now have two separate expressions for the slope of the line AB, so we can equate them:

$$ \frac{f(x_0)}{x_0 - x_1} = f'(x_0) $$

Inverting both sides gives:

$$ \frac{x_0 - x_1}{f(x_0)} = \frac{1}{f'(x_0)} $$

So:

$$ x_0 - x_1 = \frac{f(x_0)}{f'(x_0)} $$

Rearranging the terms gives the final result for $x_1$:

$$ x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} $$

This is the general formula for solving $f(x)$. The particular formula we are solving in this example is :

$$ f(x) = x^2 - 2 $$

Which has a first derivative:

$$ f'(x) = 2x $$

So our equation is

$$ \begin{align} x_1 = x_0 - \frac{x_0^2 - 2}{2x_0}\newline = x_0 - \frac{x_0^2}{2x_0} - \frac{2}{2x_0}\newline = x_0 - \frac{x_0}{2} - \frac{1}{x_0}\newline = \frac{x_0}{2} - \frac{1}{x_0} \end{align} $$

Newton-Raphson method by calculation

The method for finding the root is very simple:

  1. Start with the initial guess for $x$.
  2. Use the formula to calculate the next value of $x$.
  3. Repeat step two with the new value.

Steps 2 to 3 are repeated until a sufficiently accurate result is obtained, as described in the solving equations article. On each pass, the value of $x$ should typically get closer to the true value.

In this case the formula is:

$x_{next} = \frac{x}{2} - \frac{1}{x}$

Starting with $x = 2$, applying this equation gives:

  • $1.5$
  • $1.4\overline{6}$
  • $1.4142156862745097$
  • $1.4142135623746899$
  • $1.414213562373095$

After 5 iterations, the result is correct to about 15 decimal places!

Advantages and disadvantages of the method

The main advantage of the Newton-Raphson method is that it often converges very quickly.

There are a several disadvantages. The first is that some analysis is required for each formula. For example, it took a few lines of calculation to discover the formula for finding the square root of 2. If we wanted to find the cube root of 2 we would need to do a similar, but slightly different, calculation.

Another slight disadvantage occurs in some cases where there is more than one root. Depending on the initial value of $x$ the method will often converge on the nearest solution. But that isn't always true. In some situations, the method will end up converging on a different root that is further away from the starting value. It isn't always easy to predict which root the method will converge on from a given starting point.

In some cases, attempting to map the start value onto the final root can result in a highly complex, fractal like pattern, called a Newton Fractal. Unfortunately poor old Joseph Raphson doesn't get credited with this fractal, despite the fact that neither he nor Newton discovered it.

Finally, it is sometimes possible for the formula to never converge on a root. This can usually be corrected by choosing a different initial value.