Newton’s Method 🔎

students, in this lesson you will learn one of the most important tools in numerical analysis for finding roots of equations. A root is a value where a function becomes zero, so if $f(x)=0$, then $x$ is a root. Newton’s method is famous because it often reaches a very accurate answer in only a few steps when it works well. It is used in science, engineering, computer graphics, economics, and many other areas where exact algebraic solutions are hard or impossible to find.

What Newton’s Method Tries to Do

Suppose we want to solve $f(x)=0$. In many real problems, the equation is too complicated to solve exactly. Newton’s method starts with a first guess, called $x_0$, and improves it step by step. The basic idea is simple: instead of trying to solve the curve all at once, we zoom in near our current guess and replace the curve with its tangent line. The tangent line is the straight line that just touches the graph at one point and has the same slope there ✏️.

At the point $x_n$, the tangent line to $y=f(x)$ has equation

$$y=f(x_n)+f'(x_n)(x-x_n).$$

To get the next estimate, Newton’s method sets $y=0$ because we want the $x$-intercept of that tangent line. Solving for $x$ gives the iteration formula

$$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}.$$

This is the heart of Newton’s method. The method uses both the function value $f(x_n)$ and the derivative $f'(x_n)$, so it needs the slope at the current guess.

How the Method Works Step by Step

Here is the process students should remember:

Choose a starting value $x_0$.
Compute $f(x_0)$ and $f'(x_0)$.
Use the formula $x_{1}=x_0-\frac{f(x_0)}{f'(x_0)}$.
Repeat to get $x_2, x_3, x_4, \dots$
Stop when the values stop changing much or when $f(x_n)$ is very close to $0$.

This method can be visualized as taking a tangent line, finding where that line crosses the $x$-axis, then moving to that new point and repeating. If the starting guess is good and the function behaves nicely, the guesses can become very close to the root very quickly 🚀.

A key point is that Newton’s method is not just a formula; it is a repeated process, or an iterative method. In numerical analysis, iterative methods are important because they turn a hard problem into a sequence of easier approximations.

A Worked Example

Let’s use the function $f(x)=x^2-2$, whose root is $\sqrt{2}$. We know $\sqrt{2}$ is about $1.41421356$, but imagine we did not know that.

First compute the derivative:

$$f'(x)=2x.$$

Newton’s formula becomes

$$x_{n+1}=x_n-\frac{x_n^2-2}{2x_n}.$$

If we start with $x_0=1$, then

$$x_1=1-\frac{1^2-2}{2(1)}=1-\frac{-1}{2}=1.5.$$

Next,

$$x_2=1.5-\frac{1.5^2-2}{2(1.5)}=1.5-\frac{0.25}{3}=1.416666\dots$$

Then,

$$x_3\approx 1.414215686.$$

Already, after only three steps, the result is extremely close to $\sqrt{2}$. This is one reason Newton’s method is so powerful. When it works well, the accuracy improves very fast.

Why It Works So Well

Newton’s method works because a tangent line is a good local approximation to a smooth curve near the point of tangency. Near the root, the graph of $f(x)$ and the graph of the tangent line are often close enough that the line’s $x$-intercept is a better guess than the current point.

This idea is connected to linearization. The function $f(x)$ is replaced near $x_n$ by the line

$$f(x)\approx f(x_n)+f'(x_n)(x-x_n).$$

That approximation is accurate when $x$ is close to $x_n$ and the function is smooth. So Newton’s method is a bridge between calculus and root-finding: derivative information helps guide the search toward the root.

The method can also be understood geometrically. If the graph is steep, the tangent line may point strongly toward the root. If the graph is flat, progress may be slower or unstable. So the slope matters a lot.

Practical Issues and Common Problems

Even though Newton’s method is famous, it is not perfect. One major issue is the choice of starting value $x_0$. If the initial guess is poor, the method may converge to the wrong root, move far away from the root, or fail to converge at all.

Another issue happens when $f'(x_n)=0$ or is very close to $0$. The formula

$$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}$$

would then be undefined or produce an enormous step. This can happen near flat spots on the graph. For example, if the curve is almost horizontal, the tangent line may cross the $x$-axis far away, which can send the next guess in the wrong direction.

Newton’s method can also behave strangely for functions with repeated roots. If a root has multiplicity greater than $1$, the method may converge more slowly than expected. In ordinary cases with a simple root, convergence is often very fast once the guesses are close enough. But with repeated roots, the speed can drop.

Another practical issue is that computers use finite precision. Rounding can affect the values of $f(x_n)$, $f'(x_n)$, and the next iterate. So numerical analysis always considers not only the math behind a method, but also how the method behaves on real machines 💻.

Stopping Criteria and Accuracy

A method must stop at some point. In practice, we do not usually know the exact root, so we use stopping criteria. Common criteria include:

The change between successive approximations is small:

$$|x_{n+1}-x_n|<\varepsilon.$$

The function value is small:

$$|f(x_n)|<\varepsilon.$$

A maximum number of iterations has been reached.

Here, $\varepsilon$ is a chosen tolerance, such as $10^{-6}$ or $10^{-8}$. A small value of $|f(x_n)|$ suggests that $x_n$ is near a root, but it does not always guarantee the actual root approximation is equally accurate. Likewise, a small $|x_{n+1}-x_n|$ means the guesses are stabilizing, but the function should still be checked when possible.

In practice, numerical analysts often use more than one stopping rule. For example, they may stop when both $|x_{n+1}-x_n|<\varepsilon$ and $|f(x_n)|<\varepsilon$. This makes the result more reliable.

Newton’s Method in the Bigger Picture of Root-Finding II

Newton’s method is one of the central methods in Root-Finding II because it shows how calculus can improve root searches. Compared with simple graphing or trial-and-error, it is much more systematic. Compared with methods that do not use derivatives, Newton’s method is often faster once it is close to the answer.

It also helps explain an important theme in numerical analysis: there is often a trade-off between speed, reliability, and information needed. Newton’s method is fast, but it needs $f'(x)$ and a good initial guess. Other methods, such as the secant method, try to imitate Newton’s idea without requiring an exact derivative. That is why Newton’s method is a foundation for understanding later root-finding techniques.

In real applications, Newton’s method can be used to solve equations in circuit design, optimize shapes, analyze motion, or find equilibrium values. Whenever a problem can be rewritten as $f(x)=0$, this method may be a useful tool.

Conclusion

Newton’s method is a powerful iterative technique for solving $f(x)=0$. It uses the tangent line at a current estimate $x_n$ to produce a better estimate $x_{n+1}$ through the formula

$$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}.$$

Its strength is speed, especially when the starting guess is close to a simple root and the function is smooth. Its limitations include dependence on the derivative, sensitivity to the initial guess, and possible failure when the slope is $0$ or nearly $0$. In Root-Finding II, Newton’s method is essential because it shows how numerical methods use calculus, iteration, and stopping rules to approximate answers that are difficult to find exactly 🔍.

Study Notes

Newton’s method solves $f(x)=0$ by repeated improvement of a guess $x_n$.
The update formula is $x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}$.
The method comes from using the tangent line as a local linear approximation.
It often converges very quickly near a simple root.
A good starting value $x_0$ is important for success.
If $f'(x_n)=0$ or nearly $0$, the method can fail or behave badly.
Common stopping criteria use $|x_{n+1}-x_n|<\varepsilon$, $|f(x_n)|<\varepsilon$, or a maximum number of iterations.
Newton’s method is a key part of Root-Finding II and helps connect calculus with numerical computation.