Newton's methods vs. Newton's method in optimization In my applied regression course I was given the following equation for the Newton-Raphson's method:
$$b_{n+1}= b_n - \frac{\nabla f(b_n)}{\nabla^2 f(b_n)} $$
Reading further, I found that the Wikipedia article Newton's method defines it as
$$b_{n+1}= b_n - \frac{ f(b_n)}{\nabla f(b_n)} $$
I was quite confused why they would give me a different equation than what seems usual and then found a different article Newton's method in optimization which provides the first formula.
What are the differences between these two? When do I use which?
 A: The question that is being answered in both articles are slightly different, and you're (very understandably) getting confused by the consistent usage of a function $f$ in both articles when they are serving different purposes!
Newton's method is a method to find the root of a function $f$, i.e. the value $x^*$ such that $f(x^*) =0$. That method is given by
$$b_{n+1}= b_n - \frac{ f(b_n)}{f'(b_n)}, $$
where, just in case, I replaced $\nabla f(b_n)$ with $f'(b_n)$ as $\nabla$ is just the vector version of a first derivative to make notation consistent with both articles.
Newton's method in optimization is a method to find the (local) minimum of a twice-differentiable function $g(x)$. A basic fact is that a local minimum is achieved when the derivative of the function is zero, so we want to find the value $\tilde{x}$ where $g'(\tilde{x})=0$, and then the local minimum is given by $g(\tilde{x})$.
How do we find the point where the derivative is zero? We use the Newton's method from above! But now the function $f$ is $g'$ (since we want $g'$, and not $g$, to be zero!), so instead of $f$ and $f'$ used in Newton's method, we plug in $g'$ for $f$ and $g''$ for $f'$.
So it's the exact same thing! In both cases, we are using Newton's method. For the first, it's to find the root of $f$, for the second, given a function $f$ we want to minimize, we need the root of $f'$.
