# What is the optimal gradient descent variation for a smooth function with clear minimum?

I can implement a vanilla gradient descent alogrithm, to minimize some function $f(x,y)$.

Now if I know a priori that this function is smooth with a single global minimum (i.e. no local minima for a gradient descent algorithm to get stuck in) what is the best gradient descent variation to use in terms of speed?

I have found a description of different approaches here, but am unclear which way would be optimal.

Thanks

• the answer clearly depends on the function and what you mean by "smooth." For instance, is a function with continuous first derivative and discontinuous second smooth? – Aksakal Mar 14 '18 at 14:25
• Sure. What I mean is some simple quadratic/bowl function. – user1887919 Mar 14 '18 at 14:35
• If it's quadratic, why not quadratic optimization? foes it have to be gradient descent? Also, look at Newton method, it works best for quadratic functions – Aksakal Mar 14 '18 at 14:43
• It you are minimizing a convex quadratic with no constraints, then the global minimum can be found with a linear equation solve. – Mark L. Stone Mar 14 '18 at 19:35