This question already has an answer here:

Generally we use the gradient descent to compute the minimum of cost function. My question is why can't we use the maxima minima approach to calculate the cost function. If I am not wrong double derivative of the graph will give us the minima. And we would be able to avoid those gradient steps which are slow to execute. Please feel free If my understanding of the concept is wrong.


marked as duplicate by kjetil b halvorsen, Jeremy Miles, Community May 23 '18 at 11:55

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.


I am not sure if I understand your question correctly, but I will try paraphrasing it and then answering:

Why cannot we minimize a differentiable cost function by computing its derivative and setting it to zero?

The problem is, even if a cost function is differentiable (necessary for gradient descent anyways), it does not mean that the equation $\frac{\partial \mathcal L(w)}{dw}=0$ has an analytical solution. A typical examples are neural networks.

Note that this has already been asked before: Why use gradient descent with neural networks?


Not the answer you're looking for? Browse other questions tagged or ask your own question.