0
$\begingroup$

I don't really understand why gradient descent is so important in neural networks? Wouldn't it be much easier to define an objective function in a way to do 0=parameters of objective function ? This way you would get the most perfect parameters out imidiately.

$\endgroup$
5
$\begingroup$

Most of the problems do not have exact analytical solutions. Let aside neural networks, even logistic regression (with MSE or cross-entropy) cannot be solved analytically. Logistic regression is a simple neural network, so solving exactly for neural nets in general is impossible when you're not even be able to solve the simplest of them. Even in the presence of an exact analytical solution (commenting for models other than neural nets), sometimes the way it is calculated exactly is not numerically stable or it is computationally more expensive and we may resort to iterative methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.