I'm starting to learn a bit about gradient descent, and that method attempts to minimize the cost function. I've seen examples using linear regression, and the corresponding cost function, but now I'm wondering, given an arbitrary function $g(x)$ which computes an output $y$, will the cost function to use for gradient decent always be:

$$y_{k} - g(x_{k})$$

resulting in the gradient descent of

$$y_{k+1} = y_{k} - {d \over dx} g(x_{k})$$


1 Answer 1


Commonly used cost functions are:

  • Squared error cost function
  • Negative log likelihood

Regardless of what cost function you use the gradient descent rule is always applicable. The gradient is normally calculated analytically, which depends on the cost function used.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.