3
$\begingroup$

I think the terminology is very confusing:

Are the "Hill Climbing" (in AI literature) and "Gradient Descent" (In machine learning literature) the same thing, other than one is maximizing a function and another is minimizing a function?

$\endgroup$
5
$\begingroup$

According to wikipedia they are not the same thing, although there is a similar flavor. Hill climbing refers to making incremental changes to a solution, and accept those changes if they result in an improvement. Note that hill climbing doesn't depend on being able to calculate a gradient at all, and can work on problems with a discrete input space like traveling salesman.

$\endgroup$
3
  • 1
    $\begingroup$ +1, i agree hill climbing that are mostly used for discrete optimization cares, but gradient means the objective function is differentiable (contentious) $\endgroup$
    – Haitao Du
    May 11 '18 at 19:09
  • $\begingroup$ To add to this, the changes made with gradient descent are in the direction of ‘steepest’ improvement relative to the current point, whereas hill climbing accepts changes that make any improvement regardless of slope. $\endgroup$
    – Travis L
    May 12 '18 at 3:03
  • $\begingroup$ My previous comment is incorrect: it is stochastic hill climbing where the move isn’t necessarily the steepest uphill move. $\endgroup$
    – Travis L
    May 15 '18 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.