Gradient descent will seek to find the nearest point at which the objective function is stationary in the direction of the gradient.
According to wikipedia we might be better off finding stationary points on the likelihood estimation instead. Here is the quote;
...it has been long recognized that requiring even local minimization is too restrictive for some problems of maximum-likelihood estimation.Therefore, contemporary statistical theorists often consider stationary points of the likelihood function(or zeros of its derivative, the score function, and other estimating equations).
Source: Stochastic gradient descent
My understanding of the likelihood function is that it is simply the space of parameter likelihoods based on the input data.
How does optimising to find stationary points on the likelihood function differ from simple gradient descent in practice, and why would it be preferable?