In general I would say that is correct.
As we get in the weeds of the math (and depending on what parameterization of Gradient Descent you're using) there are of course differences. One is of course that MLE requires a distribution to be differentiable, and generally you get into problems if there is more than one local minimum or maximum (in statistical terms we measure this by determining if the Hessian Matrix is semi-positive definite, but I won't get into that here).
In general, you could say that many machine learning and statistical modeling methods in some way use a form of the function:
$X$ here is our data, $\beta$ is our parameters or model (in the case of say, a NN this is the whole network). For something like clustering f(Y) may be some combination of within and with-out group variance, while with regression this should look pretty familiar.