I find myself in the position of wanting to minimize a numerical function f(a,b) with respect to a and b for which I do not have a the analytical form f(a,b). All I have is f(a,b) for many values of a and b. Is it still possible to do a gradient descent to minimize f with respect to a and b?

$w_{k+1} = w_{k} - \alpha_{k} \nabla f(w) $

with w the vector holding a and b.

Or is numerical gradient descent impossible?

  • $\begingroup$ Can you calculate gradients of $f$ with respect to parameters? If no, then you are missing the key component. $\endgroup$ – Tim Mar 25 at 21:30

I suspect what you are after is not gradient descent per se, but any form of optimisation algorithm. There are articles on Wikipedia about derivative-free optimisation, that may be of use.

You also need to be able to find new values of $f(a,b)$. If you can't do that, your best guess at optimising $f$ is taking the minimum value for $f$ that you've found so far. (Unless of course the values for $f$ can be clearly and unambiguously extrapolated from the data you already have.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.