I find myself in the position of wanting to minimize a numerical function f(a,b) with respect to a and b for which I do not have a the analytical form f(a,b). All I have is f(a,b) for many values of a and b. Is it still possible to do a gradient descent to minimize f with respect to a and b?
$w_{k+1} = w_{k} - \alpha_{k} \nabla f(w) $
with w the vector holding a and b.
Or is numerical gradient descent impossible?
I suspect what you are after is not gradient descent per se, but any form of optimisation algorithm. There are articles on Wikipedia about derivative-free optimisation, that may be of use.
You also need to be able to find new values of $f(a,b)$. If you can't do that, your best guess at optimising $f$ is taking the minimum value for $f$ that you've found so far. (Unless of course the values for $f$ can be clearly and unambiguously extrapolated from the data you already have.)
