I am trying to optimize a function $f(x)$ of a vector of reals $x=<x_1, x_2, ...x_n>$. In my practical application, I have no expression for $f$ whatsoever, all I can do is given a vector $x$, calculate $f(x)$ via a deterministic experiment.
How do I go about calculating the gradient? What specific type of gradient descent would you suggest? Or do you suggest an alternative approach?
I was thinking, maybe try each $x_i$, shifting it up or down, see which way $f$ increases, and move in that direction. In other words, change only single $x_i$ at a time. Does this method have a name? Or is it flawed, and there is a better alternative? How do I approach this problem?