In SGD algorithms such as Adam you generally make a bad estimate of the gradient of the loss function and take that gradient to move the parameters in the desired direction. Gradient free methods such as SPSA do basically the same, they make a bad estimate of the gradient, by randomly perturbing the parameter vector in two opposite directions. Then they move the parameters along the direction of the gradient.
My question is: Would it possible to combine the gradient calculation from SPSA with the "step taking" from Adam and are there any papers on such an approach?
SPSA only uses a decaying step sequence, and Adam seems superior it this regard.
Edit: The resulting combined algorithm is not supposed to be used to train a ANN but for simulation based optimization.