0
$\begingroup$

For example:

If I have a simple 3-layer neural network that demonstrates better performance on the test set when the value of β2 is .95 when compared to the default .999 over several trials of cross-validation, is there any assumption that we can make about properties of the data or gradients?

Is there any literature on this matter?

$\endgroup$
0
$\begingroup$

No you cannot make assumptions on the data because of this. If you want literature on the matter I would suggest reading the original paper by Kingma. You should see the adam method solely as an optimizer. The optimizer often works better in certain classes of data. This does however not imply the converse: so tuning parameters does not necessarily mean that the data has certain characteristics.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.