0
$\begingroup$

I'm trying to understand why ADAM uses the gradient and the gradient squared respectively as estimators the first and second moments. I was assuming that the random variable we were estimating was the true gradient, but I cannot reason why E[g] = g and E[g^2] = g^2. Can someone help me understand this? Thanks!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.