# Do we have guarantees about Adam's convergence when we reach an region with gradient $0$?

$$m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t$$ $$v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2$$ $$\hat{m}_t = \dfrac{m_t}{1 - \beta^t_1}$$ $$\hat{v}_t = \dfrac{v_t}{1 - \beta^t_2}$$ $$\theta_{t+1} = \theta_{t} - \dfrac{\eta}{\sqrt{\hat{v}_t} + \epsilon} \hat{m}_t$$
Suppose for all steps $$t>t^*$$, the gradient $$g_t$$ is $$0$$. Then we have $$\theta_{t^*+k+1} = \theta_{t^*+k} - \eta \cdot \frac{\frac{\beta_1^k m}{1-\beta_1^{k+x}}}{\sqrt\frac{\beta_2^k v}{1-\beta_2^{k+x}}+\epsilon}$$
However, I don't see any guarantee that $$\frac{\frac{\beta_1^k m}{1-\beta_1^{k+x}}}{\sqrt\frac{\beta_2^k v}{1-\beta_2^{k+x}}+\epsilon}$$ must go to zero for large $$k$$. Is there any literature suggesting this? I would expect that in such a regime, Adam should stop making updates of significant magnitude.
In Adam there's epsilon parameter, which tries to prevent that, so you'll always be doing some steps. As we're operating in non-convex spaces, halting optimizer in local optimum wouldn't neceserally be a good idea.