Multiple articles claim that AdaGrad does not work well when the square-root in the formula is not taken. This is one such example.
$\theta_{t+1,i} = \theta_{t,i}-\dfrac{\eta}{\sqrt{G_{t,ii}+\epsilon}}\times g_{t,i}$.
Here $G_{t,ii}$ represents the summation of previous gradients. Why is it so that the square root is so important? If the reason is related to the fact that $G_{t,ii}$ will become very large and hence will prohibit learning, is it not possible to get the same effect using another hyperparameter $\beta$ and using $\beta\times G_{t,ii}$?
Or will a linear scaling down not have the same effect as a non-linear scaling down?