# How to perform adagrad stochastic gradient descent (SGD) on word2vec?

AdaGrad is an enhanced SGD that automatically determines a per-parameter learning rate.

However, in word2vec, there's no clear "parameter" to perform adagrad on. So what's the closest algorithm to adagrad for word2vec?

AdaGrad maintains a variable $G$ which just accumulates squared norms of the gradients seen so far, e.g. if you try to maximize $\log{p_w}$: