Stop-gradient operator in vector-quantized variational autoencoder The objective function in VQ-VAE (Eq. (3) here) contains
$$\left\lVert \mathrm{sg}[z_e(x)] - e \right\rVert^2 + \left\lVert z_e(x) - \mathrm{sg}[e] \right\rVert^2,$$
where $\mathrm{sg}$ is the stop-gradient operator.
(Note: The second term can have a weighting factor $\beta$, but "the results did not vary for values of $β$ ranging from $0.1$ to $2.0$. We use $β = 0.25$", so let's assume $\beta=1$.)
What are the advantages of this objective over directly optimizing
$$\left\lVert z_e(x) - e \right\rVert^2$$
instead?
 A: I have been looking for the same question. I have finally deduced the following. I think it is a learning factor that balance the importance between terms (codebook loss and commitment loss).
If the Beta factor is smaller than 1, it means that the encoder is updated more faster than the codebook.
That is interesting for example if we think about it from a centroid perspective (codebook), we do not want them to update strongly in each iteration because we have to preserve some information of the previous batches (and more important if the batch is small).
In short, we want the centroids (codebook) to move slowly and the encoder samples can be updated faster. Probably this technique can minimize the noise produced by the mini-batch sampling in contrast than use all the dataset.
This is what I have deduced, if it is not correct please someone indicate it.
A: If you take gradients, the two formulas you mentioned are the same.
However, if we write in the form of the first formula, fix $z_e$ (zero gradient), let $e$ approach $z_e$ and inversely. 
I think it would be easier for training.
