2
$\begingroup$

Very simple, as the question header says: what is the difference between SVI and VI?

I cannot seem to find a clear-cut definition online.

$\endgroup$
  • $\begingroup$ have you seen this paper? arxiv.org/abs/1206.7051 These guys say: "The coordinate ascent algorithm in Figure 3 is inefficient for large data sets because we must optimize the local variational parameters for each data point before re-estimating the global variational parameters. Stochastic variational inference uses stochastic optimization to fit the global variational parameters. " So instead of getting the gradient from the full dataset, you obtain the natural gradient from batches. $\endgroup$ – user166243 Feb 5 '18 at 16:52
  • $\begingroup$ Admittedly I only skimmed the abstract. But that is perfect. If you post that as the answer, I can accept it. $\endgroup$ – Astrid Feb 5 '18 at 16:56
3
$\begingroup$

Have a look at the paper Stochastic Variational Inference:

The coordinate ascent algorithm in Figure 3 is inefficient for large data sets because we must optimize the local variational parameters for each data point before re-estimating the global variational parameters. Stochastic variational inference uses stochastic optimization to fit the global variational parameters.

So instead of getting the gradient from the full dataset, you obtain the natural gradient from batches.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.