Very simple, as the question header says: what is the difference between SVI and VI?
I cannot seem to find a clear-cut definition online.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Have a look at the paper Stochastic Variational Inference:
The coordinate ascent algorithm in Figure 3 is inefficient for large data sets because we must optimize the local variational parameters for each data point before re-estimating the global variational parameters. Stochastic variational inference uses stochastic optimization to fit the global variational parameters.
So instead of getting the gradient from the full dataset, you obtain the natural gradient from batches.