For a long answer, see Blei, Kucukelbir and McAuliffe here. This short answer draws heavily therefrom.
- MCMC is asymptotically exact; VI is not. In the limit, MCMC will exactly approximate the target distribution. VI comes without warranty.
- MCMC is computationally expensive. In general, VI is faster.
Meaning, when we have computational time to kill and value precision of our estimates, MCMC wins. If we can tolerate sacrificing that for expediency—or we're working with data so large we have to make the tradeoff—VI is a natural choice.
Or, as more eloquently and thoroughly described by the authors mentioned above:
Thus, variational inference is suited to large data sets and scenarios where we want to
quickly explore many models; MCMC is suited to smaller data sets and scenarios where
we happily pay a heavier computational cost for more precise samples. For example, we
might use MCMC in a setting where we spent 20 years collecting a small but expensive data
set, where we are confident that our model is appropriate, and where we require precise
inferences. We might use variational inference when fitting a probabilistic model of text to
one billion text documents and where the inferences will be used to serve search results
to a large population of users. In this scenario, we can use distributed computation and
stochastic optimization to scale and speed up inference, and we can easily explore many
different models of the data.