Apologise for the clickbaity title, but it is difficult to frame this question in a single sentence. Also, the practicality of variational inference is very clear: intractable posteriors; intractable marginals get the chance to be approximated quickly using this technique.
The distinctions between Bayesian and non-Bayesian techniques I believe have been already discussed here in extensive detail. Still, I believe to some extent that variational inference begs the question - is it really a Bayesian technique?
How I see, a Bayesian technique has three characteristics:
- it has a prior quantity,
- it has a posterior quantity,
- a Bayesian technique maximises posterior probabilities.
From the first two perspective I believe that, variational inference is a Bayesian technique. The third condition is not met in my opinion.
Again, while it is practical, it still feels odd that VI is maximising the lower bound on the evidence. The evidence function in most techniques are simply not utilised, and when it is, then it is used solely for model selection.
Does that make variational inference a new "evidence-based" school, distinct from the "likelihood-based" and "posterior-based" schools?