Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\alpha,\beta,\nu,\mu,V$ are the parameters of the posterior Dirichlet distribution and of the multivariate normal-Wishart posterior. The equation above can be evaluated analytically and happens to be a mixture of $t$ distributions (*).
However, I am interested in computing the variance of the posterior pdf at a given point $x$, which boils down to computing:
$$ \left\langle p(x|\alpha,\beta,\nu,\mu,V)^2 \right\rangle. $$
The posterior covariance of a Dirichlet distribution has a simple analytical form (see here), so the only thing I need is the expected square pdf of a normal-Wishart draw. However, I cannot find references for it and the derivation from scratch seems quite involved (not even sure if it would be analytical, but I'd be happy with some approximation); I would rather see if someone has already done it. Any pointer?
The rationale is that I want a principled estimate of the relative reliability of the variational distribution at a given point $x$. I expect the above formulas to yield high relative reliability in the bulk of the distribution, and low relative reliability in the tails and for mixture components with only a few points. I am also happy if you have alternative ideas on how to achieve this.
(*) For those interested, derivation of the posterior predictive pdf for each mixture component can be found for example here, and due to the variationally induced factorization of the Dirichlet posterior and the normal-Wishart posterior, computing the expectation over different components follows easily.
