Classical variational inference uses mean field theory because of its computational benefits, i.e. assumes that the latent variables are independent. For gaussian distribution, it wants to find a variational normal distribution with diagonal covariance matrix which is close enough to the standard normal distribution as the prior density.
What is the the form of evidence lower bound (ELBO) for an especial case with dependent latent variables? As we know, in mean field family, ELBO contains two parts, a KL divergence (generally between a normal distribution with diagonal covariance matrix and the standard normal distribution) and the reconstruction error. If we assume that our latent variables have a multivariate normal distribution with a full covariance matrix(that shows they are dependent), how does the model change? (I know what is the formula for KL divergence between normal distributions, but it seems the variational ELBO should involve some additional terms). Thanks beforehand.