The explanation for the need to compute (rather then optimize) the posterior of latent variables The most common usage of the variational inference looks like to be in computing the marginal distribution $P(X)$ in the denominator of the Bayes formula when computing the posterior probability of the hidden variables, $P(Z|X)$. This is likely a dumb question but I don't understand why we need to compute $P(Z|X)$. Why don't we just optimize $P(Z|X)$ instead (find $Z$ that maximizes $P(Z|X)$), in which case we don't need to exactly compute $P(X)$ since it does not depend on the value of $Z$. In that alternative way, we can just optimize the numerator $P(X|Z)P(Z)$ as in the regular MAP (maximum aposteriori) estimation of the parameters ($\theta$) when we don't have any hidden variables $Z$. What is the difference in those two problems (estimating parameters vs. hidden variables) such that one can go with optimizing through MAP while the other needs sophisticated tools for computing (or approximating) $P(Z|X)$?

Edit: After a little more search, I learned that the MAP is not really full Bayesian, since it does not learn the actual posterior distribution, but it instead gets a point estimate of the parameters through maximizing posterior probability. Then I think what variational inference does is real Bayesian. But still, I am not sure whether I understand the need for getting the distribution itself. Is it to get a better sense of the parameter space?
 A: In many cases getting the distribution is more useful than a point estimation. Say I am an interviewer and I ask each candidate to do a list of questions, then $X$ (observed) could be the number of question the candidate answered correctly, and $Z$ (latent) is whether the candidate is qualified or not.
Candidate A has correctly answered 4 out of 5 questions, and B answered all 5 questions. If I just do a point estimation (MLE/MAP), the most probable value of $Z$ for A and B might be the same $$argmax_ZP(Z|X_A)=argmax_ZP(Z|X_B)=qualified$$
If I can know the full posterior distribution, then I get $$P(Z=qualified|X_A)<P(Z=qualified|X_B)$$

A more common situation would be there's another variable $Y$ that depends on $Z$, if we know the full distribution $p(Z|X)$ we get $$p(Y|X)=\int p(Y|Z)p(Z|X)d_Z$$
which we can't know given only a point estimation of $Z$.
For instance in the setting of recursive filtering algorithms, $Z$ is the state at time $t$, $X$ is the measurement at time $t$, and $Y$ is the state at time $t+1$.

Another example that might be interesting to illustrate the connection is the EM algorithm for Gaussian mixture models, in wich case $X$ is the observed data, $Z$ is the discrete latent variable that denotes which Gaussian component a data point belongs to.  
EM itself is an optimization (point estimation) algorithm for the model parameters $\theta$. In the E step we assign each data point a probability of belonging to each component, which is to compute $p(Z|X)$. in the M step we optimize $\theta$ based on $p(Z|X)$. 
If in the E step we do a point estimation (hard assignment) instead, then we are going from EM for GMM to K-means clustering (assuming the same identity covariance matrix).

I'm not much a statistics people, please correct me if I missed any important point. :]
