Why sometimes evidence is said to be too complex to compute and other times is negelected cause it is fixed? When motivating something like variational inference they say the denominator in Bayes rule is too complex to compute. Other times I see that the denominator is neglected and equality is replaced by proportionality. Can you please explain when do we do this or that?
 A: The denominator is useful as marginal likelihood or evidence towards evaluating the fit of the model to the data and possibly comparing different models. When considering a single model, it is not necessary to know this item as the posterior can be defined up to a normalising constant, for simulation purposes. There however exist different manners to exploit the simulation output aiming at the posterior towards computing the evidence through a Monte Carlo approximation, as for instance


*

*harmonic means (not recommended)

*bridge sampling

*path sampling

*reverse logistic regression à la Geyer

*nested sampling (a favourite with astronomers)

A: Variational inference is a method to approximate the posterior, without calculating the evidence term (i.e. the denominator) explicitly. If it were simple to calculate, we would just want to calculate the posterior term exactly, not approximate it. The proportionality argument is used in several places such as MCMC methods, MAP estimation, Bayesian analyses that use conjugate priors etc. to get round of explicitly calculating the integral factor.
A: If you check Bayes` Theorem the denominator is just used as a normalizing constant that ensures that in the continuous case the posterior density integrates to 1.
And it is easy to show that the denominator is just the marginal likelihood of your data. 
So the density of the posterior is thus proportional to the product of the likelihood and the prior density...
