Why is the normalization necesary in Bayesian inference? [duplicate]

Question

In this post it reads that:

normalization can be intractable when applying Bayes’ Theorem

And in this answer it says that:

it does not depend on the parameters since these have been integrated out.
... is nothing but a normalising constant

And in this scenario the normalising constant is dropped.

But it seems very necessary to work out the denominator Pr(data) of the right hand side of the Bayes' Theorem in Variational Bayes:

$\Pr(\textrm{params} \mid \textrm{data}) = \frac{\Pr(\textrm{data} \mid \textrm{params}) \Pr(\textrm{params})}{\Pr(\textrm{data})}$

To be specific, in this tutorial it provides two answers but I don't get the question. That's I don't understand the problem setup section. How can I relate the left hand side P(Z|X) to what is being approximated P(X)? To be more straightforward, in what problems should I consider the normalization and then need to calculate the lower bound of the Pr(data) and when it is not necessary? And why? Could anyone please exemplify it simply.

Answer 1

I got an answer from the author, and I'd like to document it here(with the agreement of the author):

For your first setup about p(params|data), we should first notice that it is an optimization problem: we want to get the best "parameters", but not necessarily the value of p(params|data). In this case, as p(data) does not depend on the variable "params", we can just omit it during optimization.

However, in inference problem just like the one in my post, we want to infer the posterior probability of the hidden variable, which means we need to calculate the value of this probability. In this case the denominator cannot be omitted because it will lead to inaccurate calculation.

Another example is the section 3 in my post. In that example, the hidden variable "l" is used to calculate the posterior probability of "y" (note that we are optimizing "W", but not "l"). As a result, we need to know the value of the probability of l (even thought we cannot get the exact value, so we use the lower bound instead).

Answer 2

For problems where the goal is to maximize the posterior, the only thing you care about is which values of $\textrm{params}$ make $\Pr(\textrm{params} \mid \textrm{data})$ bigger, since that's the quantity that you're optimizing.

$$\Pr(\textrm{params}_1 \mid \textrm{data}) > \Pr(\textrm{params}_2 \mid \textrm{data})\\ \Longleftrightarrow\\ \frac{\Pr(\textrm{data} \mid \textrm{params}_1) \Pr(\textrm{params}_1)}{\Pr(\textrm{data})} > \frac{\Pr(\textrm{data} \mid \textrm{params}_2) \Pr(\textrm{params}_2)}{\Pr(\textrm{data})}\\ \Longleftrightarrow\\ \Pr(\textrm{data} \mid \textrm{params}_1) \Pr(\textrm{params}_1) > \Pr(\textrm{data} \mid \textrm{params}_2) \Pr(\textrm{params}_2)\\ \Longleftrightarrow\\ \frac{\Pr(\textrm{data} \mid \textrm{params}_1) \Pr(\textrm{params}_1)}{\textrm{whatever}} > \frac{\Pr(\textrm{data} \mid \textrm{params}_2) \Pr(\textrm{params}_2)}{\textrm{whatever}}$$

The denominator cancels because it's identical for all values of $\textrm{params}$, and therefore it doesn't matter what value you plug in for it.