# Bayesian approach: ignoring the denominator leads to the conditional density equaling the joint density? [duplicate]

I know there are a lot of questions here about ignoring the denominator in a Bayesian approach, but I don't think mine is a duplicate of any of them.

I am reading the book "Pattern recognition and machine learning" by Cristopher Bishop.

Imagine we have a set of N observations of a (single) variable, which we collect in a vector $$\mathbf{x} \in \mathcal{R}^N$$. We would like to find the mean $$\mu$$ of the probbility density function that generated that data, using a Bayesian approach. Thus, we first need to find the posterior probability $$p(\mu|\mathbf{x})$$

We can write:

$$p(\mu|\mathbf{x}) = p(\mathbf{x}|\mu) \cdot \dfrac{p(\mu)}{p(\mathbf{x})}$$

Now as the book says, we can ignore the denominator because it is just a normalizing factor

$$p(\mu|\mathbf{x}) \propto p(\mathbf{x}|\mu) \cdot p(\mu) = p(\mathbf{x}, \mu)$$

where the last equation follows from the product rule, or the defition of conditional density for $$p(\mathbf{x}|\mu)$$ if you want.

So we are approximating a conditional distribution with a joint distribution? How is that even possible?

For one, $$p(\mu|\mathbf{x})$$ whould be a function of $$\mathbf{x}$$, while $$p(\mathbf{x}, \mu)$$ whould be a function of both $$\mathbf{x}$$ and $$\mu$$, right?

First of all, $$p(\mu|\mathbf{x})$$ is not only a function of $$\mathbf{x}$$. It is again a function of both $$\mu$$ and $$\mathbf{x}$$, as joint PDF is. Your question would make sense if we were using $$p(\mathbf{x})$$, i.e. a function of only $$\mathbf{x}$$, instead of the joint, which we don't do of course.
Another thing is, we don't approximate the conditional PDF with joint PDF. The $$\mu$$ that maximizes the joint, also maximizes the conditional. This is just MAP estimation, where we choose $$\mu$$ such that the posterior, i.e. $$p(\mu|\mathbf{x})$$, is maximized (this also means that $$p(\mu|\mathbf{x})$$ is not only a function of $$\mathbf{x}$$, but also $$\mu$$). There, you can ignore the denominator since it doesn't depend on $$\mu$$, and acts as a scalar for the specific optimization problem.
There might be cases where you don't ignore the denominator. For example, conditional mean, i.e. $$E[\mu|\mathbf{x}]$$, sometimes called Bayesian Parameter Estimation in which you generally need to explicitly find $$p(\mu|\mathbf{x})$$, (especially if it's not in a common format) and calculate the conditional mean.
• ok, so the point is that the mode of $p(\mu|x)$ equals the mode of $p(\mu, x)$? – raffaem Apr 27 '19 at 10:57
• If you take $\mathbf{x}$ as constant, i.e. a slice of $p(\mu,\mathbf{x})$, like you do in parameter estimation; yes, they have the same mode. Normally, you wouldn't say that they have the same mode since the latter is a multivariate function, and its mode is of the form $(\mu,\mathbf{x})$, i.e. both variables are changing. – gunes Apr 27 '19 at 11:03
• didn't we establish that $p(\mu|x)$ is also a function of both $\mu$ and $\mathbf{x}$ too? – raffaem Apr 27 '19 at 12:47
• Expression of $p(\mu|\mathbf{x})$ includes both $\mu$ and $\mathbf{x}$, but in this context, $\mathbf{x}$ is assumed to be given, so constant. Its mode is for $\mu$, and equal to $p(\mathbf{x},\mu)$'s mode with the same $\mathbf{x}$. – gunes Apr 27 '19 at 13:02