According to Bayes' theorem, $P(y|\theta)P(\theta) = P(\theta|y)P(y)$. But according to my econometric text, it says that $P(\theta|y) \propto P(y|\theta)P(\theta)$. Why is it like this? I don't get why $P(y)$ is ignored.

  • 1
    Notice that it does not say that the two are equal, but proportional (up to a factor, that is, $1/P(y)$) – jpmuc Jul 15 '13 at 13:16
  • 4
    $P(y)$ is not being ignored but treated as a constant because it is a function of the data $y$ which are fixed for the problem at hand. If $A(x) = cB(x)$ where $c$ is a constant (meaning not dependent on $x$), then we can write $A(x) \propto B(x)$ which simply means that $\frac{A(x)}{B(x)}$ is a (unspecified) constant. Note that the extrema of $A(x)$ and $B(x)$ occur at the same locations so that things like maximum a posteriori probability (MAP or MAPP) estimates can be found from $P(y\mid\theta)P(\theta)$ without the need to know (or compute) $P(y)$. – Dilip Sarwate Jul 15 '13 at 13:24

$Pr(y)$, the marginal probability of $y$, is not "ignored." It is simply constant. Dividing by $Pr(y)$ has the effect of "rescaling" the $Pr(y|\theta)P(\theta)$ computations to be measured as proper probabilities, i.e. on a $[0,1]$ interval. Without this scaling, they are still perfectly valid relative measures, but are not restricted to the $[0,1]$ interval.

$Pr(y)$ is often "left out" because $Pr(y)=\int Pr(y|\theta)Pr(\theta)d\theta$ is often difficult to evaluate, and it is usually convenient enough to indirectly perform the integration via simulation.

Notice that

$$ P(\theta | y) = \frac{P(\theta, y)}{P(y)} = \frac{P(y | \theta) P(\theta)}{P(y)}. $$

Since you're interested in calculating the density of $\theta$, any function that does not depend on this parameter ― such as $P(y)$ ― can be discarded. This gives you

$$ P(\theta | y) \propto P(y | \theta) P(\theta). $$

The consequence of discarding $P(y)$ is that now the density $P(\theta | y)$ has lost some properties like integration to 1 over the domain of $\theta$. This is not a big deal since one is usually not interested in integrating likelihood functions, but in maximizing them. And when you're maximizing a function, multiplying this function by some constant (remember that, in the Bayesian approach, the data $y$ is fixed), doesn't change the $\theta$ that corresponds to the maximum point. It does change the value of the maximum likelihood, but then again, one is usually interested in the relative positioning of each $\theta$.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.