7
$\begingroup$

When I studied Bayesian statistics, a question about the notation of Bayes' Theorem came to my mind. Below is the density function version of Bayes' Theorem, where $y$ is data vector and $\theta$ is the parameter vector:

$$ p(\theta|y)=\frac{p(y|\theta)p(\theta)}{p(y)} $$ The numerator on the right handside can be written as: $$ p(y,\theta) $$ which is the joint probability distribution of $y$ and $\theta$, then Bayes' theorem could be written as: $$ p(\theta|y)=\frac{p(y,\theta)}{p(y)} $$ Furthermore, $$ p(\theta|y)\propto p(y,\theta) $$ Am I right on this? I think it does not look right. Because the posterior is the proportional to the joint density function. But where is the mistake?

$\endgroup$
  • 2
    $\begingroup$ This is correct. Why do you think it is in error? $\endgroup$ – Dilip Sarwate Sep 10 '12 at 15:44
  • 1
    $\begingroup$ This is correct but i think it is more common to say that the posterior is proportional to the likelihood times the prior which is the same as saying that it is proportional to the joint denstiy of y and θ. $\endgroup$ – Michael R. Chernick Sep 10 '12 at 16:18
  • $\begingroup$ In order to add alternative notations, is it correct to say that $P(y, \theta) = P(y\cap \theta) = p(y| \theta)p(y) = p(\theta|y)p(\theta)$? $\endgroup$ – ecjb Dec 28 '19 at 10:19
12
$\begingroup$

In fact in the notation $$p(\theta|y)\propto p(y,\theta)$$ it is understood that the symbol "$\propto$" means that the two members are proportional functions of the variable $\theta$. This is not ambiguous because it is clearly understood that we are dealing with a distribution on the space of the parameter $\theta$. This notation could become ambiguous when dealing with a two parameters model, say $\theta$ and $\mu$. In such a case I personally use the notation $\underset{\theta}{\propto}$, $\underset{\mu}{\propto}$ or $\underset{\mu,\theta}{\propto}$ for precising what variable is considered in the proportionality statement.

$\endgroup$
1
$\begingroup$

You are right, posterior distribution is proportional to the joint distribution of $y$ and $\theta$. and the proportionality constant is the inverse of marginal distribution (which is a constant value) $p(y)$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.