Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
This question already has an answer here:
I learned that Bayes theorem was defined as follows :
$$p(\theta\mid y)=\frac{p(y\mid\theta)p(\theta)}{p(y)}$$
But then today I came across definition with likelihood:
$$p(\theta\mid y)=\frac{L(\theta\mid y)p(\theta)}{p(y)} = \frac{L(\theta\mid y) p(\theta)}{\int L(\theta\mid y) p(\theta) d\theta }$$
What is the link between the two?
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
$L(\theta|y) = p(y|\theta)$. I assume that $y$ is the observation here, and we are inferring the value of the parameter $\theta$, Thus, $p(y|\theta)$ can be viewed as a function $L$ over the (unknown) variables/parameters $\theta$.
For the denominator, $p(y) = \int p(y,\theta)d\theta = \int p(y|\theta)p(\theta)d\theta = \int L(\theta|y)p(\theta)d\theta$.
The second formula is wrong: the outside parts are equal to each other, but the middle part is merely proportional to (and not necessarily equal to) the outside parts. The likelihood is defined by $L(\theta \mid y) = k(y) p(y \mid \theta) \propto p(y \mid \theta)$ where $k$ is some constant-of-proportionality that does not depend on $\theta$. This means you have:
$$p(\theta \mid y) = \frac{p(y \mid \theta) p(\theta)}{p(y)} = \frac{k(y) L(\theta \mid y) p(\theta)}{p(y)} \propto \frac{L(\theta \mid y) p(\theta)}{p(y)}.$$
Using the law of total probability you also have $p(y) = \int p(y \mid \theta) p(\theta) d\theta$ which gives:
$$p(\theta \mid y) = \frac{p(y \mid \theta) p(\theta)}{p(y)} = \frac{k(y) L(\theta \mid y) p(\theta)}{k(y) \int L(\theta \mid y) p(\theta) d\theta} = \frac{L(\theta \mid y) p(\theta)}{\int L(\theta \mid y) p(\theta) d\theta}.$$
In the special case where $k(y) = 1$ you have $L(\theta \mid y) = p(y \mid \theta)$ and so in this case you get the second equation you specified. However, it is common when using likelihood functions to use a constant-of-proportionality that effectively removes multiplicative terms that do not depend on $\theta$.
