3
$\begingroup$

This question already has an answer here:

I learned that Bayes theorem was defined as follows :

$$p(\theta\mid y)=\frac{p(y\mid\theta)p(\theta)}{p(y)}$$

But then today I came across definition with likelihood:

$$p(\theta\mid y)=\frac{L(\theta\mid y)p(\theta)}{p(y)} = \frac{L(\theta\mid y) p(\theta)}{\int L(\theta\mid y) p(\theta) d\theta }$$

What is the link between the two?

$\endgroup$

marked as duplicate by Xi'an bayesian Aug 27 '18 at 19:55

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2
$\begingroup$

$L(\theta|y) = p(y|\theta)$. I assume that $y$ is the observation here, and we are inferring the value of the parameter $\theta$, Thus, $p(y|\theta)$ can be viewed as a function $L$ over the (unknown) variables/parameters $\theta$.

For the denominator, $p(y) = \int p(y,\theta)d\theta = \int p(y|\theta)p(\theta)d\theta = \int L(\theta|y)p(\theta)d\theta$.

$\endgroup$
1
$\begingroup$

The second formula is wrong: the outside parts are equal to each other, but the middle part is merely proportional to (and not necessarily equal to) the outside parts. The likelihood is defined by $L(\theta \mid y) = k(y) p(y \mid \theta) \propto p(y \mid \theta)$ where $k$ is some constant-of-proportionality that does not depend on $\theta$. This means you have:

$$p(\theta \mid y) = \frac{p(y \mid \theta) p(\theta)}{p(y)} = \frac{k(y) L(\theta \mid y) p(\theta)}{p(y)} \propto \frac{L(\theta \mid y) p(\theta)}{p(y)}.$$

Using the law of total probability you also have $p(y) = \int p(y \mid \theta) p(\theta) d\theta$ which gives:

$$p(\theta \mid y) = \frac{p(y \mid \theta) p(\theta)}{p(y)} = \frac{k(y) L(\theta \mid y) p(\theta)}{k(y) \int L(\theta \mid y) p(\theta) d\theta} = \frac{L(\theta \mid y) p(\theta)}{\int L(\theta \mid y) p(\theta) d\theta}.$$

In the special case where $k(y) = 1$ you have $L(\theta \mid y) = p(y \mid \theta)$ and so in this case you get the second equation you specified. However, it is common when using likelihood functions to use a constant-of-proportionality that effectively removes multiplicative terms that do not depend on $\theta$.

$\endgroup$
  • $\begingroup$ It is wrong if they define it as unnormalized, but I've seen it defined as $L(\theta | y) = p(y | \theta)$ in some texts... $\endgroup$ – Tim Aug 27 '18 at 9:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.