2
$\begingroup$

Simple question here about deriving posteriors.

Suppose I have some likelihood in mind for my data, $p(y|\theta)$, and I also have a particular conjugate prior in mind $p(\theta)$.

Now, I have obtained a single observation, $y_1$, which I use to update my prior into a posterior: $p(\theta|y_1) \propto p(y_1|\theta) p(\theta)$.

Now suppose I have more data points, say three in total $\{y_1, y_2, y_3\}$. I'm now interested in the posterior $p(\theta| y_1, y_2, y_3)$ which might commonly be written as $\prod_{i=1}^3 p(y_i|\theta) p(\theta)$ if each data point were drawn independently.

My question is how to carry out this product - is it just over the likelihood or also the prior? That is, should I think of the posterior as

(i) $p(\theta|y_N) \propto p(\theta) \prod_{i=1}^3 p(y_i|\theta) $

or

(ii) $ p(\theta|y_N) \propto p(\theta)^3 \prod_{i=1}^3 p(y_i|\theta) $ ?

That is, do I multiply by the prior probability 3 times?

Improve this question
$\endgroup$
1
$\begingroup$

It's (i). Think about the typical formula when you think of $y$ as your complete data vector:

$p(\theta|y) \propto p(\theta) p(y|\theta)$ $= p(\theta) \prod\limits_{i=1}^{n} p(y_i|\theta)$

To say it another way, that product is the formula for your likelihood. The formula for your prior doesn't change based on your sample size.

Improve this answer
$\endgroup$
1
  • $\begingroup$ great thanks! I typically see the notation $p(\theta|y) \propto \prod_i p(y_i|\theta) p(\theta)$, so my uncertainty was about the extent of that product operation, but I guess another way to look at it is that the product is indexed by $i$ and so is the data $y_i$, but nothing in the prior is indexed by that product operation. $\endgroup$
    – user66126
    Jan 8 '15 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.