I must be missing something in terms of the way the posterior distribution is parameterized when a conjugate (normal) prior is applied to normal data.

From https://en.wikipedia.org/wiki/Conjugate_prior#cite_note-ppredNt-8 I gather that with for normally distributed data with a known variance, the mean is: \begin{align} ((\mu_0 / \sigma^2_0) + (\sum_{i=1}^{n}(x_i) / \sigma^2)) / ((1 / \sigma^2_0) + (n / \sigma^2)) \end{align} and the variance is: \begin{align} ((1 / \sigma^2_0) + (n / \sigma^2)) ^{-1} \end{align} I think I am missing something fundamental because I would expect that the posterior mean and variance could not be smaller than both the prior variance and the variance of the data, but the most simple example suggests that this is not the case:

For the variance, if all values are 1:

(1/1 + 1/1)^-1 = 0.5

Intuitively, as the sample size increases, the posterior variance should approach the known variance sigma squared. Obviously I am misinterpreting something. Help is appreciated!

EDIT: I think it makes sense for the posterior mean, But I am still lost regarding the variance.


It turns out that I was getting confused because I was thinking of the variance as the variance of the population, rather than the variance around the modeled parameter (the mean in this case). It makes sense that the posterior variance around the mean could get lower than the known variance and the prior variance in some cases.

  • $\begingroup$ Specifically, both the prior and the data add information about the parameter, so making the estimate more precise (indeed, the precision -- inverse of variance -- increases additively here) $\endgroup$ – Glen_b -Reinstate Monica Mar 15 '16 at 22:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.