Gelman et al. pg.52 (Bayesian Data Analysis, 3rd Ed) state that

We return to the problem of estimating the mean $\theta$ of a normal model with known variance $\sigma^2$, with a $N(\mu_0,\tau_0^2)$ priot distribution on $\theta$. If the prior precision, $1/\tau_0^2$, is small relative to the data precision, $n/\sigma^2$ , then the posterior distribution is approximately as if $\tau_0^2=\infty$:

$$p(\theta|y)\approx N(\theta|\bar{y},\sigma^2/n).$$

I am not sure I follow this. Here's the reason why.

Assume a

$Likelihood$, ~$N(\theta, 5)$

$Prior$, ~ $N(4, 8)$

$Prior Precision$, $1/\tau_0 ^2$ = $1/8$ = $0.125$

$Data Precision$, $n/\tau ^2$ = $1/5$ = $0.2$ ( for a single data point)

$Posterior Precision$, $1/\tau_{post} ^2$ = $1/\tau_{0} ^2$ + $1/\sigma ^2$ = $1/5$ + $1/8$= $0.3125$

How is this ($1/\tau_{post} ^2$) ~ $\infty$ ? I get some reasonable finite value ? Will be interested in your thoughts on this topic.

  • 3
    $\begingroup$ 1. I believe that you have not correctly quoted your source (there are at least a couple of errors that I believe Gelman would be unable to make). Please carefully quote the passage. 2. Your questions in the line at the end are unclear. I don't understand what you mean, since your setup doesn't correspond to what Gelman is talking about (but it may be due to you misunderstanding what was written). 3. Note also that "interested in your thoughts" is too broad. $\endgroup$ – Glen_b -Reinstate Monica Sep 4 '17 at 23:36
  • 1
    $\begingroup$ ... to add to Glen_b's comment, "Gelman et al." isn't sufficient to identify the source; you should put the name of the book (he's written more than one) and the edition. It's also better to provide an exact quote rather than to paraphrase. $\endgroup$ – jbowman Sep 5 '17 at 0:12
  • $\begingroup$ @Glen_b, I have included the url for the source. I am not saying Gelman is wrong :), You are right, this is a situation where I have misunderstood something. My question is , how can posterior distribution $1/\tau_{post} ^2$ be approximately $\infty$, when it is less than $n/\sigma^2$ $\endgroup$ – Science11 Sep 5 '17 at 0:57
  • 3
    $\begingroup$ You have not correctly quoted the source as I already explained. Words you've replaced or omitted alter the meaning. I have included a quote of the original in your post so that people can see what was said by Gelman et al. $\endgroup$ – Glen_b -Reinstate Monica Sep 5 '17 at 2:12

The (standard) argument in Gelman et al. (2012) is that when you push the hyperparameter $\tau_0$ to infinity, you recover a $N(\theta|\bar{y},\sigma^2/n)$ posterior. There is no mention made of $1/\tau_{post} ^2$ being $\infty$ in the quote reported by Glen_b, since indeed it is $1/\tau_{0} ^2+n/\sigma^2$.


The exact posterior precision in this case is:

$$\text{Posterior Precision} \equiv \frac{1}{\mathbb{V}(\theta|y)} = \frac{1}{\tau_0^2} + \frac{n}{\sigma^2}.$$

So all he is saying is that if $n/\sigma^2 \gg 1/\tau_0^2$ then:

$$\text{Posterior Precision} = \frac{1}{\tau_0^2} + \frac{n}{\sigma^2} \approx \frac{n}{\sigma^2} = \frac{1}{\infty} + \frac{n}{\sigma^2}.$$

Note that he is using the approximation $\tau_0^2 = \infty$, which gives $1/\tau_0^2 =0$, so his approximating result is not infinite. In the example you have used the condition $n/\sigma^2 \gg 1/\tau_0^2$ is not met, so the approximation is poor.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.