1
$\begingroup$

Given prior $ \mu \sim \mathcal{N}(\mu_0, \tau^2) $, likelihood $ X_i | \mu \sim \mathcal{N}(\mu, \sigma^2) $, we know the closed-form solution of posterior is $ \mu | X_1, X_2, \ldots, X_n \sim \mathcal{N}(\mu_n, \tau_n^2) $ with $ \frac{1}{\tau_n^2} = \frac{1}{\tau^2} + \frac{n}{\sigma^2} $ and $ \mu_n = \frac{\frac{1}{\tau^2} \mu_0 + \frac{n}{\sigma^2} \bar{X}}{\frac{1}{\tau^2} + \frac{n}{\sigma^2}} $. It can be observed that given the same number of observations $n$,the larger the $\sigma$ is, the larger the variance in the posterior is. I find it hard to get an intuition of this, because I feel fitting the same number of observations (i.e., MLE) should have the same level of certainty regardless of how spread out the observations are. In other words, I feel it is easy to understand as the number of observations gets large, the uncertainty in posterior diminishes, but the uncertainty should be independent of the sample variance (obviously I am wrong but I want to have the intuition why that is the case).

$\endgroup$

1 Answer 1

1
$\begingroup$

Think about a simpler problem: estimating the mean $\mu$ in a frequentist setting.

It is sensible to use the MLE, $\bar X$, as our estimator. How accurate is this on average? We know that $\bar X \sim \mathcal{N}(\mu, \sigma^2/n)$ so the mean squared error is $\sigma^2/n$, i.e proportional to the population variance.

This is quite a fundamental point. It is easier to estimate a population mean when the population variance is low, because (on average) your observations will be closer to this mean. For example, if the effect of a medical intervention is very similar for all patients (low $\sigma$), you don't need a large sample to estimate the mean effect accurately.

$\endgroup$
1
  • $\begingroup$ Hey thanks for the explanation. I agree the intuition for the sample variance on the sample mean is clearly explained by your illustration, but mean is just one specific case of the parameter. For example, it is still not clear how will the error on the sample variance estimator change w.r.t. the true variance. $\endgroup$
    – Sam
    Sep 14, 2023 at 0:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.