The role of variance of the distribution plays in Bayesian inference

Question

Given prior $ \mu \sim \mathcal{N}(\mu_0, \tau^2) $, likelihood $ X_i | \mu \sim \mathcal{N}(\mu, \sigma^2) $, we know the closed-form solution of posterior is $ \mu | X_1, X_2, \ldots, X_n \sim \mathcal{N}(\mu_n, \tau_n^2) $ with $ \frac{1}{\tau_n^2} = \frac{1}{\tau^2} + \frac{n}{\sigma^2} $ and $ \mu_n = \frac{\frac{1}{\tau^2} \mu_0 + \frac{n}{\sigma^2} \bar{X}}{\frac{1}{\tau^2} + \frac{n}{\sigma^2}} $. It can be observed that given the same number of observations $n$,the larger the $\sigma$ is, the larger the variance in the posterior is. I find it hard to get an intuition of this, because I feel fitting the same number of observations (i.e., MLE) should have the same level of certainty regardless of how spread out the observations are. In other words, I feel it is easy to understand as the number of observations gets large, the uncertainty in posterior diminishes, but the uncertainty should be independent of the sample variance (obviously I am wrong but I want to have the intuition why that is the case).

Answer 1

Think about a simpler problem: estimating the mean $\mu$ in a frequentist setting.

It is sensible to use the MLE, $\bar X$, as our estimator. How accurate is this on average? We know that $\bar X \sim \mathcal{N}(\mu, \sigma^2/n)$ so the mean squared error is $\sigma^2/n$, i.e proportional to the population variance.

This is quite a fundamental point. It is easier to estimate a population mean when the population variance is low, because (on average) your observations will be closer to this mean. For example, if the effect of a medical intervention is very similar for all patients (low $\sigma$), you don't need a large sample to estimate the mean effect accurately.