Suppose $Y_1,\dots,Y_n\mid\mu,\sigma^2 \sim \text{ iid } N(\mu,\sigma^2)$ and suppose the priors $\mu \mid \sigma^2 \sim N(\mu_0, \sigma^2 / \kappa_0)$ and $1/\sigma^2 \sim \text{gamma}(\nu_0/2, \nu_0 \sigma_0^2 / 2)$ are placed on the unknown parameters. Then \begin{align*} p(\sigma^2 \mid y_1,\dots,y_n) &\propto p(\sigma^2)p(y_1,\dots,y_n\mid\sigma^2) \\ &= p(\sigma^2) \int p(y_1,\dots,y_n \mid \mu,\sigma^2)p(\mu\mid\sigma^2)d\mu \end{align*} It turns out (see Hoff) with these priors that the marginal posterior distribution on $\sigma^2$ is $$1/\sigma^2\mid y_1,\dots,y_n \sim \text{gamma}(\nu_n/2,\nu_n\sigma_n^2/2)$$ where $$\nu_n = \nu_0 + n, \text{ and } \sigma_n^2 = \frac{1}{\nu_n}\left( \nu_0\sigma_0^2 + (n-1)s^2 + \frac{\kappa_0 n}{\kappa_0 + n} ( \bar{y}-\mu_0 )^2 \right)$$ where $\bar{y}$ and $s^2$ are the sample mean and unbiased sample variance. Thus we have that $$ \text{Var}\left[\frac{1}{\sigma^2} \mid y_1,\dots,y_n\right] = (\nu_n/2) / (\nu_n\sigma_n^2/2)^2 = \frac{2}{\nu_n\sigma_n^4}. $$

Now suppose we fix all hyperparameters except $\kappa_0$ and analyze the effect of increasing the variance of $\mu\mid\sigma^2$ (by decreasing $\kappa_0$). Since $f(\kappa_0)=\kappa_0 n / (\kappa_0 + n)$ is monotonically increasing as a function of $\kappa_0$, we see that \begin{align*} \kappa_{0,a} < \kappa_{0,b} &\Rightarrow \kappa_{0,a} n / (\kappa_{0,a} + n) < \kappa_{0,b} n / (\kappa_{0,b} + n) \\ &\Rightarrow \sigma_{n,a}^2 < \sigma_{n,b}^2 \\ &\Rightarrow \sigma_{n,a}^4 < \sigma_{n,b}^4 \\ &\Rightarrow \frac{2}{\nu_n\sigma_{n,b}^4} < \frac{2}{\nu_n\sigma_{n,a}^4}, \end{align*} i.e., increasing the variance of $\mu\mid\sigma^2$ results in an increase in the marginal posterior variance $\text{Var}\left[1/\sigma^2\mid y_1,\dots,y_n\right]$.

Intuitively, this result makes a lot of sense because if $p(\mu \mid \sigma^2)$ is more spread out, the averaging (integral) over $p(\mu \mid \sigma^2)$ above makes $p(\sigma^2 \mid y_1,\dots,y_n)$ more spread out.

This makes me think that this result should be able to be proved in general, i.e., given a sampling model $f_{Y\mid\theta,\phi}$ depending on two unknown parameters $\theta$ and $\phi$, and priors $f_{\theta\mid\phi}$ and $f_{\phi}$, an increase in $\text{Var}\left[ \theta\mid\phi\right]$ should result in an increase in $\text{Var}\left[ \phi \mid y_1,\dots,y_n \right]$. Does anyone know of a general result like this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.