5
$\begingroup$

Suppose $Y_1,\dots,Y_n\mid\mu,\sigma^2 \sim \text{ iid } N(\mu,\sigma^2)$ and suppose the priors $\mu \mid \sigma^2 \sim N(\mu_0, \sigma^2 / \kappa_0)$ and $1/\sigma^2 \sim \text{gamma}(\nu_0/2, \nu_0 \sigma_0^2 / 2)$ are placed on the unknown parameters. Then \begin{align*} p(\sigma^2 \mid y_1,\dots,y_n) &\propto p(\sigma^2)p(y_1,\dots,y_n\mid\sigma^2) \\ &= p(\sigma^2) \int p(y_1,\dots,y_n \mid \mu,\sigma^2)p(\mu\mid\sigma^2)d\mu \end{align*} It turns out (see Hoff) with these priors that the marginal posterior distribution on $\sigma^2$ is $$1/\sigma^2\mid y_1,\dots,y_n \sim \text{gamma}(\nu_n/2,\nu_n\sigma_n^2/2)$$ where $$\nu_n = \nu_0 + n, \text{ and } \sigma_n^2 = \frac{1}{\nu_n}\left( \nu_0\sigma_0^2 + (n-1)s^2 + \frac{\kappa_0 n}{\kappa_0 + n} ( \bar{y}-\mu_0 )^2 \right)$$ where $\bar{y}$ and $s^2$ are the sample mean and unbiased sample variance. Thus we have that $$ \text{Var}\left[\frac{1}{\sigma^2} \mid y_1,\dots,y_n\right] = (\nu_n/2) / (\nu_n\sigma_n^2/2)^2 = \frac{2}{\nu_n\sigma_n^4}. $$

Now suppose we fix all hyperparameters except $\kappa_0$ and analyze the effect of increasing the variance of $\mu\mid\sigma^2$ (by decreasing $\kappa_0$). Since $f(\kappa_0)=\kappa_0 n / (\kappa_0 + n)$ is monotonically increasing as a function of $\kappa_0$, we see that \begin{align*} \kappa_{0,a} < \kappa_{0,b} &\Rightarrow \kappa_{0,a} n / (\kappa_{0,a} + n) < \kappa_{0,b} n / (\kappa_{0,b} + n) \\ &\Rightarrow \sigma_{n,a}^2 < \sigma_{n,b}^2 \\ &\Rightarrow \sigma_{n,a}^4 < \sigma_{n,b}^4 \\ &\Rightarrow \frac{2}{\nu_n\sigma_{n,b}^4} < \frac{2}{\nu_n\sigma_{n,a}^4}, \end{align*} i.e., increasing the variance of $\mu\mid\sigma^2$ results in an increase in the marginal posterior variance $\text{Var}\left[1/\sigma^2\mid y_1,\dots,y_n\right]$.

Intuitively, this result makes a lot of sense because if $p(\mu \mid \sigma^2)$ is more spread out, the averaging (integral) over $p(\mu \mid \sigma^2)$ above makes $p(\sigma^2 \mid y_1,\dots,y_n)$ more spread out.

This makes me think that this result should be able to be proved in general, i.e., given a sampling model $f_{Y\mid\theta,\phi}$ depending on two unknown parameters $\theta$ and $\phi$, and priors $f_{\theta\mid\phi}$ and $f_{\phi}$, an increase in $\text{Var}\left[ \theta\mid\phi\right]$ should result in an increase in $\text{Var}\left[ \phi \mid y_1,\dots,y_n \right]$. Does anyone know of a general result like this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.