Calculate mean and variance of a distribution of a distribution

Question

I am using hierarchical distributions, of the following form:

$\theta\sim N(\mu,\sigma)$

$\mu\sim N(a,b)$

I can calculate the mean, and variance using Mathematica, and find:

$\mathrm{E}(\theta)=a$

$var(\theta)=b^2 + \sigma^2$

However, I would like to know how I could go about doing this with pen and paper. Is there a simpler way than working out the pdf for $\theta$?

Another example, that is similar in nature, that Mathematica is struggling with is:

$\theta\sim N(\mu,\sigma)$

$\mu\sim t-student(\nu,a,b)$

Similarly, does anyone have a methodology for dealing with these type 'distribution of distribution' settings? Additionally could someone tell me the name of this type of probability theory?

Best,

Ben

Answer 1

The expectation and variance computations in your example can be handled with the law of total expectation and law of total variance.

The law of total expectation in your case reads:

$$ E(\theta) = E_{\mu} ( E_{\theta} (\theta \mid \mu ) ) $$

where the subscripts indicate which variable is being averaged over in the expectation. The inside expectation is

$$ E_{\theta} ( \theta \mid \mu ) = \mu $$

and so the law gives the total expectation as

$$ E(\theta) = E_{\mu} ( \mu ) = a $$

The law of total variance in your case reads

$$ var (\theta) = E_{\mu}( var_{\theta} (\theta \mid \mu ) ) + var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) $$

where, again, the subscripts are telling us what is being averaged over.

We can calculate the first term as

$$ E_{\mu}( var_{\theta} (\theta \mid \mu ) ) = E_{\mu}( \sigma^2 ) = \sigma^2 $$

and the second as

$$ var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) = var_{\mu} ( \mu ) = b^2 $$

which recovers your result from mathematica.