I am using hierarchical distributions, of the following form:

$\theta\sim N(\mu,\sigma)$

$\mu\sim N(a,b)$

I can calculate the mean, and variance using Mathematica, and find:


$var(\theta)=b^2 + \sigma^2$

However, I would like to know how I could go about doing this with pen and paper. Is there a simpler way than working out the pdf for $\theta$?

Another example, that is similar in nature, that Mathematica is struggling with is:

$\theta\sim N(\mu,\sigma)$

$\mu\sim t-student(\nu,a,b)$

Similarly, does anyone have a methodology for dealing with these type 'distribution of distribution' settings? Additionally could someone tell me the name of this type of probability theory?




1 Answer 1


The expectation and variance computations in your example can be handled with the law of total expectation and law of total variance.

The law of total expectation in your case reads:

$$ E(\theta) = E_{\mu} ( E_{\theta} (\theta \mid \mu ) ) $$

where the subscripts indicate which variable is being averaged over in the expectation. The inside expectation is

$$ E_{\theta} ( \theta \mid \mu ) = \mu $$

and so the law gives the total expectation as

$$ E(\theta) = E_{\mu} ( \mu ) = a $$

The law of total variance in your case reads

$$ var (\theta) = E_{\mu}( var_{\theta} (\theta \mid \mu ) ) + var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) $$

where, again, the subscripts are telling us what is being averaged over.

We can calculate the first term as

$$ E_{\mu}( var_{\theta} (\theta \mid \mu ) ) = E_{\mu}( \sigma^2 ) = \sigma^2 $$

and the second as

$$ var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) = var_{\mu} ( \mu ) = b^2 $$

which recovers your result from mathematica.

  • $\begingroup$ Many thanks - I hadn't thought to use those! Best, Ben $\endgroup$
    – ben18785
    Commented Jul 22, 2015 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.