I am using hierarchical distributions, of the following form:

$\theta\sim N(\mu,\sigma)$

$\mu\sim N(a,b)$

I can calculate the mean, and variance using Mathematica, and find:


$var(\theta)=b^2 + \sigma^2$

However, I would like to know how I could go about doing this with pen and paper. Is there a simpler way than working out the pdf for $\theta$?

Another example, that is similar in nature, that Mathematica is struggling with is:

$\theta\sim N(\mu,\sigma)$

$\mu\sim t-student(\nu,a,b)$

Similarly, does anyone have a methodology for dealing with these type 'distribution of distribution' settings? Additionally could someone tell me the name of this type of probability theory?




The expectation and variance computations in your example can be handled with the law of total expectation and law of total variance.

The law of total expectation in your case reads:

$$ E(\theta) = E_{\mu} ( E_{\theta} (\theta \mid \mu ) ) $$

where the subscripts indicate which variable is being averaged over in the expectation. The inside expectation is

$$ E_{\theta} ( \theta \mid \mu ) = \mu $$

and so the law gives the total expectation as

$$ E(\theta) = E_{\mu} ( \mu ) = a $$

The law of total variance in your case reads

$$ var (\theta) = E_{\mu}( var_{\theta} (\theta \mid \mu ) ) + var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) $$

where, again, the subscripts are telling us what is being averaged over.

We can calculate the first term as

$$ E_{\mu}( var_{\theta} (\theta \mid \mu ) ) = E_{\mu}( \sigma^2 ) = \sigma^2 $$

and the second as

$$ var_{\mu} ( E_{\theta} ( \theta \mid \mu ) ) = var_{\mu} ( \mu ) = b^2 $$

which recovers your result from mathematica.

  • $\begingroup$ Many thanks - I hadn't thought to use those! Best, Ben $\endgroup$
    – ben18785
    Jul 22 '15 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.