I am reading Statistical Rethinking (Section 4.3).

When describing how to define a model for height in population the author says:

To complete the model, we're going to need some priors. The parameters to be estimated are both $\mu$ and $\sigma$, so we need a prior $Pr(\mu, \sigma)$, the joint probability for all parameters. In most cases, priors are specified independently for each parameter, which amounts to assuming $Pr(\mu, \sigma) = Pr(\mu)Pr(\sigma)$. Then we can write: $$h_i \sim \text{Normal}(\mu, \sigma) $$ $$\mu \sim \text{Normal}(178, 20) $$ $$\sigma \sim \text{Uniform}(0, 50) $$

Regarding the assumption that lead to the independent specification of priors for each parameter:

  • Are they specified independently because parameters are considered independent each one with regards to the others?
  • In which case should they be described only using the joint probability distribution?
  • Is it possible to always describe the prior of each parameter independently of the others?
  • $\begingroup$ These are choices made by the analyst. Other choices, involving dependence between the [parameter] variables, are equally valid. $\endgroup$ – Xi'an Feb 18 '18 at 9:16

They are specified as independent when you do not want to assume that they are a priori informative about each other. That is, knowing the value of one would not change your mind about any of the others, before seeing any data. If, on the other hand, you thought that e.g. larger means tended to correspond to smaller standard deviation you would specify $p(\mu, \sigma)$ such that $\mu$ and $\sigma$ were negatively correlated.

In that case it would only make sense to talk about $p(\mu, \sigma)$ as the prior over $\mu$ and $\sigma$. Although the marginals $p(\mu)$ and $p(\sigma)$ certainly exist -- just integrate out $\sigma$ from $p(\mu, \sigma)$ to get $p(\mu)$ for example -- the product $p(\mu)p(\sigma)$ isn't the prior. When the two quantities have independent priors, of course, it is, because $p(\mu)p(\sigma) = p(\mu, \sigma)$ by definition.

Hence it is always possible to describe the prior of each parameter separately, but those are only marginals of the actual prior, which you can't recover from them unless it just happens to be a product.

To get your intuition going to begin with, it's helpful to follow the book in thinking of the the prior as a big multivariate thing, which all gets updated in the light of data, rather than the end result of a lot of little priors. In some happy circumstances we might model some of the big thing's component parts as informationally uncoupled, and in still happier circumstances we can think of it decomposing into a product of univariate distributions, but these are special cases.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.