5
$\begingroup$

I'm studying prior choice, and as far as I understand, when more than one parameter in a distribution is unknown, it is both possible to place one prior for each of the parameters in the likelihood, as well as to place a joint prior over all of said parameters.

For example, for a normal distribution $N(\mu,\sigma^2)$, it is possible to place, say, a normal prior on $\mu$ and another normal prior on $\sigma^2$, but it is also possible to place a bivariate normal distribution as a prior on both $\mu$ and $\sigma^2$.

One of the reasons for using joint priors, I'm guessing, is that we can construct conjugate joint priors for some distributions. However, when not using a conjugate prior, like above, what is the difference between using a joint prior vs separate priors? Is it to encode our belief in the correlation between the parameters? Beyond that, is there any reason to prefer joint priors over separate priors?

$\endgroup$
  • $\begingroup$ Quick note - A normal distribution on $\sigma^2$ isn't conjugate and would be an odd choice. $\endgroup$ – Dason Jul 10 '17 at 13:12
  • $\begingroup$ Thanks for the note! I deliberately avoided inverse-gamma (or normal-inverse-gamma for the joint version) to bring up a situation where conjugate priors aren't used. Of course, we can use conjugate priors for the normal distribution, but I'm also very interested in the case for a distribution where the conjugate prior doesn't exist or is computationally intractable. $\endgroup$ – peco Jul 10 '17 at 18:30
  • $\begingroup$ I believe that copulas can be hard to specify reasonably $\endgroup$ – user795305 Jul 12 '17 at 18:11
9
$\begingroup$

I think the correct way to phrase this is whether the priors are independent or not. The priors can always be described as (for example in your Normal example) $p(\mu, \sigma^2)$, but the question is does that joint prior factorize as $p(\mu, \sigma^2) = p(\mu)p(\sigma^2)$ or not.

Once we have that phrasing in place I think it becomes a little easier to think about it. Are the parameters related in some way? Do changes in one impact another? Then you should consider a prior that includes covariance between the parameters. If not you can consider independent priors. More often than not, independent priors are considered for computational reasons.

$\endgroup$
  • 4
    $\begingroup$ (+1) "More often than not, independent priors are considered for computational reasons.". Actually, I would say that independent priors are usually chosen because it's difficult for users to properly consider joint priors. $\endgroup$ – Cliff AB Jul 10 '17 at 16:10
12
$\begingroup$

All the priors you mention are "joint" priors in that they define a joint distribution on the parameter vector $\mathbf{\theta}=(\theta_1,\ldots,\theta_p)$. When the prior writes down as $$\prod_{i=1}^p \pi_i(\theta_i)$$ each component $\pi_i(\theta_i)$ can also be interpreted as a (marginal) prior on the component $\theta_i$ [provided all components are proper] and the components are independent a priori. Since all priors are acceptable within the Bayesian paradigm, there is no foundational reason to favour independent priors over dependent priors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.