4
$\begingroup$

Consider a Bayesian linear regression model $$\mathbf{Y=X\beta} + \boldsymbol{\varepsilon}$$ where $\mathbf{Y} \in \mathbb{R}^n$ and $\mathbf{X} \in \mathbb{R}^{n,p}$ are given, $\boldsymbol{\varepsilon} \sim\mathcal{N}(0,\sigma^2 \mathbf{I})$ is a Gaussian noise term and $\mathbf{\beta} \in \mathbb{R}^p$ is a parameter vector with a Gaussian prior $$\mathbf{\beta} \sim \mathcal{N}(0,\alpha \mathbf{V}).$$

If you choose $\mathbf{V} = (\mathbf{X}^T\mathbf{X})^{-1}$, you end up with Zellner's prior which has many merits (in particular, the marginal likelihood is available in a simple form).

If you choose $\mathbf{V} = \mathbf{I}$, you end up with Bayesian ridge regression which is usually less popular but has some perks as well (for example, it is suitable for $p>n$ cases, contrarily to Zellner's prior).

I found an extensive literature on the problem of choosing the remaining hyper-parameter $\alpha$ for Zellner's prior. For example, in Fernández (2001) or Liang (2008), they propose a lot of different strategies for choosing it. For example, the unit information prior corresponds to $\alpha = n \sigma^2$ or the risk inflation criterion corresponds to $\alpha = p^2 \sigma^2$.

However, I didn't come across any classical strategies for the ridge prior. Do you know of any strategy for hyperparameter choice is this case ?

References

Fernández, C., Ley, E., and Steel, M. F. (2001), “Benchmark Priors for Bayesian Model Averaging,” Journal of Econometrics, 100, 381–427.

Liang, F.; Paulo, R.; Molina, G.; Clyde, M. A.; Berger, J. O. (2008). "Mixtures of g priors for Bayesian variable selection". Journal of the American Statistical Association. 103 (481): 410–423.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.