What is the theoretical justification for Zellner's g prior for linear regression? I cannot see how it is possible to justify from a purely Bayesian perspective, in which probabilities are epistemic, and a prior distribution represents prior knowledge. Zellner's g prior for the regression coefficients depends on the matrix of covariates X, and in general there is no reason to expect this to provide any information about the regression coefficients -- especially when the data come from an experiment, and the experimenter chose X.
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ The entire regression model is conditional on X so I have no difficulty with the prior also depending on X. Besides this point, the main advantage in using Zellner's prior is one of convenience, in the spirit of conjugate priors, and there is no strong theoretical argument in using it (when compared with, say, Jeffreys' prior). $\endgroup$– Xi'anCommented Jan 5, 2021 at 5:43
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You are corrent that Zellner's g-prior depends on the design matrix in the regression. The prior is often used in "empirical Bayesian" analysis where there is no objection to this, but it is not a "pure" Bayesian procedure. One counter-argument to this is that the design matrix is a conditioning object in regression, and so if we treat this object as a fixed value, then the dependence of the g-prior is no more than for other "objective" Bayesian procedures. Nevertheless, I agree with your view that the prior is not compatible with the "pure" Bayesian approach, which would require the prior to be formulated without regard to the observed data.
As to the theoretical justification, a big part of it is just the mathematical convenience of using a conjugate prior, and the resulting simplicity of the posterior distribution. The g-prior leads to a simple and tractable updating mechanism where the posterior mean is a weighted average of the prior mean and the OLS estimator, with a corresponding diminution of the variance. There is some useful discussion in Marayuma and George (2011) with respect to the reasonableness of this prior, though this discussion is with respect to other criteria, not its independence from the data.
