# Risk bounds for Gaussian regression

Does anyone know references for non-asymptotic risk bounds for Gaussian regression. Specifically, I am interested in Bayesian regression with a Gaussian prior on the estimator space, and a Gaussian likelihood. I would like to know any results that bound the difference between the risk of the Bayesian estimator and the risk of the "best" estimator, from among the space of estimators on which the prior is defined. I am new to the field, and the only results I have been able to find by googling are regret bounds for online Bayesian regression by Kakade and Ng. Is this a standard result, if so, can anyone recommend a textbook? Thanks much!

-
How do you define the "risk" of an estimator? By expected squared error loss? –  Xi'an Jan 13 '12 at 7:29
Yes, expected square loss or equivalently in the case of Gaussian regression, the negative logarithm of the prediction probability. I realized that the Maximum A-Posteriori(MAP) of the Bayesian posterior distribution on the estimator space, in the case of Gaussian regression, is the same as ridge regression (up to scaling). So in other words, I am looking for risk bounds for Ridge Regression. After some searching I found this stat.wharton.upenn.edu/~skakade/courses/stat991_mult/lectures/… Can I find a textbook or paper reference for this? Thanks! –  D Jay Jan 13 '12 at 19:52
Gaussian regression or Gaussian mean estimation are about the same from a risk point of view. If you consider (by sufficiency) a Gaussian observation $x\sim\mathcal{N}_p(\theta,I_p)$, the "best [unbiased]" estimator is $x$. Any Bayesian estimator based on a conjugate [normal] prior is of the form $\alpha\theta_0+(1-\alpha)x$. While those estimators are admissible (i.e. cannot be dominated uniformely), they are not minimax and, in fact, the maximum of the risk difference with the risk of $x$ [equal to $p$] is infinity. In addition, in dimension 4 or less, only generalised Bayes estimators (i.e. estimators associated with improper priors) can be minimax. (This was shown by William Strawderman in 1973.)