# Regularized fit from summarized data

I have a multiple linear regression problem $y=X\beta+\epsilon$. The number of observations $m$ is large, so by the time the data gets to me it's been summarized into:

• $m$
• $X^TX$
• $X^Ty$
• $y^Ty$
• $\sum_{i=1}^m{y_i}$

The above list appears sufficient to compute the OLS estimate. But can it be used to compute a regularized estimate of some sort (e.g. ridge regression)? If not, can the list be augmented in the same spirit to enable regularization?

It would be nice to have in addition the covariance matrix of the residuals $\hat{\Sigma}$ to draw common inferences about the significance of estimated parameters, or if you are sure it is homoscedastic, then just $\hat{\sigma}^2$.
As for the regularizations of generalised least squares type (probably including instrumental variables estimators) the answer will be no. You need the original data matrices (though if you are supplied by $X^T \Omega^{-1}X$ and $X^T\Omega^{-1}y$ you may do GLS, but you loose the control for the choice of $\Omega$ anyway).
Regarding ordinary ridge regression it is sufficient, since all you need to do in this case is just to add elements to the diagonal $X^TX+\delta I$, where $I$ is an identity matrix. Thus in this particular case it works well.
• Thanks a lot for the insights. It looks like ordinary ridge regression is the way to go for my application. I've posted a follow-on question about choosing a good value for $\delta$ (stats.stackexchange.com/questions/9467/…) – NPE Apr 12 '11 at 8:22