PRESS statistic for ridge regression In ordinary least squares, regressing a target vector $y$ against a set of predictors $X$, the hat matrix is computed as
$$H = X (X^tX)^{-1} X^t$$
and the PRESS (predicted residual sum of squares) is calculated by
$$SS_P = \sum_i \left(  \frac{e_i}{1-h_{ii}}\right)^2$$
where $e_i$ is the $i$th residual and the $h_{ii}$ are the diagonal elements of the hat matrix.
In ridge regression with penalty coefficient $\lambda$, the hat matrix is modified to be
$$H = X (X^t X + \lambda I)^{-1} X^t$$
Can the PRESS statistic be calculated in the same way, using the modified hat matrix?
 A: yes, I use this method a lot for kernel ridge regression, and it is a good way of selecting the ridge parameter (see e.g. this paper [doi,preprint]).
A search for the optimal ridge parameter can be made very efficient if the computations a performed in canonical form (see e.g. this paper), where the model is re-parametersied so that the inverse of a diagonal matrix is required.
A: The following approach can be taken to apply L2 regularisation and get the PRESS statistic.
The method uses a data augmentation approach.
Assume you have N samples of Y, and K explanatory variables X1,X2...Xk....XK


*

*Add additional variable X0 that has 1 over the N samples

*Augment with K additional samples where: 


*

*Y value is 0 for each of the K samples

*X0 value is 0 for each of the K samples

*Xk value is SQRT(Lambda * N) * [STDEV(Xk) over N samples] if on diagonal, and 0 otherwise


*There are now N+K samples and K+1 variables. A normal linear regression can be solved with these inputs.

*As this a regression done in one step the PRESS statistic can be calculated as normal.

*The Lambda regularisation input has to be decided. Reviewing the PRESS statistic for different inputs of Lambada can help determine a suitable value.    

