How can I estimate coefficient standard errors when using ridge regression? I am using ridge regression on highly multicollinear data. Using OLS I get large standard errors on the coefficients due to the multicollinearity. I know ridge regression is a way to deal with this problem, but in all the implementations of ridge regression that I've looked at, there are no standard errors reported for the coefficients. I would like some way of estimating how much the ridge regression is helping by seeing how much it is decreasing the standard errors of specific coefficients. Is there some way to estimate them in ridge regression?
 A: I think boostrap would the best option to obtain robust SEs. This was done in some applied work using shrinkage methods, e.g. Analysis of North American Rheumatoid Arthritis Consortium data using a penalized logistic regression approach (BMC Proceedings 2009).
There is also a nice paper from Casella on SE computation with penalized model, Penalized Regression, Standard Errors, and Bayesian Lassos (Bayesian Analysis 2010 5(2)). But they are more concerned with lasso and elasticnet penalization.
I always thought of ridge regression as a way to get better predictions than standard OLS, where the model is generally not parcimonious. For variable selection, the lasso or elasticnet criteria are more appropriate, but then it is difficult to apply a bootstrap procedure (since selected variables would change from one sample to the other, and even in the inner $k$-fold loop used to optimize the $\ell_1$/$\ell_2$ parameters); this is not the case with ridge regression, since you always consider all variables. 
I have no idea about R packages that would give this information. It doesn't seem to be available in the glmnet package (see Friedman's paper in JSS, Regularization Paths for Generalized Linear Models via Coordinate Descent).
However, Jelle Goeman who authored the penalized package discuss this point too. Cannot find the original PDF on the web, so I simply quote his words:

It is a very natural question to ask
  for standard errors of regression
  coefficients or other estimated
  quantities. In principle such standard
  errors can easily be calculated, e.g.
  using the bootstrap.
Still, this
  package deliberately does not provide
  them. The reason for this is that
  standard errors are not very
  meaningful for strongly biased
  estimates such as arise from penalized
  estimation methods. Penalized
  estimation is a procedure that reduces
  the variance of estimators by
  introducing substantial bias. The bias
  of each estimator is therefore a major
  component of its mean squared error,
  whereas its variance may contribute
  only a small part. 
Unfortunately, in
  most applications of penalized
  regression it is impossible to obtain
  a sufficiently precise estimate of the
  bias. Any bootstrap-based cal-
  culations can only give an assessment
  of the variance of the estimates.
  Reliable estimates of the bias are
  only available if reliable unbiased
  estimates are available, which is
  typically not the case in situations
  in which penalized estimates are used.
Reporting a standard error of a
  penalized estimate therefore tells
  only part of the story. It can give a
  mistaken impression of great
  precision, completely ignoring the
  inaccuracy caused by the bias. It is
  certainly a mistake to make confidence
  statements that are only based on an
  assessment of the variance of the
  estimates, such as bootstrap-based
  confidence intervals do.

A: Assuming that the data generating process follows the standard assumptions behind OLS the standard errors for ridge regression is given by:
$ \sigma^2 (A^T A + \Gamma^T \Gamma)^{-1} A^T A (A^T A + \Gamma^T \Gamma)^{-1}$  
The notation above follows the wiki notation for ridge regression. Specifically,
$A$ is the covraiate matrix,
$\sigma^2$ is the error variance.
$\Gamma$ is the Tikhonov matrix chosen suitably in ridge regression.
A: Ridge regression is a subset of Tikhonov regularization (Tk) that normalizes the smoothing factors. The more general regularizing term $\Gamma ^T\Gamma$ is replaced in ridge regression by $\text{$\lambda $I}$, where $\text{I}$ is the identity matrix, and $\lambda $  is a Lagrange (i.e., constraint) multiplier, also commonly called the smoothing, shrinkage, Tikhonov or damping factor. Both Tk and ridge regression are used to solve ill-posed integrals and other inverse problems. "An inverse problem in science is the process of calculating from a set of observations the causal factors that produced them: for example, calculating an image in computer tomography, source reconstructing in acoustics, or calculating the density of the Earth from measurements of its gravity field. here" SPSS contains supplementary code that gives the standard deviation of all the parameters and additional parameters can be derived using error propagation as in the appendix to this paper.
What is generally misunderstood about Tikhonov regularization is that the amount of smoothing has very little to do with fitting the curve, the smoothing factor should be used to minimize the error of the parameters of interest. You would have to explain a lot more about the specific problem you are trying to solve to use ridge regression properly in some valid inverse problem context, and many of the papers on selection of smoothing factors, and many of the published uses of Tikhonov regularization are a bit heuristic.
Moreover Tikhonov regularization is only one inverse problem treatment among many. Follow the link to the journal Inverse Problems.
