The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping the uses of shrinkage methods like ridge regression, lasso, and least angle regression (LAR). Typically, I'm interested in the parameter estimates themselves and in achieving unbiasedness or at least consistency. Shrinkage methods don't do that.
It seems to me that these methods are used when the statistician is worried that the regression function becomes too responsive to the predictors, that it considers the predictors to be more important (measured by the magnitude of the coefficients) than they actually are. In other words, overfitting.
But, OLS typically provides unbiased and consistent estimates.(footnote) I've always viewed the problem of overfitting not of giving estimates that are too big, but rather confidence intervals that are too small because the selection process isn't taken into account (ESL mentions this latter point).
Unbiased/consistent coefficient estimates lead to unbiased/consistent predictions of the outcome. Shrinkage methods push predictions closer to the mean outcome than OLS would, seemingly leaving information on the table.
To reiterate, I don't see what problem the shrinkage methods are trying to solve. Am I missing something?
Footnote: We need the full column rank condition for identification of the coefficients. The exogeneity/zero conditional mean assumption for the errors and the linear conditional expectation assumption determine the interpretation that we can give to the coefficients, but we get an unbiased or consistent estimate of something even if these assumptions aren't true.