How to choose smoothing parameter when aiming for estimation? When using a smoothing method for a linear model such as Ridge Regression. It is usual to choose the regularization parameter  $\lambda$ as the one that minimizes the cross validation error. However this criterion is focused on minimizing prediction error for new observations. I understand that the amount of regularization needed for predicting is usually less than for estimating. So, if estimation is the objective, what differents approachs are useful for determining $\lambda$?
I belive a possible answer should be estimating the estimation error (MSE) by bootstrap.  If i had to choose from two different models that is what I would do. But now I have a continium of models. Ofcourse I could boostrap in a finite grid for $\lambda$. But Is this a good approach?
 A: Here is a half-way solution. I do not conclude with a clear answer, but hopefully the thought process can be of some use.

Take a linear model
$$
y=X\beta+\varepsilon,
$$
and assume it is correctly specified (i.e. the model is actually the one that generates the data).
Denote $\hat\beta:=\hat\beta^{ridge}(\lambda)$ the ridge estimate due to penalty intensity $\lambda$ of the parameter vector $\beta$.
The forecast error for a new observation will be
$$
\begin{aligned}
\hat\varepsilon_i &= y_i-X_i\hat\beta \\
&= (X_i\beta+\varepsilon_i)-X_i\hat\beta \\
&= X_i(\beta-\hat\beta)+\varepsilon_i.
\end{aligned}
$$
When aiming to minimize the expected squared forecast error $\mathbb{E}(\hat\varepsilon_i^2)$, we cannot do anything about $\varepsilon_i$ in the above expression, because $\varepsilon_i$ is totally unpredictable (after all, it is a random error). 
Since $\varepsilon$ is also independent of the variables in $X$, the goal 
$$
\mathbb{E}(\hat\varepsilon_i^2) = \mathbb{E}( [X_i(\beta-\hat\beta)+\varepsilon_i]^2 ) \rightarrow\min_{\hat\beta}
$$ 
becomes 
$$
\mathbb{E}([X_i(\beta-\hat\beta)]^2)\rightarrow\min_{\hat\beta}.
$$
That is, we can focus on the part $X_i(\beta-\hat\beta)$ alone. But this is almost the same goal as estimating $\beta$ as accurately as possible in terms of mean squared error:
$$
\mathbb{E}((\beta-\hat\beta)^2)\rightarrow\min_{\hat\beta},
$$
which is what you are interested in.
Clearly, the true $\beta$ is the optimal solution for both the forecasting and the estimation problems, there is no tradeoff there. 

I am not sure how this works empirically, though, when $\lambda$ is selected by cross validation taking the mean squared forecast error as the loss to be minimized in the test sample. And I really do not see why less intensive regularization should be used for forecasting as compared to estimation, as you note in the question.
