Consider the mean estimator $$\hat{\mu}(\lambda) = \lambda \frac{1}{n}\sum_{i = 1}^nY_i $$ (for $n$ iid Gaussian variates $Y_i$). After calculating the bias and the variance of this estimator, I found the optimal value which minimizes the prediction error $$ \text{Err}_\lambda = E(\mu - \hat{\mu}(\lambda) )^2= (\lambda-1)^2\mu^2+\lambda^2 \frac{\sigma^2}{n}$$ to be $$ \lambda^*=\frac{\mu^2}{\frac{\sigma^2}{n}+\mu^2}.$$ Can I find this parameter by cross validation (k fold) and if so, how can I do that? For example, if I simulate a Gaussian with $\mu=0.1$, $\sigma = 0.2$, and $n = 30$, I should find $\lambda^*= 0.88$ with cross validation.

  • $\begingroup$ An interesting aspect of this approach is that in finding $\lambda^*$ based on the data, it becomes a function of the data themselves--a statistic--rather than a constant, whence $\text{Err}_\lambda$ no longer has the same form as your formula, implying $\lambda^*$ is no longer necessarily optimal! $\endgroup$ – whuber Nov 2 '11 at 19:19
  • $\begingroup$ i am agree but method like ridge or lasso for regression to ? $\endgroup$ – user7114 Nov 2 '11 at 20:05
  • $\begingroup$ moreover if i consider i have an unbiased estimator of both $\mu$ and $\sigma$, i have an unbiased estimator of the error and the optimal parameter ? $\endgroup$ – grant Nov 3 '11 at 16:10
  • $\begingroup$ Whuber do you have any idea about ? thanks $\endgroup$ – grant Nov 7 '11 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.