1
$\begingroup$

I am reading Cross Validation with Confidence by Jing Lei. In section two he introduces cross-validation and makes a law of large numbers argument that CV gives you a good estimate.

enter image description here

enter image description here

Essentially, for a hold-out data point $(X, Y)$, we consider the loss $\ell(\hat f(X), Y)$ conditional on our fit model $\hat f$. The population equivalent is $\mathbb{E} \left[ \ell(\hat f(X), Y) \vert \hat f \right]$, which is what we would like to estimate.

Now, if we have multiple held-out data points, say $(X_{n+1}, Y_{n+1}), (X_{n+2}, Y_{n+2}), ...$, and we get corresponding estimates of out-of-sample loss $\ell(\hat f(X_{n+1}), Y_{n+1}), \ell(\hat f(X_{n+2}), Y_{n+2}), ...$, we can make an LLN argument that their average converges to our estimand provided $\ell(\hat f(X_{n+1}), Y_{n+1})$ and $\ell(\hat f(X_{n+2}), Y_{n+2})$, etc, are independent.

I see how $\epsilon \perp X$ is enough for independence of out of sample losses, but not how $\mathbb{E}( \epsilon | X) = 0$ (the condition in the text) is sufficient for this independence.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.