Are Studentized deleted residuals a form of k-fold cross validation when K=N? (this question is asked in the context of the discussion here)
$\begingroup$
$\endgroup$
asked May 7, 2014 at 14:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
A deleted residual is $d_i = Y_i - \hat{Y}_{i(i)}$, which can be shown to be algebraically equivalent to $$ d_i = \dfrac{e_i}{1-h_{ii}}$$ where $e_i$ is the normal residual $Y_i - \hat{Y}_i$ and $h_{ii}$ is the 'hat' matrix $h_{ii} = \mathbf{X}_i^\intercal(\mathbf{X}^\intercal\mathbf{X})^{-1}\mathbf{X}_i$.
Studentizing typically means dividing the residual by its standard error. It can be shown that $$\text{Var}\left[d_i\right] = MSE_{(i)}(1+\mathbf{X}_i^\intercal(\mathbf{X}^\intercal\mathbf{X})^{-1}\mathbf{X}_i) = \dfrac{MSE_{(i)}}{1-h_{ii}}$$
The term you were referring to in the link would be $$ \dfrac{1}{N}\sum_{i=1}^N L(y_i,\hat{y}_{i(i)}) =\dfrac{1}{N}\sum_{i=1}^N d_i^2 $$ if $L$ is squared error loss
So the concepts are related, but perhaps not exactly the same. I should add that a lot of this is under the standard linear model framework.
