# Cross-validation for RMSE

The formula I have for cross-validation error is the following: $$\hat{R}_{CV}(f) = \frac{1}{N}\sum_{i=1}^N L\left( y_i, \hat{f}^{-k(i)}(x_i) \right)$$

where $\hat{f}^{-k(i)}$ is the model trained with $x_i$ excluded from the training set and $L$ is a loss function. Note that $k(i)$ is not necessarily injective e.g for 10-fold cross-validation $k(i)$ maps each training sample to one of the 10 folds $\{1,...,10\}$.

For variance I have the following: $$\hat{\sigma}^2 = \frac{1}{N} \sum_{i=1}^N (L_i - \bar{L})^2$$

where $$L_i = L(y_i, f^{-k(i)}(x_i)) \\ \bar{L} = \frac{1}{N} \sum_{i=1}^N L_i$$

I want to use root mean squared error which is: $$RMSE(f) = \sqrt{\frac{\sum_{i=1}^N (y_i - f(x_i))^2}{N}}$$

My problem is that I am not sure how to adapt these formulas to RMSE. To calculate the variance it seems that I need to be able to decompose my error function to a sum of $L_i$ but I cannot do that because of the square root. How can I calculate the variance if my loss function is RMSE?

• You seem to be using leave-one-out-CV - in which case the loss for the ith fold should be the same as the absolute difference between y_i and the model's prediction. I think your formulas for the variance then just go through as normal? – rje42 May 8 '18 at 15:24
• Well not really, $k(i)$ is not necessarily injective. It maps each training sample to the subset that does not include that particular point. i.e for 10-fold it maps each training sample to {1,..,10} – Andreas G. May 8 '18 at 15:49
• @AndreasG. please make that clear in the text of your question: give a definition of $k(i)$. Then I'll provide an answer to your question. – Jim May 8 '18 at 20:28
• @Jim I've made the change – Andreas G. May 8 '18 at 21:35
• @Jim No they are all of roughly the same size. It's non-injective because you can have $i \neq j$ s.t $k(i)=k(j)$ i.e two different samples mapped to the same set – Andreas G. May 10 '18 at 15:31