# K fold cross validation clarification

I don't know if I understood K-fold CV correctly. Here I will explain briefly what I am trying to do. I have a matrix X which I split into 3 folds, 2 training $X_1$, $X_2$ and one testing $X_3$:

Step 1

• I concatenate $X_1$ and $X_2$ so I have $X_{1,2}$, then I generate a $Y_{1,2}$. I compute the $\hat{\beta_{1,2}}$ from the formula: $\hat{\beta_{1,2}}=(X^tX+\lambda I_p)^{-1}X^t Y$ and I compute the predicted $\hat{Y_{1,2}}=X_{1,2}\hat{\beta_{1,2}}$

step 2 I reshuffle. Then I have:

I do the same as in step 1, I compute $X_{1,3}$, $Y_{1,3}$, $\hat{\beta_{1,3}}$ and $\hat{Y_{1,3}}$

Then I reshuffle again and get:

And I compute again: $X_{2,3}$, $Y_{2,3}$, $\hat{\beta_{2,3}}$ and $\hat{Y_{2,3}}$

step 3 Computing the error (This is the part where I'm not quite sure if it is correct) The formula for the square loss is: $L(Y)=(Y-\hat{Y})^2$

Would then these be correct:

$L(Y_1)=(Y_{1,2}-\hat{Y_{2,3}})^2$,

$L(Y_2)=(Y_{1,3}-\hat{Y_{2,3}})^2$,

$L(Y_3)=(Y_{2,3}-\hat{Y_{1,2}})^2$

step 4 Calculating the mean of the above errors $CV = \frac{1}{n}\sum_j^n[L(Y)]$

Let's do a mini example with 6 samples:

Let $X_1$, $X_2$ and $X_3$ be three sets: $$X_1 = \{x_1,x_2\}\\ X_2 = \{x_3,x_4\}\\ X_3 = \{x_5,x_6\}\\$$

Each $x_i$ here is a vector. Judging from your previous question, this will be a 1x15 vector.

You also have: $$Y_1 = \{y_1, y_2\}\\ Y_2 = \{y_3, y_4\}\\ Y_3 = \{y_5, y_6\}\\$$

Those $y_i$ are single values.

Then you create three sets by merging those three folds in different combinations: $$X_{1,2} = X_1 \cup X_2\\ X_{1,3} = X_1 \cup X_3\\ X_{2,3} = X_2 \cup X_3$$

Once you train the three models you have $\beta_{1,2}$, $\beta_{1,3}$ and $\beta_{2,3}$.

Here is the important part. To calculate the predictions you do: $$\hat{Y_1} = X_1\beta_{2,3}$$

Similarly for the other two: $$\hat{Y_2} = X_2\beta_{1,3}\\ \hat{Y_3} = X_3\beta_{1,2}$$

Then your error is: $$L(y_1) = (y_1 - \hat{y_1})^2\\ L(y_2) = (y_2 - \hat{y_2})^2\\ L(y_3) = (y_3 - \hat{y_3})^2\\ L(y_4) = (y_4 - \hat{y_4})^2\\ L(y_5) = (y_5 - \hat{y_5})^2\\ L(y_6) = (y_6 - \hat{y_6})^2\\ CV=\frac{1}{n}\sum_i[L(y_i)]$$

Note that $\hat{y_1}$ and $\hat{y_2}$ are samples from your matrix $\hat{Y_1}$. $\hat{y_3}$ and $\hat{y_4}$ are part of $\hat{Y_2}$. $\hat{y_5}$ and $\hat{y_6}$ is in $\hat{Y_3}$.

I find your notation a bit hard but I think I get it, additionally I think you are slightly off.

Once you have calculated $\hat \beta_{12}$ you want to use this to create predicted values for the values of $Y$ that were not in your training data, Lets call this $\hat Y_{3 \mid 1,2}$ . For clarity that is to say the predicted values of $Y_3$ after fitting a model using data from $X_{12}$ . You then calculate your error as $(Y_3 - \hat Y_{3 \mid 1,2}) ^ 2$. Your last line is then correct that you then repeat this for all your folds of data and take the mean across all your predicted errors.