I have a small dataset of 60 points and used an SVM regression model (with linear kernel) to train a model to predict $\bf{y}$ from two features, $\bf{x_{1}}$ and $\bf{x_{2}}$. I used LOOCV to provide an estimate of the model's performance. In addition to reporting the statistics (e.g. MAE, $r^{2}$) from the LOOCV procedure, I'd like to make a parity plot of $\bf{y_{pred}}$ vs. $\bf{y_{actual}}$. My question is: what are the values of $\bf{y_{pred}}$ that I report? Are they the values obtained from the LOOCV procedure, or are they the predicted values obtained by training the model on all the data points. I imagine, an ideal scenario, the best option would be to have a separate dataset to test the model on, but I do not have the luxury of additional data.
1 Answer
what are the values of ypred that I report? Are they the values obtained from the LOOCV procedure,
yes, because the generalization error (predicting unknown cases) is typically the value of interest.
or are they the predicted values obtained by training the model on all the data points.
These are typically (with high dimensional data) of interest only together with the CV predictions in order to check how much overfitting occured: if these autopredictions are not much better than the CV predictions then you're quite fine in terms of overfitting.