1
$\begingroup$

I have a small dataset of 60 points and used an SVM regression model (with linear kernel) to train a model to predict $\bf{y}$ from two features, $\bf{x_{1}}$ and $\bf{x_{2}}$. I used LOOCV to provide an estimate of the model's performance. In addition to reporting the statistics (e.g. MAE, $r^{2}$) from the LOOCV procedure, I'd like to make a parity plot of $\bf{y_{pred}}$ vs. $\bf{y_{actual}}$. My question is: what are the values of $\bf{y_{pred}}$ that I report? Are they the values obtained from the LOOCV procedure, or are they the predicted values obtained by training the model on all the data points. I imagine, an ideal scenario, the best option would be to have a separate dataset to test the model on, but I do not have the luxury of additional data.

$\endgroup$

1 Answer 1

1
$\begingroup$

 what are the values of ypred that I report? Are they the values obtained from the LOOCV procedure,

yes, because the generalization error (predicting unknown cases) is typically the value of interest.

or are they the predicted values obtained by training the model on all the data points.

These are typically (with high dimensional data) of interest only together with the CV predictions in order to check how much overfitting occured: if these autopredictions are not much better than the CV predictions then you're quite fine in terms of overfitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.