Linked Questions

Can anyone tell me what exactly a cross-validation analysis gives as result? Is it just the average accuracy or does it give any model with parameters tuned? Because, I heard somewhere that cross-...

My understanding is that with cross validation and model selection we try to address two things: P1. Estimate the expected loss on the population when training with our sample P2. Measure and report ...

I know that performing hyperparameter tuning outside of cross-validation can lead to biased-high estimates of external validity, because the dataset that you use to measure performance is the same one ...

I'm trying to figure out if my understanding of nested cross-validation is correct, therefore I wrote this toy example to see if I'm right: ...

Suppose I do K-fold cross-validation with K=10 folds. There will be one confusion matrix for each fold. When reporting the results, should I calculate what is the average confusion matrix, or just sum ...

I have read the following posts for nested cross validation and still am not 100% sure what I am to do with model selection with nested cross validation: Nested cross validation for model selection ...

I often see people talking about 5x2 cross-validation as a special case of nested cross validation. I assume the first number (here: 5) refers to the number of folds in the inner loop and the second ...

I have my data stratified in 10 folders. So far I was using 9 of them to train the model, and the remaining one for testing it. A sudden thought just crossed my mind saying "you might be cheating". ...

I'm wondering how to approach properly training and testing a LASSO model using glmnet in R? Specifically, I'm wondering how to do so if a lack of an external test data set necessitates I use cross-...

Suppose we have training data set and a test data set. The outcome variable is binary. Is it usually necessary to split the training data set so that there is a cross validation data set? Or can you ...

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...

I have a basic question on using cross-validation for model parameter tuning (model training) and model evaluation (testing) similar to this Model Tuning and Model Evaluation in Machine Learning I ...

In general, if we have a large dataset, we can split it into (1) training, (2) validation, and (3) test. We use validation to identify the best hyperparameters in cross validation (e.g., C in SVM) and ...

If you have several linear models, say model1, model2 and model3, how would you cross-validate it to pick the best model? (In R) I'm wondering this because my AIC and BIC for each model are not ...

I would like to do model selection using backward stepwise procedure and cross validation. https://www.otexts.org/fpp/5/3 I have used stepAIC in ...