Linked Questions

I have read several threads and articles about the need to use nested cross validation when, for example, choosing between different machine learning algorithms. Here are two describing the use of ...

Can anyone tell me what exactly a cross-validation analysis gives as result? Is it just the average accuracy or does it give any model with parameters tuned? Because, I heard somewhere that cross-...

My understanding is that with cross validation and model selection we try to address two things: P1. Estimate the expected loss on the population when training with our sample P2. Measure and report ...

I know that performing hyperparameter tuning outside of cross-validation can lead to biased-high estimates of external validity, because the dataset that you use to measure performance is the same one ...

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...

I'm trying to figure out if my understanding of nested cross-validation is correct, therefore I wrote this toy example to see if I'm right: ...

Suppose I do K-fold cross-validation with K=10 folds. There will be one confusion matrix for each fold. When reporting the results, should I calculate what is the average confusion matrix, or just sum ...

I often see people talking about 5x2 cross-validation as a special case of nested cross validation. I assume the first number (here: 5) refers to the number of folds in the inner loop and the second ...

I have read the following posts for nested cross validation and still am not 100% sure what I am to do with model selection with nested cross validation: Nested cross validation for model selection ...

I have my data stratified in 10 folders. So far I was using 9 of them to train the model, and the remaining one for testing it. A sudden thought just crossed my mind saying "you might be cheating". ...

I'm wondering how to approach properly training and testing a LASSO model using glmnet in R? Specifically, I'm wondering how to do so if a lack of an external test data set necessitates I use cross-...

I have a basic question on using cross-validation for model parameter tuning (model training) and model evaluation (testing) similar to this Model Tuning and Model Evaluation in Machine Learning I ...

In general, if we have a large dataset, we can split it into (1) training, (2) validation, and (3) test. We use validation to identify the best hyperparameters in cross validation (e.g., C in SVM) and ...

Suppose we have training data set and a test data set. The outcome variable is binary. Is it usually necessary to split the training data set so that there is a cross validation data set? Or can you ...

This might sound like a very simple question, but I haven't been able to find an answer to it, yet: Assuming I am working on a binary classification task and I am using k-fold cross-validation to ...