34 questions linked to/from Choice of K in K-fold cross-validation
3k views

Best K in K-fold cross validation [duplicate]

I'm using $k$-fold cross validation technique for generating train, test and validation indexes for a neural network. My sample size is 230~700. What is best $k$ for cross validation here. Now I'm ...
157 views

Understanding stratified K fold cross validation results (for LSTM binary classification model) [duplicate]

I am performing Binary Classification task with LSTM’s. Data_size (205, 100, 4) - Out of 205 samples 110 belongs to class 0 &...
33 views

why is k=10 is commonly used in cross validation? [duplicate]

many blogs say that k=10 is the most commonly number of folds in cross validation. But what is the mathematical explanation behind?
43k views

Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after cross-validation? Put it another way, is it ok to train with all the samples in my dataset and not being able to check if this particular ...
37k views

Bias and variance in leave-one-out vs K-fold cross validation

How do different cross-validation methods compare in terms of model variance and bias? My question is partly motivated by this thread: Optimal number of folds in $K$-fold cross-validation: is leave-...
9k views

Optimal number of folds in $K$-fold cross-validation: is leave-one-out CV always the best choice?

Computing power considerations aside, are there any reasons to believe that increasing the number of folds in cross-validation leads to better model selection/validation (i.e. that the higher the ...
3k views

Variance of $K$-fold cross-validation estimates as $f(K)$: what is the role of “stability”?

TL,DR: It appears that, contrary to oft-repeated advice, leave-one-out cross validation (LOO-CV) -- that is, $K$-fold CV with $K$ (the number of folds) equal to $N$ (the number of training ...
9k views

What is the procedure for “bootstrap validation” (a.k.a. “resampling cross-validation”)?

"Bootstrap validation"/"resampling cross-validation" is new to me, but was discussed by the answer to this question. I gather it involves 2 types of data: the real data and simulated data, where a ...
21k views

How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
3k views

Any theory on how to split the data?

Splitting data for learning and evaluation is a pretty common practice. I've seen people do (train, test, dev) = (50%, 25%, 25%) or (50%, 30%, 20%), etc. Clearly one point is to have enough data for ...
7k views

When to divide data into training & test set in logistic regression?

I am using Logistic Regression in a low event rate situation. Overall universe: 46,000 Events: 420 Conventional logistic regression models divide the data into ...
2k views

Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

I work with R and let's say that I have a train set and a test set. I want to test different algorithms (for example neural networks and svm). I will perform a first 10-folds cross validation on my ...
1k views

How to evaluate/select cross validation method?

How can I decide which cross validation method is appropriate for my problem and data type? For instance, choosing among leave-one-out, or K-fold (and which K is appropriate?). Most of my searches end ...