34 questions linked to/from Choice of K in K-fold cross-validation
3k views

### Best K in K-fold cross validation [duplicate]

I'm using $k$-fold cross validation technique for generating train, test and validation indexes for a neural network. My sample size is 230~700. What is best $k$ for cross validation here. Now I'm ...
157 views

### Understanding stratified K fold cross validation results (for LSTM binary classification model) [duplicate]

I am performing Binary Classification task with LSTM’s. Data_size (205, 100, 4) - Out of 205 samples 110 belongs to class 0 &...
33 views

### why is k=10 is commonly used in cross validation? [duplicate]

many blogs say that k=10 is the most commonly number of folds in cross validation. But what is the mathematical explanation behind?
43k views

### Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after cross-validation? Put it another way, is it ok to train with all the samples in my dataset and not being able to check if this particular ...
37k views

### Bias and variance in leave-one-out vs K-fold cross validation

How do different cross-validation methods compare in terms of model variance and bias? My question is partly motivated by this thread: Optimal number of folds in $K$-fold cross-validation: is leave-...
9k views

### Optimal number of folds in $K$-fold cross-validation: is leave-one-out CV always the best choice?

Computing power considerations aside, are there any reasons to believe that increasing the number of folds in cross-validation leads to better model selection/validation (i.e. that the higher the ...
3k views

### Variance of $K$-fold cross-validation estimates as $f(K)$: what is the role of “stability”?

TL,DR: It appears that, contrary to oft-repeated advice, leave-one-out cross validation (LOO-CV) -- that is, $K$-fold CV with $K$ (the number of folds) equal to $N$ (the number of training ...
9k views

### What is the procedure for “bootstrap validation” (a.k.a. “resampling cross-validation”)?

"Bootstrap validation"/"resampling cross-validation" is new to me, but was discussed by the answer to this question. I gather it involves 2 types of data: the real data and simulated data, where a ...
21k views

### How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
3k views

### Any theory on how to split the data?

Splitting data for learning and evaluation is a pretty common practice. I've seen people do (train, test, dev) = (50%, 25%, 25%) or (50%, 30%, 20%), etc. Clearly one point is to have enough data for ...
7k views

### When to divide data into training & test set in logistic regression?

I am using Logistic Regression in a low event rate situation. Overall universe: 46,000 Events: 420 Conventional logistic regression models divide the data into ...
2k views

### Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

I work with R and let's say that I have a train set and a test set. I want to test different algorithms (for example neural networks and svm). I will perform a first 10-folds cross validation on my ...
1k views

### How to evaluate/select cross validation method?

How can I decide which cross validation method is appropriate for my problem and data type? For instance, choosing among leave-one-out, or K-fold (and which K is appropriate?). Most of my searches end ...