Linked Questions

0
votes
1answer
3k views

Best K in K-fold cross validation [duplicate]

I'm using $k$-fold cross validation technique for generating train, test and validation indexes for a neural network. My sample size is 230~700. What is best $k$ for cross validation here. Now I'm ...
0
votes
0answers
157 views

Understanding stratified K fold cross validation results (for LSTM binary classification model) [duplicate]

I am performing Binary Classification task with LSTM’s. Data_size (205, 100, 4) - Out of 205 samples 110 belongs to class 0 &...
0
votes
0answers
33 views

why is k=10 is commonly used in cross validation? [duplicate]

many blogs say that k=10 is the most commonly number of folds in cross validation. But what is the mathematical explanation behind?
146
votes
5answers
43k views

Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after cross-validation? Put it another way, is it ok to train with all the samples in my dataset and not being able to check if this particular ...
88
votes
7answers
37k views

Bias and variance in leave-one-out vs K-fold cross validation

How do different cross-validation methods compare in terms of model variance and bias? My question is partly motivated by this thread: Optimal number of folds in $K$-fold cross-validation: is leave-...
47
votes
2answers
9k views

Optimal number of folds in $K$-fold cross-validation: is leave-one-out CV always the best choice?

Computing power considerations aside, are there any reasons to believe that increasing the number of folds in cross-validation leads to better model selection/validation (i.e. that the higher the ...
37
votes
3answers
3k views

Variance of $K$-fold cross-validation estimates as $f(K)$: what is the role of “stability”?

TL,DR: It appears that, contrary to oft-repeated advice, leave-one-out cross validation (LOO-CV) -- that is, $K$-fold CV with $K$ (the number of folds) equal to $N$ (the number of training ...
15
votes
2answers
9k views

What is the procedure for “bootstrap validation” (a.k.a. “resampling cross-validation”)?

"Bootstrap validation"/"resampling cross-validation" is new to me, but was discussed by the answer to this question. I gather it involves 2 types of data: the real data and simulated data, where a ...
10
votes
2answers
21k views

How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
5
votes
4answers
3k views

Any theory on how to split the data?

Splitting data for learning and evaluation is a pretty common practice. I've seen people do (train, test, dev) = (50%, 25%, 25%) or (50%, 30%, 20%), etc. Clearly one point is to have enough data for ...
3
votes
3answers
7k views

When to divide data into training & test set in logistic regression?

I am using Logistic Regression in a low event rate situation. Overall universe: 46,000 Events: 420 Conventional logistic regression models divide the data into ...
5
votes
4answers
2k views

Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

I work with R and let's say that I have a train set and a test set. I want to test different algorithms (for example neural networks and svm). I will perform a first 10-folds cross validation on my ...
7
votes
2answers
1k views

How to evaluate/select cross validation method?

How can I decide which cross validation method is appropriate for my problem and data type? For instance, choosing among leave-one-out, or K-fold (and which K is appropriate?). Most of my searches end ...
4
votes
1answer
4k views

Leave-p-out or k-fold cross-validation for small dataset?

I'm using 5-fold cv for parameter optimisation in a regression problem. I have very few samples: around 50. Should I use leave-p-out cv instead? (with, say, p=5) What are the (theoretical, ignoring ...
3
votes
2answers
5k views

What's the difference between Leave-One-Out and K-Fold Cross validation?

As far as I know in K-fold cross validation the samples are split into k sets and at round k-1 of these are used for the training of the model and the last one is used for testing the model and ...

15 30 50 per page