Linked Questions

3
votes
5answers
628 views

Cross-validation: Which classifier to use in the end? [duplicate]

This might sound like a very simple question, but I haven't been able to find an answer to it, yet: Assuming I am working on a binary classification task and I am using k-fold cross-validation to ...
0
votes
1answer
427 views

Which fold will be chosen after applying k-fold cross validation? [duplicate]

I think I barely understand how k-fold CV is working and I have a question about it. For example, when I run 5-fold cross-validation (in SAS), I use 5 subsets of the training data, and end up with 5 ...
2
votes
0answers
396 views

Validation set used for early-stopping. Do I retrain the model using the validation data as training data? [duplicate]

I've read posts mentioning that after selecting your model with the validation set you train with the full set (training + validation set) and test the model on the test set. However, what if the ...
-2
votes
1answer
223 views

Model selection method with k-fold cross validation [duplicate]

Can you please provide method to get final best model from cross validation.k-fold cross validation we have k models and accuracy estimate by average of k models accuracy.I need to know about how we ...
0
votes
0answers
31 views

How does k-fold validation affect the model coefficients? [duplicate]

My question on k-fold is not about evaluations of the model, but rather the coefficients that are returned by it. In the R code below, I am performing 3-fold cross validation, which is to say the ...
142
votes
5answers
42k views

Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after cross-validation? Put it another way, is it ok to train with all the samples in my dataset and not being able to check if this particular ...
77
votes
6answers
30k views

Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
7
votes
2answers
9k views

How to evaluate the final model after k-fold cross-validation

As this question and its answer pointed out, k-fold cross validation (CV) is used for model selection, e.g. choosing between linear regression and neural network. It's also suggested that after ...
23
votes
2answers
2k views

Should final (production ready) model be trained on complete data or just on training set?

Suppose I trained several models on training set, choose best one using cross validation set and measured performance on test set. So now I have one final best model. Should I retrain it on my all ...
5
votes
2answers
9k views

Final Model Prediction using K-Fold Cross-Validation and Machine Learning Methods

Similar threads: Feature selection for "final" model when performing cross-validation in machine learning How to choose a predictive model after k-fold cross-validation? My question is ...
17
votes
1answer
2k views

How to build the final model and tune probability threshold after nested cross-validation?

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...
3
votes
2answers
4k views

What is the purpose of crossvalidation?

In this post on stackexchange, the answer states that "The purpose of cross-validation is model checking, not model building." A very good explanation for that is given as follows: "(...) selecting ...
2
votes
1answer
3k views

AIC BIC Mallows Cp Cross Validation Model Selection

If you have several linear models, say model1, model2 and model3, how would you cross-validate it to pick the best model? (In R) I'm wondering this because my AIC and BIC for each model are not ...
2
votes
1answer
4k views

Building final model in glmnet after cross validation

This is my first time working with regularized regression so I apologize if the answer to this is obvious. I am planning on using GLMnet to run a regularized logistic regression on my data set using ...
5
votes
1answer
1k views

Model Stacking algorithm

I'm trying the stacking method to see if it improves my results, but before using some R package, I decided to code it by myself. Here's a pseudocode of what I'm doing: ...

15 30 50 per page