Linked Questions

163
votes
6answers
103k views

How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
146
votes
5answers
43k views

Training with the full dataset after cross-validation?

Is it always a good idea to train with the full dataset after cross-validation? Put it another way, is it ok to train with all the samples in my dataset and not being able to check if this particular ...
8
votes
3answers
28k views

How to evaluate results of linear regression

I have a linear regression problem. In short, I have a dataset, I divided it into two subsets. One subset is used to find the linear regression (training subset), another is used to evaluate it (...
6
votes
2answers
9k views

Does cross-validation on simple or multiple linear regression make sense?

Does it make sense to apply train-test split or k-fold cross-validation to a simple linear regression model or multiple linear regression model? I'm really confused about this because I saw this ...
5
votes
2answers
2k views

How to compare performance of regression and classification?

I built a linear regression model and evaluated it with respect to R-Squared and RMSE (the latter cross-validated). Then, I built a logistic regression classifier on the same data. It answers the ...
3
votes
1answer
970 views

Why not using cross validation for estimating the error of a linear model?

Cross validation (CV) seems to be a simple and useful tool for estimating the out-of-sample error of a linear regression model, even though it is rarely used for this purpose. Why that? Is there a ...
0
votes
2answers
4k views

Using and interpreting $k$-fold cross validation for regression

I have created a model using using regression and was trying to check if it is an underfit or overfit by running CV on top of it using Python (sklearn). I have used a higher degree relation between $x$...
0
votes
3answers
1k views

How to compare the performance of two classification methods? (logistic regression and classification trees)

I'm struggling a little bit with comparing these two classification methods. Although I know it is often a bad idea to use stepwise logistic-regression, I still want to perform it and analyse the ...
2
votes
1answer
2k views

Comparing two GLMs using cross validation

Does anybody know how to arrange a cross validation so I can compare two models (negative binomial with quasi-Poisson)? I know some theory beyond cross validation, but don't know what kind of cross ...
5
votes
1answer
2k views

Performing Cross Validation to Compare Lasso and Other Regression Models in R

I have built some "regular" and robust regression models, using the standard lm function as well as rlm and lmrob. While I know that there is some discussion about using stepwise regression, I have ...
3
votes
1answer
651 views

Cross-validation for Comparing Clustering Techniques

I'm working on comparing multiple clustering algorithms to each other using the adjusted Rand index for a given dataset. We have a gold standard that we'd like to compare the obtained clustering ...
1
vote
1answer
300 views

How to pick the best model with cross validation?

Based on my understanding the leave one out cross validation is to hold a sample out as the test set and fit a model with remaining data and then calculate the error of prediction of the test sample ...
1
vote
0answers
422 views

Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
0
votes
1answer
185 views

How to compare classification methods in terms of performance?

I'd like to compare logistic regression to classification trees. In a first step, I compared the theoretical framework of the two classifiers. In a second step, I compared the performance using a ...
1
vote
0answers
172 views

Classification trees - how to avoid “cherry picking”?

This post How to compare the performance of two classification methods? (logistic regression and classification trees) represents a very similar problem I am just trying to solve - I have a dataset ...

15 30 50 per page