2
votes
How large should my data set be before I can do a train-test split (cross validation)?
Frank Harrell recommends having on the order of 20,000 observations before you consider doing a train/test data split. That's based on experience and simulations, not a subjective assessment.
If you ...
- 98k
1
vote
How to properly report results from Cross Validation, specifically standard deviation
Let accuracy (or any other performance metric) of fold $i$ be $X_i$, out of $n$ folds. You'd likely want to report the overall success of your model considering all the folds, which is usually chosen ...
- 57.9k
1
vote
Is it o.k. to stack out-of-sample predictions from separate cross-validation rounds?
I don't think that there's much of a problem with stacking results from multiple cross-validation runs for LASSO per se,* but I also don't think that will do what you want with respect to things like ...
- 98k
1
vote
Rademacher Bound, An Alternative to Cross Validation for Ridge?
This answer is based on intuition and experience with other bounds, so YMMV, as they say.
It should be possible to perform model selection using this bound - I suspect it has already been done, but ...
- 55.2k
Only top scored, non community-wiki answers of a minimum length are eligible