Skip to main content
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Saves in:saves
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options not deleted user 108829

Repeatedly withholding subsets of the data during model fitting in order to quantify the model performance on the withheld data subsets.

2 votes
1 answer
2k views

Do I stick with the tuned model parameters even if they produce worse test scores?

The shorter and more general version of this question: If tuning a model via cross-validation (within training set) produces worse results on the test set than my previous default/baseline model, do I …
blu's user avatar
  • 533
13 votes
2 answers
7k views

Is there a rule of thumb for using standard deviation of k-fold cross-validation scores to p...

Let's say I'm comparing 60 different model hyperparameter value combinations using 10-fold cross-validation. It's tempting to simply select the hyperparameter combination whose mean accuracy is highe …
blu's user avatar
  • 533
16 votes
2 answers
8k views

How does k-fold cross validation fit in the context of training/validation/testing sets?

My main question is with regards trying to understand how k-fold cross-validation fits in the context of having training/validation/testing sets (if it fits at all in such context). Usually, people s …
blu's user avatar
  • 533
1 vote
2 answers
3k views

Can I use test data to fit a TFIDF Transformer/Vectorizer?

From the examples I've seen, it seems to me that generally people build a TFIDF transformer based on only on text in the training data, and then use that transformer to transform the test data. Why …
blu's user avatar
  • 533