0
$\begingroup$

In a k-fold cross-validation scenario for validating a regression model, how/when should I do preprocessing of the data? I'd like to know especially when to remove outliers, empty columns etc, i.e. if I should remove outliers only on the training/test set and do this preprocessing on each k-fold iteration or if I should do these preprocessing tasks on the overall data set and then just split and train/test.

I've also had a look at this question: Data preparation for cross validation, but as already stated I'm more interested in removing columns and outliers.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ It’s the exact same answer as the linked post, though. $\endgroup$
    – Arya McCarthy
    Commented May 3, 2021 at 13:55

1 Answer 1

Reset to default
1
$\begingroup$

The answer in the linked thread applies.

  • removing columns is obviously a preprocessing step that involves more than one case. => decide anew for each fold and apply this decision to the respective test data. I'd recommend to keep track on how stable these decisions are across the folds/runs.
  • Outlier removal that is based on external knowledge and applied to each case separately such as e.g. removing cases where some sensor reading was saturated and the signal threshold is known beforehand can be done as a common pre-processing before the CV.
    Outlier removal that establishes what data is OK and what are outliers from the data needs to be evaluated as part of the CV: decide the boundary within each fold's training data and apply that boundary to the respective test set.
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your thorough answer. There's just one thing left that is unclear to me: when you say 'decide anew for each fold and apply this decision to the respective test data', that means I make a decision based on the training set (e.g. in column A there are >90% NaN values, so I drop them) and then drop the selected columns also in the test set? So, the test set shouldn't influence the decision whether a column should be dropped basically? $\endgroup$
    – krautgortna
    Commented May 4, 2021 at 11:39
  • $\begingroup$ @krautgortna: yes. All decisions you take must be based on the training set only. CV simulates that new data is coming in after the model is completely trained and all parameters/decisions fixed. $\endgroup$
    – cbeleites
    Commented May 6, 2021 at 6:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.