0
$\begingroup$

When using step wise regression to select variables to include, and if running multiple regressions over the same population within different subsets of the population, is it necessary to run the variable selection process over each subset of the data and find the relevant variables for each subset, or simply to find the overall relevant variables of the whole sample and continue from there?

For example, if I wanted to run one regression for people aged between 25 and 40, one for those aged 40-64 and another for people aged 65+.

If the latter, how would you go from there?

$\endgroup$

1 Answer 1

0
$\begingroup$

If you're building different models for different subsets of the population, it is presumably because you believe that the different populations are governed by a different set of rules.

In such a case, I don't see a problem in running a different variable selection process on each subset. You do, however, run the risk of over-fitting that can be avoided by some form of cross-validation. Just make sure that you're running cross-validation on each subset to identify the important predictors.

$\endgroup$
2
  • $\begingroup$ Thanks, that is the case yes, I was hoping to allow for the possibility of different variables being relevant in different subsets. What form of cross-validation would you recommend? $\endgroup$
    – Green90
    Mar 31, 2018 at 16:18
  • $\begingroup$ k-fold cross validation, with say k=10, can be used. Make sure that the feature selection is part of the model-building process. You want to evaluate the best set of features based on the performance on the held-back samples averaged over k-folds. $\endgroup$ Mar 31, 2018 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.