3
$\begingroup$

There are some variables that I measured but strongly suspect are useless because (for example) almost all my data points scored the same on that (binary) variable.

It's been put to me that I may well leave them in, because in a regularized multiple regression it won't hurt to leave them in, even though it probably won't help either.

Is that true? If so/not, why?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

I suggest trying it both ways and seeing if there is a difference. Then you can decide.

If there is a very small difference in the resulting models (try plotting the predicted values from each model against each other; also compare the parameter estimates for the other variables) and there is no good substantive reason for including those variables, then you might as well take them out.

But why are they included in the first place?

Improve this answer
$\endgroup$
1
  • $\begingroup$ When I took the data I didn't realise that almost all my data points would score the same on these variables. Now they're in my data set, so it would be be a (minor) effort to remove them. I'm wondering if in principle leaving them in would be expected to be minorly useful or minorly deleterious. In practice I'll follow your suggestion and see if there's a difference. $\endgroup$
    – user1205901 - Слава Україні
    Aug 27, 2013 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.