2
$\begingroup$

I have two categorical predictors (one is binary, one has six categories) that I'd like to include in a regression. However, it seems to me that they must be correlated in some way, by looking that the following contingency table in R:

         1   2   3   5   6   7
     0   1  76  28 108   1  19
     1   0   0 273   0   1   2

It seems that knowing I'm in category 1 in the binary predictor tells me I'm likely in category 3 in the six-category predictor i.e. one predictor seems to predict the other predictor to some extent. If I wanted to include both of these in a regression since I know they're both important, what would be a good way to proceed? Could I somehow combine them?

$\endgroup$
  • $\begingroup$ What's wrong with just including them both? Correlated predictors are practically unavoidable and not necessarily the end of the world. $\endgroup$ – dsaxton Mar 22 '16 at 3:40
  • $\begingroup$ Can't doing this effect the variance of the estimates? $\endgroup$ – roccomay Mar 22 '16 at 4:09
  • $\begingroup$ It increases the variance, yes. $\endgroup$ – dsaxton Mar 22 '16 at 14:00
  • $\begingroup$ So isn't the point to try to limit the effect of them, if possible? $\endgroup$ – roccomay Mar 22 '16 at 16:38
1
$\begingroup$

(1) you have an issue (with the 6-level variable) of sparse categories. Unlikely that category 1 or 6 will be meaningful in analysis and you might want to collapse the categories (without looking at outcome)
(2) chose one of the variables. Since they appear to be providing the same information there is not benefit to having both variables. Likely the six-level will be more useful. -- and if you don't mind some data snooping (this is a learning project) you should try with both variables and see what happens.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.