0
$\begingroup$

I'm building a logistic regression model in R using glm(y ~ x1 + x2 + x3 + x4, data = train.set, family = binomial(link = 'logit')). Among 4 predictors x1, x2, x3, x4, they all are categorical. However x1, x2, x3 are on a scale of 0 to 10, and x4 is binary (0 or 1).

My question is how should i properly pre-process x4? I'm asking because I know it is a really important variables in terms of prediction, but it's showing a pretty low importance in the summary() due to the fact that it's on a different scale as the rest of predictors.

Could someone who has experienced similar situation share your approach? Thanks a lot!

edit: all predictors x1, x2, x3, x4 are factors. I do understand that as long as they are all factors, it shouldn't matter what values each predictors they have. but we expect x4 to be a more important predictors, but varImp shows the opposite.

$\endgroup$

migrated from stackoverflow.com Jun 26 at 14:00

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ Could you provide a reproducible example ? Also, is x4 (as well as the other predictors) a factor variable ? $\endgroup$ – nghauran Jun 26 at 13:47
  • 1
    $\begingroup$ Your regression will set up dummy variables for the RHS categorical variables. This might help: r-bloggers.com/regression-on-categorical-variables $\endgroup$ – bjorn2bewild Jun 26 at 13:47
  • 1
    $\begingroup$ Just make sure you set your categorical variables as factors. $\endgroup$ – James Jun 26 at 13:51
  • $\begingroup$ How many observations do you have? It's possible you do not have enough for the amount of combinations three 10 level factors and one 2 level factor has. $\endgroup$ – spazznolo Jun 26 at 13:52
1
$\begingroup$

SJ, if these are all categorical (assuming they are factors) then the scale doesn't really matter. The logistic regression isn't analyzing the actual number, but rather the presence or absence of that variable. You could easily rename all of the variables A,B,C,D, etc. within x1, x2, x3, and x4 and you would get the same results because they are categorical.

$\endgroup$
  • $\begingroup$ Hi Jenks, thanks for your response! I do have all variables as factors. but we just expect x4 with higher importance. $\endgroup$ – S.J Jun 26 at 14:04
  • 1
    $\begingroup$ There are many reasons for this. 1. Your beliefs may be incorrect 2. like @spazznolo said previously above you may not have enough power 3. Your subset you are measuring is not representative $\endgroup$ – Jenks Jun 26 at 14:08
0
$\begingroup$

x1 through x3 are not binary. If they are categorical with 10 categories and not ordinal, you can create 10 indicator variables for the presence or absence of each category.

$\endgroup$
  • $\begingroup$ Thans @M Waz! Actually, x1 to x3 are ordinal. is converting to factors is the best way? $\endgroup$ – S.J Jun 27 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.