2
$\begingroup$

I am using a GLM to model my data. The response variable is binary and I have three predictors of which two are continuous variables and one is binary. Would the distribution of predictors be important when I am fitting the model? That is, would it be a problem if the distribution of the predictor is not uniform or Gaussian, for example?

After fitting the model, if I want to find the importance of each predictor, would their distribution be important here?

Improve this question
$\endgroup$
1
  • $\begingroup$ There are some relevant words in Scortchi's answer here $\endgroup$
    – Glen_b
    Commented Jan 20, 2017 at 21:28

1 Answer 1

Reset to default
4
$\begingroup$

In general, the distribution of predictors is irrelevant. It certainly does not matter if they are normal or uniform, for example, and regression type models can handle binary predictors quite naturally.

There can be some additional nuances in practice, however. For example, it is possible to have a strongly outlying value in a predictor, and that value will typically have great leverage (cf., Interpreting plot.lm()). That means that your results will be driven to a degree by that point. This leads to the development and utilization of robust analyses in statistics. Another issue is that it is not generally possible to say what variable is more 'important' when the variables are incommensurate (see the last paragraph of my answer here: Multiple linear regression for hypothesis testing). However, if you have variables that are truly binary (say, not dichotomized), and the proportions are roughly equal, you can say that the response is more strongly related to one than the other. That is, being intrinsically binary makes variable commensurable.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks for the answer. Could you please explain me why standardization is not the solution to find what variable is more important. I read your other comments, I couldn't fully understand it. $\endgroup$
    – Mina
    Commented Jan 20, 2017 at 21:44
  • 1
    $\begingroup$ @Mina, standardizing your variables just puts them in units of your sample SDs. Eg, your sample SD for weight might be 10 Kg, & your sample SD for heart rate might be 5 beats per minute. So then you are equating 10 Kg w/ 5 bpm. If you think Kg & bpm are incommensurate units (I do), then how much sense does this make? If you think your sample SDs are the true SDs in the population, then you could make the claim that more variation in Y will correspond to the variation that tends to occur in the population, but population distributions shift over time & geography, so even that isn't reliable. $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 21, 2017 at 0:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.