5
$\begingroup$

In real world problems, it is quite often that we build prediction models with both continuous and categorical variables. My most naive approach to preprocessing is:

  1. Turn categorical variables into integers. (male,female)-> 0,1 etc
  2. then normalize all features, both categorical and continuous variables
  3. experiment with different predicting models and parameters...

I stopped asking questions long time about weather it makes sense turning categorical variables into integers and then normalize it. Nor do I even consider if it makes sense to put variables of both type into any model and test it.

My question is do we treat categorical variables just like continuous variables during prediction?

$\endgroup$
  • 4
    $\begingroup$ By normalize do you mean "standardize" to mean 0 and sd 1, "scale to length 1", "convert to be on [0,1]", "make approximately normal" or something else? $\endgroup$ – Glen_b Mar 9 '17 at 6:01
  • $\begingroup$ @Glen_b, Yes, that's what exactly what I mean. $\endgroup$ – user6396 Mar 9 '17 at 6:07
  • 4
    $\begingroup$ @user6396 Which one of the four options I mentioned is exactly what you mean? $\endgroup$ – Glen_b Mar 9 '17 at 6:29
  • $\begingroup$ This method only works to a point. Coding the categorical data in one of the four ways Glen_b mentioned makes the maths easier to analyze models but in the end, the chances that your data only works for hermaphrodites when you only ran your test with men and women doesn't make sense, so you will need to interpret your model. Depending on your software, there are usually methods for identifying coded categorical data. $\endgroup$ – Tavrock Mar 9 '17 at 7:00
  • $\begingroup$ @Glen_b, I use them all, depending on different situations, I use different methods. $\endgroup$ – user6396 Mar 9 '17 at 7:09
2
$\begingroup$

This is still a matter of debate and it also depends strongly on the algorithms you're using.

For example, if you're using Lasso for feature selection, all of the features should be on the same scale and standardization of binary features is recommended (see The Elements of Statistical Learning by Tibshirani et al.: http://web.stanford.edu/~hastie/ElemStatLearn/).

Logistic regression doesn't profit as much from normalization of binary variables: Should you ever standardise binary variables?.

Interestingly enough, Andrew Gelman suggested standardizing by dividing with two times the standard deviation (not only for binary variables), such that you can interpret the influence of the regression coefficients more easily: http://www.stat.columbia.edu/~gelman/research/published/standardizing7.pdf.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.