11
$\begingroup$

In real world problems, it is quite often that we build prediction models with both continuous and categorical variables. My most naive approach to preprocessing is:

  1. Turn categorical variables into integers. (male,female)-> 0,1 etc
  2. then normalize all features, both categorical and continuous variables
  3. experiment with different predicting models and parameters...

I stopped asking questions long time about weather it makes sense turning categorical variables into integers and then normalize it. Nor do I even consider if it makes sense to put variables of both type into any model and test it.

My question is do we treat categorical variables just like continuous variables during prediction?

Improve this question
$\endgroup$
5
  • 6
    $\begingroup$ By normalize do you mean "standardize" to mean 0 and sd 1, "scale to length 1", "convert to be on [0,1]", "make approximately normal" or something else? $\endgroup$
    – Glen_b
    Commented Mar 9, 2017 at 6:01
  • $\begingroup$ @Glen_b, Yes, that's what exactly what I mean. $\endgroup$
    – user6396
    Commented Mar 9, 2017 at 6:07
  • 6
    $\begingroup$ @user6396 Which one of the four options I mentioned is exactly what you mean? $\endgroup$
    – Glen_b
    Commented Mar 9, 2017 at 6:29
  • $\begingroup$ This method only works to a point. Coding the categorical data in one of the four ways Glen_b mentioned makes the maths easier to analyze models but in the end, the chances that your data only works for hermaphrodites when you only ran your test with men and women doesn't make sense, so you will need to interpret your model. Depending on your software, there are usually methods for identifying coded categorical data. $\endgroup$
    – Tavrock
    Commented Mar 9, 2017 at 7:00
  • $\begingroup$ @Glen_b， I use them all, depending on different situations, I use different methods. $\endgroup$
    – user6396
    Commented Mar 9, 2017 at 7:09

1 Answer 1

Reset to default
11
$\begingroup$

This is still a matter of debate and it also depends strongly on the algorithms you're using.

For example, if you're using Lasso for feature selection, all of the features should be on the same scale and standardization of binary features is recommended (see The Elements of Statistical Learning by Tibshirani et al.: http://web.stanford.edu/~hastie/ElemStatLearn/).

Logistic regression doesn't profit as much from normalization of binary variables: Should you ever standardise binary variables?.

Interestingly enough, Andrew Gelman suggested standardizing all numeric variables by dividing with two times the standard deviation and leave binary variables as they are, such that you can interpret and compare the influence of the regression coefficients more easily: http://www.stat.columbia.edu/~gelman/research/published/standardizing7.pdf.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.