3
$\begingroup$

Can someone explain in which cases it is a good idea to convert a categorical variable to numerical in order to use it in our model (either regression or classification)? I have seen cases where even the male/female gender category has been converted to 0/1 ... Also, what's the measure to confirm that this transformation must be included to our model i.e., is a good feature.

$\endgroup$
3
$\begingroup$

Consider a linear regression model with response $Y_i$. Further, say that we only have one predictor here, being either male or female, as you mentioned. Then this simple model is clearly $$ Y_i = \beta_0 + \beta_1I_{MALE} + \epsilon_i $$ It is necessary to encode this categorical variable as being either 0 or 1 in order to obtain the least squares estimates of our coefficients $\hat{\beta_i}$. In this way, if a particular observation is a female then the indicator is 0 and the response is simply $\hat{Y}_i=\hat\beta_0$. In short, it's necessary in order for the math to make sense.

Also, to confirm that this variable is significant to our model, we can perform a Wald test of significance for the coefficient in question (in the example $\hat \beta_1$).

$\endgroup$
  • $\begingroup$ You could get least-squares estimates (or others for that matter) if you encoded males as 7 and females as 42. So, 0-1 encoding isn't necessary for that purpose. 0-1 encoding is also a good idea because it allows simple interpretations of the coefficients, as you do convey. 0-1 encoding of binary variables is also a good idea because the mean of (0, 1) values has a direct interpretation as the proportion of values that are coded 1. $\endgroup$ – Nick Cox Dec 24 '19 at 12:10
1
$\begingroup$

The question has two possible interpretations depending on what exactly is meant by "convert to numerical".

When we have a categorical variable we have to parameterize it somehow. For a dichotomous variable this is easy: Call one option 1 and the other 0. So, if you have two choices for "sex" (but be careful, you may want more) then you could change the variable from "sex" to "male" and then code Yes as 1 and No as 0.

If the variable has more choices then you can use dummy coding, effect coding, Helmert coding, etc. All those involve using 0's and 1's.

But, while 0 and 1 are clearly numbers, I'm not sure I'd say that is "converting to numerical" and I worry that the OP or other readers (especially people new to statistics) may get the wrong notion.

If, say, we have a variable: "What was your major in college?" then one of the above methods could be used. But it would not be right to (say) make math = 1, history = 2, English = 3 and so on. And that sort of thing is what comes to my mind on reading the question.

$\endgroup$
  • $\begingroup$ Encoding to numeric -- even for unordered categories -- is often essential because your software won't do you want otherwise, say whenever you want a categorical predictor to be treated as a set of indicator variables. Software you don't use always has a strange way to do it compared with software you do use. The point is even stronger for ordered categories, such as a scale from strongly agree to strongly disagree. $\endgroup$ – Nick Cox Dec 24 '19 at 13:16
  • $\begingroup$ @NickCox Yes. I agree. But I am not sure that what the OP is asking about is setting up indicator variables (which is fine) or something like what is in my final paragraph (which isn't fine at all). Was I not clear about that? If not, how can I edit it to make it clearer? $\endgroup$ – Peter Flom - Reinstate Monica Dec 24 '19 at 13:24
  • 1
    $\begingroup$ I think you're clear in what you say. I am just expanding. $\endgroup$ – Nick Cox Dec 24 '19 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.