Let's say that we want to perform classification on a dataset e.g. whether a customer is going to buy again from a shop or not. There could be a categorical variable, let's say the customer's title (mr, mrs, dr., etc.) which we are mapping to an ordinal one (mr->0, mrs->1, ...).

After this point we are able to create a model e.g. Naive Bayes and perform classification. However it is said, that a feature like this one (i.e. title), makes no sense to carry a meaning for order, then we should transform it into one-hot encoding.

How is this ordinal variable going to create classification issues?

  • $\begingroup$ Assigning numeric codes (labels) 0, 1, 2... Does not automatically make it numeric or ordinal feature. You should inform a statistical procedure or metadata what these values mean. $\endgroup$
    – ttnphns
    Commented Sep 22, 2018 at 14:55
  • $\begingroup$ Thanks @ttnphns. How would you inform a statistical procedure what these values mean? $\endgroup$
    – Michael
    Commented Sep 22, 2018 at 20:11

1 Answer 1


With ordinal inputs, you enforce ordered output. (Assuming your models can actually treat ordinal inputs any different than categorical inputs.)

For instance, if you ordinally encode $\text{Mr} > \text{Mrs} > \text{Dr}$, then you enforce that your fits $\hat{y}$ will (all else being equal) have either $$\hat{y}_{\text{Mr}} > \hat{y}_{\text{Mrs}} > \hat{y}_{\text{Dr}}$$ or $$\hat{y}_{\text{Mr}} < \hat{y}_{\text{Mrs}} < \hat{y}_{\text{Dr}}.$$

That is, the ordinal relationship between the inputs will be preserved and map to an ordinal relationship between the outputs.

If you have a linear model, e.g., a straightforward regression, you will even enforce a linear relationship between the outputs: $$\hat{y}_{\text{Mr}} - \hat{y}_{\text{Mrs}} = \hat{y}_{\text{Mrs}} - \hat{y}_{\text{Dr}}$$

Now, such fits may make sense, depending on your data. We don't know. And that is precisely the point: if you don't have a very good reason why such constraints should hold, it makes no sense to choose a variable encoding that will enforce them.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.