What is an example of how ordinal data can yield bad classification results in contrast with one-hot encoding?

Question

Let's say that we want to perform classification on a dataset e.g. whether a customer is going to buy again from a shop or not. There could be a categorical variable, let's say the customer's title (mr, mrs, dr., etc.) which we are mapping to an ordinal one (mr->0, mrs->1, ...).

After this point we are able to create a model e.g. Naive Bayes and perform classification. However it is said, that a feature like this one (i.e. title), makes no sense to carry a meaning for order, then we should transform it into one-hot encoding.

How is this ordinal variable going to create classification issues?

Answer 1

With ordinal inputs, you enforce ordered output. (Assuming your models can actually treat ordinal inputs any different than categorical inputs.)

For instance, if you ordinally encode $\text{Mr} > \text{Mrs} > \text{Dr}$, then you enforce that your fits $\hat{y}$ will (all else being equal) have either $$\hat{y}_{\text{Mr}} > \hat{y}_{\text{Mrs}} > \hat{y}_{\text{Dr}}$$ or $$\hat{y}_{\text{Mr}} < \hat{y}_{\text{Mrs}} < \hat{y}_{\text{Dr}}.$$

That is, the ordinal relationship between the inputs will be preserved and map to an ordinal relationship between the outputs.

If you have a linear model, e.g., a straightforward regression, you will even enforce a linear relationship between the outputs: $$\hat{y}_{\text{Mr}} - \hat{y}_{\text{Mrs}} = \hat{y}_{\text{Mrs}} - \hat{y}_{\text{Dr}}$$

Now, such fits may make sense, depending on your data. We don't know. And that is precisely the point: if you don't have a very good reason why such constraints should hold, it makes no sense to choose a variable encoding that will enforce them.