In practice, why do we convert categorical class labels to integers for classification

Question

This might be a naive question, but I am wondering why we (or maybe it's just me) are converting categorical class labels to integers before we feed them to a classifier in a software package such as Python's scikit-learn ML library?

Let's take the simple Iris dataset, why do we convert the class labels from "Setosa", "Virginica", and "Versicolor" to e.g., 0, 1, and 2?

This question came up when I was collaboratively working on a project and one of my colleagues didn't use a label encoder to convert the class labels from strings to integer. It worked (she was using scikit-learn); I intuitively "corrected" it (inserted a label encoder) and she asked me why: Well, I really had no good answer to that other than "most machine learning algorithms work better this way" (this is something I read sometime ago somewhere).

Now that I thought about it: What is the rationale behind it? Since in typical classification tasks class labels are nominal, not ordinal variables, is it computational efficiency (storing and processing less "data")?

Answer 1

Scikit learn only handles real numbers I believe. So you need to do something like one hot encoding where n numerical dimensions are used to represent membership in the categories. If you just pass in strings they'll get cast to floats in unpredictable ways.

There are mathematical reasons some methods (like svm) need floats. IE they are only defined in the space of real numbers. Representing 3 categories as values 1,2,3 in a single method might work but it may also yield suboptimal performance compared to one hot encoding since the split (1,3) vs (2) is difficult to pick up on unless the method can capture very non linear behavior like that.

Other methods like random forest can be made to work directly on categorical values. Ie during decision learnings you can propose potential splits as diffrent combinations of categories. For such methods it is often convenient to use ints to represent the categories because an array of ints is much nicer to work with then an array of strings on a computational level. You can also do things like generate all possible combinations of n categories by looking at the bit values of an n-bit integer you are incrementing which can be much faster and memory efficient then searching for splits over n-floats.

Answer 2

For binary classification you usually use 0/1 or -1/1. Due to symmetry it does not matter which label corresponds to which class. For multiclass classification e.g. for 3-class classification you cannot use 0, 1 and 2 because this way of labeling implies an order (I am not familiar with Iris dataset though) and cannot be used for categorical data. One way to encode categorical labels is to use (1 0 0), (0 1 0) and (0 0 1). You can think of these labels as vertices of a an equilateral triangle in 3-D. Therefore, no order is implied. However if you are using a binary classifier (such as SVM) instead of a truly multiclass classifier we cannot use this labeling. Instead multiple binary classifiers are trained and their results are somehow combined with each other. For example if you have N categories you can train ${N \choose 2}$ classifiers and for each pair you use labels 0/1 to indicate the two classes (out of N) you are training against each other. At the test time majority vote between all ${N \choose 2}$ classifiers can be used to make a prediction.

If you are using an interface, perhaps it converts your 0/1/2 labels before interacting with the classifier(s) depending on what that classifier is.

Answer 3

It's just a matter of being practical. For binary classification the simplest way is using booleans, for multiclass it's integers. Most back-end libraries are written in statically typed languages (C/C++), and typically use the most basic type that gets the job done without losing information.

Answer 4

Some algorithms can handle only numerical inputs, this might be main reason although storage is other reason.

Of course some algorithms can do conversion implicitly.

Answer 5

There are few algorithms that by default take care of basic label encoding.

But as a developer, you need to make sure that the data which is being passed to the model, is correct representation of reality present in data.

For e.g. if your data has column 'Engineer's Role' then Senior > Junior > Fresher. In this case you need to LabelEncode the values to 3 > 2 > 1 respectively.