# How to deal with label switching issues with Matlab's classification trees?

I have a situation that seems trivial but I can't figure it out. I have a dataset in Matlab that has categorical values. For example:

Outlook,Temperature,Humidity,Windy,Play
sunny,hot,high,false,no
sunny,hot,high,true,no
overcast,hot,high,false,yes
rainy,mild,high,false,yes
rainy,cool,normal,false,yes
rainy,cool,normal,true,no
overcast,cool,normal,true,yes


In order to use ClassificationTree (or other algorithm in machine learning), I need to convert the dataset to a matrix. I use (assuming that data contains the dataset):

double(data)


which assigns numbers to the values. Let's say rainy=1, overcast=2, sunny=3 and so on. I train the ClassificationTree and it works fine. But here is my problem. If I want to predict on test data:

overcast,cool,normal,false,yes


I need to know the numbers that double() assigned to each of the categories. i.e. overcast=2. Using double() on the test set does not work because there is no guarantee that the numbers assigned by double() are the same to those assigned to the training dataset.

I have found a really twisted way of doing it. For example, for Outlook:

d = zeros(size(test));
ls = getlevels(training.Outlook);
n = size(ls,2);
for i = 1:n
d(test.Outlook == ls(i),1) = i;
end


This will assign the numbers correctly because I check against each of the values for Outlook that I found in the training dataset. This is far from elegant, and there has to be something better.

Any help would be appreciated.

• Maybe you should ask at StackOverflow, since it's a programming issue. – cyborg Dec 21 '11 at 19:11
• Drop Matlab and use a language/tool having actual factor variables (like, for instance, R)? – user88 May 28 '12 at 13:30

Oh yes, I love this one. I am not 100% sure, but most probably double(data) actually calls grp2idx, and that's one crazy function http://www.mathworks.com/help/toolbox/stats/grp2idx.html. Every time I hit this function (e.g. every time I call crosstab), it drives me nuts, and I end up doing somethings really awkward, although not as twisted as your code.

The best way I found around this is to avoid grp2idx entirely by prescribing my own order of labels; see, for example, the options for ordinal http://www.mathworks.com/help/toolbox/stats/ordinal.html -- you can prescribe the labels and the order.

The details will depend on how exactly you build that dataset, but the point is -- instead of letting things happen and then using getlevels, prescribe the levels yourself. It will still be less nice than you'd like it to be, but nowhere near the level of that for loop of yours.

Several of the machine learning algorithms in Statistics Toolbox provide explicit support for categorical inputs. For example, the "fitensemble" function supports an input argument named "CategoricalPredictors"

http://www.mathworks.com/help/toolbox/stats/fitensemble.html

It's probably not a good idea to hand numerical values for categorical data. If you do need to do something like this it would be a good idea to code these using dummy variables.

• Well, ClassificationTree for example requires a matrix of numeric values. Information about conversion class-number is lost. Hence my question. But I just decided to use my option. – x-A-x Jan 2 '12 at 22:36