My question is about machine learning to predict ordinal variables.

Most ML models for classification that I have seen do not make any assumption about the order of different categories. I can see that ordinal categorical variables have been used in ML as a predictor variable, but not seen it as a predicted variable.

I am using the caret package in R and in specific looking to predict an ordinal categorical variable using C5.0

Is this possible? and also if the "values" of the order of the categories was 1,2,3 for three categories, would the predicted value ever be something like 1.7 which would suggest that the predictors expect the value to be between 1 and 2 but closer to 2?

If it is possible could a dummy example be shown of how this would be set up?

Thanks in advance

  • $\begingroup$ Also, you can use "tree" package if your data is categorical. here is the link to you tube for further references "youtube.com/watch?v=GOJN9SKl_OE" $\endgroup$ – shivam9523 Dec 19 '16 at 12:25

I don't know the caret package but one method that does use the ordinality of the variable is ordinal logistic regression. In addition, Alan Agresti wrote an entire book on dealing with ordinal data:


You could treat the problem as a classification problem instead of an ordinal one. Use the C5.0 command in the C50 package. I found this youtube video which does a good job of showing you how to do it on an example dataset that comes prepackaged with R.


EDIT: The problem with using tree based regression methods to predict ordinal categorical variables is that output can be inconclusive. For instance, does output of 1.49999 or 1.500001 belong to class 1 or class 2? The accuracy of the result is called seriously into question.

Ordinal logistic regression will output integer values. Try using the polr command in the MASS package. A rough outline of the code would look something like:

model <- polr(y ~., data = train.data, Hess = T)

If you are dead set on using the C50 package, though, I would suggest approaching it as a classification task and ignore the ordinality of the categorical response variable. The C5.0 command is capable of predicting multilevel categorical variables (so is the tree package, incidentally). If you want to use the C5.0 command the code would look something like:

model <- C5.0( y ~., data = train.data)

In either case be sure you're splitting your data into a training set and test set. Check accuracy of both tests, and maybe make a confusion matrix.

  • $\begingroup$ Hello Adam. Could you expand your answer a little bit so that it is standalone, in case the video gets removed from YouTube at some point later? $\endgroup$ – Patrick Coulombe Mar 9 '16 at 1:31
  • $\begingroup$ See the edits above $\endgroup$ – Adam Mar 10 '16 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.