# Do random forests work better than multinomial logistic regression for prediction of categorical non-binary variables? Why?

I posted another question that was well received. I am posting this new question because it was suggested by other members of Cross Validated. Here is the link of the original question that I posted: In R Linear Regression , a categorical variable is changed to numeric to build a model. Would that trick work to predict a categorical variable?

Do Random Forests work better than Multinomial Logistic Regression for Prediction of Categorical Non-Binary Variables? Why?

In the previous question was suggested that Multinomial Logistic Regression works better for inference and that Random Forest works better for prediction. One of the goals of this question is to learn more about that answer.

***Also, I would like to learn the R code for Multinomial Logistic Regression. In the other question, I published a R code for Linear Regression that worked well, in the sense that I got no warnings, but the results had no sense according to @RobertLong This question is a follow up of the previous question.

The data is in the following link: Data for this question

I am trying to predict the last column called classe.

***With random forests I get 99% of accuracy. It is difficult to improve that. However, I would like to compare it with multilinear logistic regression. Also, I would love to learn more about strengths and weaknesses of multilinear logistic regression for this kind of problem. *** Here is my new code with the random forests algorithm. It seems that everything is right because I am getting 99% accuracy.

library(caTools)
library(randomForest)
library(ggplot2)
library(caret)
library(lattice)

set.seed(1234)
dataset <- dataset[, -c(1:7)]

#only keep columns with at least 50% non-blanks

dataset <- dataset[, colSums(is.na(dataset)) < nrow(dataset) * 0.5]

# Keep only the complete rows
dataset <- dataset[complete.cases(dataset), ]

#We eliminate zeroVar and nearZeroVar columns

x = nearZeroVar(dataset, saveMetrics = TRUE)
str(x, vec.len=2)
dataset <- dataset[!x$$zeroVar] x = nearZeroVar(dataset, saveMetrics = TRUE) str(x, vec.len=2) dataset <- dataset[!x$$nzv]

write.csv(dataset, file="data.csv")
dataset <- dataset[, -1]

#classe is opened as chr but it is a categorical variable

dataset$$classe<- factor(dataset$$classe)

#We divide our data in a training set
#and a testing set

split= sample.split(dataset$classe, SplitRatio = 7/10) training_set = subset(dataset, split==TRUE) testing_set = subset(dataset, split==FALSE) #We train our model with the training set ##At this point we have 53 variables. 52 predictors. #We need to use a number close to #sqrt 52 #We are going to use 7 rf_mod <- train( classe ~ ., data = training_set, metric = "Accuracy", method = "rf", trControl= trainControl(method="none"), tuneGrid = expand.grid(.mtry=7) ) #We measure the accuracty of our model #with the testing set rf_pred <- predict(rf_mod, testing_set) confusionMatrix(rf_pred, testing_set$classe)


@Sycorax wrote that we cannot generalize. I agree and disagree at the same time. I agree with his idea that everything depends on the kind of problem. However, I disagree because my question has some particular features that affect the algorithm performance. I am a beginner, but I am reading the book "Practical machine Learning in R," written by Fred Nwanganga and Mike Chapple. They mention that every algorithm has weaknesses and strengths. As we are predicting a non-binary categorical variable, that affects which one is the best algorithm for that purpose. Additionally, I read that some algorithms do not work well with "a large" number of continuous features. In this question, we are analyzing this particular dataset, and by extension, datasets similar to this one.

@jluchman asked me a great question: what better means? I am just a beginner, but correct me if I am wrong. In my perspective, linear regression gives us ideas that can be explained easier than results that are obtained with other methods. However, I just bought some new books, and I am seeing that many authors are skipping multinomial logistic regression for this type of problems and they are going more with naive Bayes, random forests, etc. However, I still need to check some linear regression books.

• Multinomial logistic regression is machine learning… Jun 9, 2021 at 21:29
• You might want to review how multinomial logistic regression relates to logistic regression before you dive into more sophisticated machine learning models.
– Dave
Jun 9, 2021 at 21:39
• Random forest is better for problems where random forest does better. Multinomial logistic regression is better for problems where multinomial logisitc regression is better. If you've found that one is better/worse on your problem, then perhaps we can conclude certain properties about your problem, the available data, the choice of features, or the choice of hyper-parameters, but any categorical statements about the utility of the methods across all problems would be overstated.
– Sycorax
Jun 9, 2021 at 23:59
• One thing that could help to answer the question is a sense for what "better" means for the use of the model? Are you seeking to find an easy to explain model for a specific audience? Are you seeking to find a predictive function that predicts as well as possible for some production application? I anticipate the best tool for the job likely depends on what the need for the model is. Jun 11, 2021 at 12:54
• Hi, I don't think the constant small edits to this question are beneficial. It's hard to answer a question that's a moving target, and it diminishes the visibility of other questions. This one is already highly visible because of the bounty. Jun 15, 2021 at 0:25