Do random forests work better than multinomial logistic regression for prediction of categorical non-binary variables? Why?

Question

I posted another question that was well received. I am posting this new question because it was suggested by other members of Cross Validated. Here is the link of the original question that I posted: In R Linear Regression , a categorical variable is changed to numeric to build a model. Would that trick work to predict a categorical variable?

Do Random Forests work better than Multinomial Logistic Regression for Prediction of Categorical Non-Binary Variables? Why?

In the previous question was suggested that Multinomial Logistic Regression works better for inference and that Random Forest works better for prediction. One of the goals of this question is to learn more about that answer.

***Also, I would like to learn the R code for Multinomial Logistic Regression. In the other question, I published a R code for Linear Regression that worked well, in the sense that I got no warnings, but the results had no sense according to @RobertLong This question is a follow up of the previous question.

The data is in the following link: Data for this question

I am trying to predict the last column called classe.

***With random forests I get 99% of accuracy. It is difficult to improve that. However, I would like to compare it with multilinear logistic regression. Also, I would love to learn more about strengths and weaknesses of multilinear logistic regression for this kind of problem. *** Here is my new code with the random forests algorithm. It seems that everything is right because I am getting 99% accuracy.

library(caTools)
library(randomForest)
library(ggplot2)
library(caret)
library(lattice)

dataset <- read.csv("data.csv", header=T)
set.seed(1234)
dataset <- dataset[, -c(1:7)]

#only keep columns with at least 50% non-blanks

dataset <- dataset[, colSums(is.na(dataset)) < nrow(dataset) * 0.5]

# Keep only the complete rows
dataset <- dataset[complete.cases(dataset), ]

#We eliminate zeroVar and nearZeroVar columns

x = nearZeroVar(dataset, saveMetrics = TRUE)
str(x, vec.len=2)
dataset <- dataset[!x$zeroVar]
 x = nearZeroVar(dataset, saveMetrics = TRUE)
str(x, vec.len=2)
dataset <- dataset[!x$nzv]

write.csv(dataset, file="data.csv")
dataset <- read.csv("data.csv", header=T)
dataset <- dataset[, -1]

#classe is opened as chr but it is a categorical variable

dataset$classe<- factor(dataset$classe)

#We divide our data in a training set
#and a testing set

split= sample.split(dataset$classe, SplitRatio = 7/10)
training_set = subset(dataset, split==TRUE)
testing_set = subset(dataset, split==FALSE)

#We train our model with the training set
##At this point we have 53 variables. 52 predictors.
#We need to use a number close to 
#sqrt 52
#We are going to use 7

rf_mod <- train(
  classe ~ .,
  data = training_set,
  metric = "Accuracy",
  method = "rf",
  trControl= trainControl(method="none"),
  tuneGrid = expand.grid(.mtry=7)
 )

 #We measure the accuracty of our model
 #with the testing set

 rf_pred <- predict(rf_mod, testing_set)
 confusionMatrix(rf_pred, testing_set$classe)

---

@Sycorax wrote that we cannot generalize. I agree and disagree at the same time. I agree with his idea that everything depends on the kind of problem. However, I disagree because my question has some particular features that affect the algorithm performance. I am a beginner, but I am reading the book "Practical machine Learning in R," written by Fred Nwanganga and Mike Chapple. They mention that every algorithm has weaknesses and strengths. As we are predicting a non-binary categorical variable, that affects which one is the best algorithm for that purpose. Additionally, I read that some algorithms do not work well with "a large" number of continuous features. In this question, we are analyzing this particular dataset, and by extension, datasets similar to this one.

@jluchman asked me a great question: what better means? I am just a beginner, but correct me if I am wrong. In my perspective, linear regression gives us ideas that can be explained easier than results that are obtained with other methods. However, I just bought some new books, and I am seeing that many authors are skipping multinomial logistic regression for this type of problems and they are going more with naive Bayes, random forests, etc. However, I still need to check some linear regression books.

Answer 1

Random forest is better for problems where random forest does better. Multinomial logistic regression is better for problems where multinomial logisitc regression is better. -Sycorax (in the comments)

This is really all there is to it, but it might help to unpack this comment.

If you have a relationship that (at least approximately) follows the multinomial logistic model, then that is the (approximately) correct model. It might be that a random forest can achieve high in-sample performance, but when it comes to making out-of-sample predictions, it will have achieved that awesome performance by fitting to coincidences in the training data rather than the true relationship between the features and the outcome. In such a scenario, the logistic regression outperforms the random forest, even in terms of pure predictive ability, contradicting your source.

To be fair to your source, however, relationships in real life tend to be complicated. Simple models like multinomial logistic regressions are likely to miss important relationships that flexible models like random forests are able to detect. If you control for overfitting concerns, then letting a random forest "go do its thing" might make for a better out-of-sample predictor.

Yes, there are theorems like Stone-Weierstrass saying that (generalized) linear models with polynomial features can be as good at approximating nonlinear relationships as is demanded (under decent conditions), but you have to engineer those polynomial features and know which ones to engineer. For a random forest, all you do is program considerable flexibility and let it take care of the rest (for better or for worse).