Are Random Forests more powerful than generalized linear models?

Question

I have never used Random Forests, but I have read some about it. Until now I have used GLM/GAMLSS extensively.

I would like to know:

1. What are the advantages that RF provides over GLM/GAMLSS?
2. What are the disadvantages of using Random Forests?

I am starting this new study which has around 25 predictor variables, and I was wondering if I should check random forests.

One disadvantage that I could find: Random Forests, at least the popular applications, are non-updatable. Is this True? This is important for me as I will run this on some real-time data and would need to assimilate new information.

Answer 1

You should try lots of models. The 'no free lunch' theorem states that there is no one best model - every situation is different. Logistic regression for example is desirable when it works because parameters are very interpretable. Random forests are great because they can deal with very difficult patterns, but forget about interpreting them.

The point is - never stick to just one approach.

Answer 2

One point to consider is are you interested in making predictions or understanding associations and carrying out inference (confidence intervals around effects). Although random forests provide a variable-importance summary, this technique is primarily aimed at prediction; there is no inference. Many researchers think they are interested in making predictions, but often there is a mismatch with their goals.

With that said, you can make predictions with glm and gamlss. You also have the flexibility of regression. A benefit to the random forests is that you do not have to specify aspects such as interactions and because of this, it may discover patterns in your data. Further it handles the case where there are more predictors than observations.

Still, there is evidence that techniques such as random forests do not use the data as efficiently as conventional techniques do. More research is needed.

I think both techniques are updatable in the same manner.

Reference: van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014). Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC medical research methodology, 14(1), 137.

Answer 3

Let's have some simple examples to show the differences. Our example have a single independent variable x and a single dependent variable - either real y or categorical z:

x    y    z
...
0    0.01 A
1    1.98 A
2.01 4.02 B
2.99 6.01 B
...

One can see that y grows as x grows and that z=B for values of x grater than something around 1.5. That is an example when GLM and similar methods rock. It is easy to make good predictions for both y and z for a new value of x. For example, for x=0.5 you would predict y=1 and z=A ("You can understand the direct influence, and direction, of each variable" as @HEITZ wrote)

x    y    z
...
0    0.01 A
1    1.98 B
2.01 0    A
2.99 2.01 B
4    0    A
5.01 2.00 B
...

We can see a clear pattern in the data again, however GLM and similar methods cannot, the connection between x and y or z is not linear nor even additive. That is when other methods as random forests needs to be used. Prediction based on GLM for x=3 would be y=1 and more or less randomly z=A or z=B. However RF or similar methods can predict what we would expect: y=2 and z=B.

Generally I would do:

1. Try GLM and similar, if they model the data well, use it, because you can understand the model well; if not
2. Try other methods as RF (or neural networks, etc.)