I hope this question helps shed some light on trees vs neural models. I recently came across a model tree, or a recursive partitioning model. It is basically a decision tree that has linear regression models trained for every leaf node of this tree. A most prominent example of this method could be the M5 model from Quinlan, 1992. Basically what got me here is this thread. I thought about this model and it seemed to me that it is relatively similar to a feed-forward neural network using ReLU as activation functions. I played around a little bit with this nice visualization tool from Karpathy. Try playing around with how only ReLU activations would get joined together to create a nice approximation for the input data.

I was wondering if anyone could explain the main, fundamental differences about what these models can or cannot do; or in what way they are the same/different. All your answers are appreciated!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.