Hello I am running a Regression Tree experiment.

I am new to Regression Trees, and I am using Mean Squared error to test my tree.

I am confused because I am getting a large Mean Squared Error but I am not sure how to evaluate if it is too high.

Should different successful trials expect varying MSE?

Do there exist methods for checking this?


I don't know for sure whether or not there is something wrong with your numbers/code, but if your error is really high, this would actually be pretty standard when you're working with isolated regression trees.

If your error really is high though, try gradient boosting. The algorithm is a subset of boosting algorithms (look into AdaBoost too; it's in the video below). It essentially fits more trees to the residuals of your other models. Check out this video by Prof. Trevor Hastie of H20.ai (& Stanford): https://www.youtube.com/watch?v=wPqtzj5VZus

Hope this helps. Also, if you want to excel, read through this paper by Prof. Robert Schapire and Prof. Yoav Freund, founders of AdaBoost: https://cseweb.ucsd.edu/~yfreund/papers/IntroToBoosting.pdf It isn't about gradient boosting, but maybe you will find AdaBoost or LogitBoost to be more effective.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.