I am currently performing an analysis in a domain that, in current literature, is predominantly populated by likelihood-based linear models (MLR, Poisson Regression, etc.).

My intent is to implement tree-based regressors (gradient boosting/random forest) for similar tasks and report the predictive performance. I've done my due diligence with respect to proper cross validation, train/test splits, etc. etc., but would like to have at least some acknowledgement of how the performance metrics of my models compare to those linear models that have been empirically implemented.

I know this is not a 1:1 comparison, as pure predictive modelling is certainly a different task than coefficient/parameter based analysis, but I want to cover all of my bases if I can.

So with this in mind, would it be appropriate to derive metrics such as R squared and MAPE by training entirely on my full dataset (as most linear model implementations technically do), and report these alongside my testing performance?


1 Answer 1


As to the titular question, yes. We know that tree based estimators are low bias and potentially high variance. It would be absolutely no surprise that the training error is smaller for the tree estimator.

Even if you present this with the testing error, the question becomes "why?". In my opinion, it would come across as putting a number forth which serves to make your tree model seem better. I would opt for testing error only.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.