Does a regression tree strictly dominate OLS in prediction?

Question

Since OLS tries to measure E[Y|X], and regression trees try to partition the data into different branches, then take means of the response under different branches, is it reasonable to say that regression trees strictly dominate OLS in prediction? Also, since OLS is not robust to outliers, would it be reasonable to say that regression trees are a better first, off-the-shelf algorithm to try in a prediction problem?

For interpretation, it definitely makes sense to me that knowing $R^2$, regression coefficients, p-values, etc. are more useful than seeing the tree structure.

Answer 1

No Regression trees do not dominate OLS regression. OLS regression is intended for models where you want to estimate $E[Y|X]$ where $X$ is a set of predictors and the residuals from the model are continuous and Gaussian with mean $0$. Under that setting OLS should be superior to the regression tree. Remember that the regression model takes acoount of the value of the covariate whereas the tree splits or partitions it into discrete segments and thus does not fully use all the information in the data other than to use it to find the best places to split. On the other hand when there are outliers or the residual component is very non normal the OLS regression puts too much weight on outliers and leverage points and the regression tree or a robust linear regression may do much better. Also even though you may be more comfortable with $R^2$, $p$-values and regression coefficients one of the importnat points about CART that Richard Olshen pointed out in the CART book is that when they applied classification and regression trees to medical problems the physicians found the tree structure very intuitive and more believable than a linear regression or linear discriminant analysis.