# ML regression methods that generalize linear regressions

It seems to me that despite the huge variety and development of ML methods (I'm specifically interested in regression methods), OLS is still considered, and often cited, as a benchmark - which makes sense at least because of its easy interpretation.

Despite this, I failed to find regression methods that can be thought to generalize a multivariate OLS, in the sense of being able to identify linear relations between features. I find this surprising because in many real world examples (I work with economic data) linear relations coexist with nonlinearities, and because... in principle, it does not seem like a difficult task.

For instance, the value predicted by a regression tree for an observation only depends on the leaf it ends into. Why not fit a linear regression on each leaf instead, and use the resulting (leaf-specific) coefficients for the prediction? Yes, the cost would (if I understand correctly) dominate the cost of building the tree... but on the other hand, it would still be asymptotically equivalent to the cost of an OLS (with a lower memory footprint). And as long as the tree is not too deep (that is: each leaf still contains enough observations), this would for instance perfectly predict a pure linear relation. It would be a sort of multi-dimensional automatic piecewise linear regression.

Or vice-versa, it would be tempting to run a regression tree on OLS residuals. If done by using wisely the (example from sklearn) min_impurity_decrease argument, this would, it seems to me, reliably dominate OLS in terms of explanatory power, at least for large samples (with the two coinciding in the case of a perfectly linear relation).

Is there a reason why techniques of this kind are not widespread? Or are they?