Can Tree-based regression perform worse than plain linear regression? Hi I'm studying regression techniques.
My data has 15 features and 60 million examples (regression task).
When I tried many known regression techniques (gradient boosted tree, Decision tree regression, AdaBoostRegressor etc) linear regression performed great. 
Scored almost best among those algorithms.
What can be the reason for this? Because my data has so many examples so DT based method can fit well.


*

*regularized linear regression ridge, lasso performed worse


Can anyone tell me about other performing-well regression algorithms?


*

*Is Factorization Machine and Support vector regression good regression technique to try?   

 A: It is a well known fact that trees are ill-suited to model truly linear relationships. Here is an illustration (Fig 8.7) from the ISLR book:

Top Row: A two-dimensional classification example in which
the true decision boundary is linear, and is indicated by the shaded regions.
A classical approach that assumes a linear boundary (left) will outperform a decision
tree that performs splits parallel to the axes (right).
So if your dependent variable depends on the regressors in a more or less linear fashion, you would expect that "linear regression performs great".
A: Any decision-tree based approach (CART, C5.0, random forests, Boosted regression trees etc.) identify homogeneous areas in your data and assign the mean value of the data contained in that region to the corresponding 'leave'. So, they are granular and then, they must show a series of steps in the outputs. Those based on 'forests' do not show that phenomenon pronouncedly but it is still there. The aggregation of a large number of trees nuances it. When a given value is outside the original range the datum is assigned to the ‘leave’ that includes the extreme condition found in the training dataset and the output is consequently the mean value of the values contained in that leave. Thus, no extrapolation is possible.
By the way ANNs are poor extrapolators. You can check: Pichaid Varoonchotikul - Flood Forecasting using Artificial Neural and Hettiarachchi et al. The extrapolation of artificial neural networks for the modelling of rainfall—runoff relationships they are very illustrative and are easy to find in the net!
Good luck!
A: You should not just throw the data at different algorithms and look at the quality of the predictions.  You need to understand your data better, and the way of going about that is to first, visualize your data (the marginal distributions).   Even if you are only interested finally in the predictions, you will be in a better position to make better models if you understand the data better.  So, first, try to understand the data (and simple models fitted to the data) better, and then you are in a much better position to create more complex, and hopefully better, models.
Then, fit linear regression models, with your 15 variables as precictors (later you can look at possible interactions).  Then, calculate the residuals from that fit, that is, 
$$
    r_i  = Y_i - \hat{Y}_i, \qquad i=1, 2,\dots, n
$$ 
If the model is adecuate, that is, it was able to extract the signal (structure) from the data, then the residuals should show no patterns. Box, Hunter & Hunter: "Statistics for Experimenters" (which you should have a look at, its one of the best ever books on statistics) compares this with an analogy from chemistry: The model is a "filter" designed to catch impurities from water (the data).  What is left, which passed through the filter, should then be "clean" and analysis of it (residuals analysis) can show that, when it does not contain impurities (structure).  See Checking residuals for normality in generalised linear models 
To know what to check for you need to understand the assumptions behind linear regression, see What is a complete list of the usual assumptions for linear regression? 
One usual assumption is homoskedasticity, that is, constant variance. To check that, plot the residuals $r_i$ against the predicted values, $\hat{Y}_i$. To understand this procedure see:  Why are residual plots constructed using the residuals vs the predicted values?.  
Other assumptions is linearity. To check those, plot the residuals against each of the predictors in the model. If you see any curvity in those plots, that is evidence against linearity.  If you find non-linearity, either you can try some transformations or (more modern approch) include that non-linear predictor in the model in a non-linear way, maybe using splines (you have 60 million examples so should be quite feasible!). 
Then you need to check for possible interactions.  The above ideas can be used also for variables not in the fitted model.  Since you fit a model without interactions, that include interaction variables, like the product $x_i \cdot z_i$ for two variables $x$, $z$.  So plot the residuals against all this interaction variables.  A blog post with many example plots is http://docs.statwing.com/interpreting-residual-plots-to-improve-your-regression/
A book-length treatment is R Dennis Cook & Sanford Weisberg: "Residuals and influence in regression", Chapman & Hall.  A more modern book-length treatment is Frank Harrell: "Regression modeling Strategies". 
And, coming bact to the question in the title: "Can Tree-based regression perform worse than plain linear regression?"  Yes, of course it can.  Tree-based models has as regression function a very complex step function.  If the data truly comes from (behave as simulated from) a linear model, then step functions can be a bad approximation. And, as shown by examples in the other answer, tree-based models might extrapolate badly outside the range of the observed predictors.  You could also try randomforrest and see how much better that is than a single tree. 
A: Peter Ellis has a 
very simple example

where linear regression performs better than regression trees, extrapolating beyond the observed values in the sample.
In this image the black points are the observed values, and the colored points are the predicted values. The actual data are generated according to a simple line with some noise, so linear regression and the neural network do a good job of extrapolating beyond the observed data. The tree based models do not.
Now, with 60 million data points you might not be worried about this. (The future always manages to surprise me though!) But it is an intuitive illustration as to one situation in which trees will fail.
