If we think back to linear models for a moment, we have Ordinary Least Squares (OLS) versus Generalized Linear Models (GLM). Without going too in-depth, it can be said that GLMs "improve" upon OLS by relaxing some of the assumptions, making it more robust to different types of data. The underlying training algorithm is also somewhat different; OLS minimizes the root mean squared error (RMSE) while GLMs minimize deviance. (I realize that RMSE is a special case of deviance). This allows us to build linear models based on, say, the gamma distribution, inverse gaussian, etc.
My question is: does the same logic hold true for gradient boosted trees? Since we're working with tree based algorithms now, I'd think that it's not subject to the same assumptions/distributional restrictions as linear models. In the XGBoost package, for example, the default objective function for regression is RMSE. You can define a custom objective if you wish, but does it matter? Does it make sense to do so?
In other words, can we possibly improve our predictive power by setting XGBoost to minimize deviance (say, of a gamma distribution) versus RMSE?