2 votes
Accepted

Adj. $R^2$ with tree ensembles

No, adjusted $R^2$ is not a valid metric of tree ensembles. In order to have an "adjusted" $R^2$ we need some agreed concept of degrees of freedom. Tree ensembles don't have one, so the idea ...
user avatar
1 vote
Accepted

XGBoost Objective Derivation Problem

Since you mention probabilities, I assume you are thinking about a binary classification, in which case the trees all operate in the log-odds space, the $x$s in your notation. So the $\hat{y}$ and $...
user avatar
  • 2,836
1 vote
Accepted

Boosting usa bootstraping?

We always subsample our original training set in each booster iteration. Usually this is controlled by a parameter like: bagging, ...
user avatar
1 vote
Accepted

Linear regression has good performance in validation set despite not meeting the linearity assumption

The idea that you'd need to "make the data meet" certain model assumptions is wrong, as model assumptions are never perfectly fulfilled anyway. In particular, formal model assumptions almost ...
user avatar
1 vote

XGBoost and how to input feature interactions

In theory, tree based models like gradient boosted decision trees (XGBoost is one example of a GBDT model) can capture feature interactions by having first a split for one variable and then one for ...
user avatar
  • 22.2k

Only top scored, non community-wiki answers of a minimum length are eligible