The Stack Overflow podcast is back! Listen to an interview with our new CEO.

New answers tagged

0

If it is a regression model (objective can be reg:squarederror), then the leaf value is the prediction of that tree for the given data point. The leaf value can be negative based on your target variable. The final prediction for that data point will be sum of leaf values in all the trees for that point. If it is a classification model (objective can be ...


2

Deep neural networks are very successful over a wide range of applications on raw and unstructured data, such as image pixels, signals, texts, etc. Their main advantage in these contexts is their ability to identify relevant features in the data. This task-oriented, data-driven feature identification usually outperforms generic hand-crafted feature ...


0

It's easy to derive $P(p_i, p'_i)$ from the equation of the two formulations you used for $LL$. Remember that $t_i$ can only be 0 or 1. $$ P(p_i, p'_i) = \begin{cases} p'_i\,^{w'_i}, & t_i = 1\\ (1 - p'_i)^{w'_i} -1, & t_i = 0 \end{cases} $$ In this terms alone, a function $w'_i= w(p_i, t_i)$ that makes $P(p_i, p'_i)$ independent on $t_i$ is surely ...


0

The https://xgboost.readthedocs.io/en/latest/tutorials/model.html [documentation of XGBoost][1] provides a great introduction to the boosted trees algorithm. Perhaps, I can help out answering your first question: I think you state two false alternatives. The algorithm uses the entire training set (leaving aside the fact that you can sample observations in ...


0

In boosting we try to pick the dataset on which the algorithm results were poor instead of randomly choosing the subset of data. These hard examples are important ones to learn, so if the data set has a lot of outliers and algorithm is not performing good on those ones than to learn those hard examples algorithm will try to pick subsets with those examples.


3

The main difference between AdaBoost and other "generic" boosting algorithms is that AdaBoost uses the (deviance) residuals as weights while "generic" gradient boosting algorithms use the residuals as the learning target itself. This different between AdaBoost and other "generic" Gradient Boosting Machine (GBM) methodologies is more prominent when we ...


0

Decision Trees - Ada Boosting from sklearn.tree import DecisionTreeClassifier from sklearn.ensemble import AdaBoostClassifier from sklearn.metrics import accuracy_score Decision Trees with No Boosting clf_entropy_no_ada = DecisionTreeClassifier(criterion = "entropy", random_state = 100, max_depth=5, min_samples_leaf=5) clf_entropy_no_ada.fit(X_train, ...


1

Yes, it works perfectly for the case of classification because we use the deviance residuals. Deviance residuals are directly related with the contribution each point has to the overall likelihood score. The deviance residuals are commonly used to check the model fit at each observation for generalized linear models; a gradient boosting machine is no ...


0

A nice thing about decision trees is that they exploit high-dimensional interactions among the features. A seemingly useless split at level 1 may lead to ground-breaking splits at levels, 2, 3 and further. No simple statistical test will be able to "see the big picture" if applied at level 1. This will hold no matter whether there is approximate normality in ...


0

1) and 2) use different models as reference. 1) Compared to the simple base learner (e.g. a shallow tree), boosting increases variance and reduces bias. 2) If you boost a simple base learner, the resulting model will have lower variance compared to some high variance reference like a too deep decision tree.


Top 50 recent answers are included