# Using Boosting tree to generate feature in sklearn

I am referring to this link to Feature Transformation using tree ensembles for the context.

Specifically for below part of code, in the sample of the link, the method of (1) using Boosting tree to generate feature, then using LR to train, outperforms (2) using Boosting tree itself. Questions,

1. Wondering if it is true in general case using Boosting tree to generate feature (and using another classifier to classify) is better than using Boosting tree to do classification itself?
2. And also wondering why using Boosting tree to generate feature, then using LR to train, outperforms using Boosting tree itself?

grd = GradientBoostingClassifier(n_estimators=n_estimator)
grd_enc = OneHotEncoder()
grd_lm = LogisticRegression()
grd.fit(X_train, y_train)
grd_enc.fit(grd.apply(X_train)[:, :, 0])
grd_lm.fit(grd_enc.transform(grd.apply(X_train_lr)[:, :, 0]), y_train_lr)


The better performance of GBT+LR in the mentioned example is just because larger training set than GBT. Please note the GBT use X_train for training, while GBT+LR use X_train and X_train_lr. I prepared a new classificator GBT-ext, that is learned on the full training set, same as the GBT+LR.

grd_ext = GradientBoostingClassifier(n_estimators=n_estimator)
grd_ext.fit(np.concatenate([X_train, X_train_lr]), np.concatenate([y_train,
y_train_lr]))

y_pred_grd_ext = grd_ext.predict_proba(X_test)[:, 1]
fpr_grd_ext, tpr_grd_ext, _ = roc_curve(y_test, y_pred_grd_ext)


Results of the GBT-EXT are almost the same as the GBT+LR. See the picture below: