Are there any ways to upweight any particular feature for GBM (tree boosting)?

The motivation is:

Assuming one would like to put a few dummy indicators, e.g., X_A, X_B, X_C, ..., into GBM, for the original feature X that is categorical. The data is pretty thin for X == 'A'.

Normally GBM would treat each feature equally at each tree split, and as a result it is very likely that GBM would not split using X_A at all if on training sample the residual for X == 'A' is not significant enough.

However, if one already knows the residual of X == 'A' observed on the training sample is actually real based on some other knowledge (business, prior, etc.), is there any way to force GBM to upweight this particular feature X_A when splitting so that the output model can actually fit well to the observed residual?



1 Answer 1


Not sure if you know much about offsets, but it sounds like you could tell the GBM to fit the effect you're certain of as an offset.

y~ x1 + x2 + offset(some function of X_A)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.