# Is there a way to enforce factor importance in random forest/xgboost

Suppose I have 3 predictors to predict stock returns. 1 of the 3 is known for ages and is still doing well. The rest 2 are newly found ones. So in a crude portfolio construction fashion, I'd do $$s = 0.8s_1 + 0.1s_2 + 0.1s_3$$ and then take position proportional to $$s$$, to reflect that I'm more confident in the first factor than the other two.

Now I have say 50 factors and 5 of them are more traditional and hence more trustworthy (by my subjective opinion). When I try fitting a random forest or xgboost using all the factors, is there a way to tell the model that I trust some factors more than the others?

What I want is to control the resulting prediction to be at least $$x\%$$ correlated with the traditional factors. And of course, I'm happy to sacrifice some predictive power for that.

• Hmm... can you maybe reformulate this? Strictly speaking, "at least x% correlated with" is inapplicable for a model that is not explicitly models correlations. In the first instance, why not have two prediction functions, one ($f_{Trad}$) with the 5 "traditional predictors only" and one ($f_{All}$) with all 50 together and then use a voting regressor with weights $[x, 100-x]$? And note that even in that case if for example $f_{Trad}$ outputs $0$, and $f_{All}$ non-zeros, all the variability will be due to $f_{All}$... Dec 8, 2023 at 3:22
• Maybe replicate the more trustful columns a couple of times and then use column subsampling in XGBoost (random forest does it by default, except in Scikit-Learn)? Dec 8, 2023 at 10:00
• @MichaelM: Even then there is no guarantee it will be picked multiple times. Maybe that would work if the OP used Extremely Randomized Forest but that's not on the available algorithms per se. Dec 8, 2023 at 10:19
• This is true. However, if we pick features per split, and the column subsampling rate is 1/p (p is the number of features), then it will work better. Dec 8, 2023 at 10:44

1. Specify forced splits for certain features. Those would be applied at the root of each tree, before tree fitting begins (forcedsplits_filename in LightGBM)
3. Increase number of histogram bins for selected features (max_bin)