I am trying to get direct connection between Gain and Logloss for XGBoost. It looks to me that in Xgboost paper formula 6:
for a model with depth 1 and number of trees=1 this formula contains Similarity score for the node. On the other hand formula 6 approximates minimum loss and should be equal to logloss value of the model.
However when I calculate by hands at google spreadsheet these two values are different. Please have a look at:
- sheet "One tree depth2"
- Logloss in Cell K47=0,30 (which matches python code below)
- Similarity score for the node Cell O47=0,05 - these values doesn't match.
Code to train model which return values mentioned above (you have to download input data from google spreadsheet at sheet "raw_data":
df = pd.read_excel('Xgboost calculation.xlsx', sheet_name='raw_data') x=df[[0,1]].values y=df['TARGET'].values model = xgb.XGBClassifier(verbosity = 0,n_estimators=1,max_depth=1,learning_rate=1,reg_alpha=0,reg_lambda=0,subsample=1, min_child_weight=0) model.fit(x, y,eval_set=[(x, y)])
My question is why doesnt Logloss match Similarity score in this example? Is it due to taylor's approximation?