I want to try if using Pearson Correlation as custom objective in the XGBoost R package gives me better results than the MSE. The idea was already referenced here and here, and that it might or might not be helpful when modelling on a highly noisy output. What would the custom objective function look like for correlation, and especially what would the gradient and hessian be?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
A separable loss function computes the loss for each point individually & independently: $L(y_i, \hat{y_i})$ for all $i$. XGBoost requires separable loss functions. The xgboost documentation says
XGBoost is designed to be an extensible library. One way to extend it is by providing our own objective function for training and corresponding metric for performance monitoring. This document introduces implementing a customized elementwise evaluation metric and objective for XGBoost.
I've bolded the part indicating that XGBoost requires separability.
The Pearson correlation is computed as
$$ r_{xy}=\frac{\sum_{i=1}^n(x_i - \bar x)(y_i - \bar y)}{\sqrt{\sum_{i=1}^n(x_i - \bar x)^2}\sqrt{{\sum_{i=1}^n(y_i - \bar y)^2}}} $$ which is not separable.
