1
$\begingroup$

I want to try if using Pearson Correlation as custom objective in the XGBoost R package gives me better results than the MSE. The idea was already referenced here and here, and that it might or might not be helpful when modelling on a highly noisy output. What would the custom objective function look like for correlation, and especially what would the gradient and hessian be?

$\endgroup$

1 Answer 1

2
$\begingroup$

A separable loss function computes the loss for each point individually & independently: $L(y_i, \hat{y_i})$ for all $i$. XGBoost requires separable loss functions. The xgboost documentation says

XGBoost is designed to be an extensible library. One way to extend it is by providing our own objective function for training and corresponding metric for performance monitoring. This document introduces implementing a customized elementwise evaluation metric and objective for XGBoost.

I've bolded the part indicating that XGBoost requires separability.

The Pearson correlation is computed as

$$ r_{xy}=\frac{\sum_{i=1}^n(x_i - \bar x)(y_i - \bar y)}{\sqrt{\sum_{i=1}^n(x_i - \bar x)^2}\sqrt{{\sum_{i=1}^n(y_i - \bar y)^2}}} $$ which is not separable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.