2
$\begingroup$

I want to develop a customized objective function with weights given by both label and prediction, for Xgboost.

Example, let's say you have 2 classes

I want to assign a penalties according to this cases:

  1. predicts=0, label=1 => assign big penalty (so the algorithm focuses on preventing that specific error).

  2. predicts=1, label=0 => assign small penalty (so the algorithm WON'T focus on preventing that specific error).

Mathematically I wish I had a confusion like matrix, where for each slot I could assign the penalties for the multi-class case.

And if I did the classic threshold solution it won't work for test cases as I won't have the labels!

So far I modified a log-likelihood objective by making labels and the gradient bigger on certain cases, see https://stackoverflow.com/questions/34178287/difference-between-objective-and-feval-in-xgboost):

# user define objective objective, given prediction, return gradient and second order gradient
my_custom_obj <- function(preds, dtrain) {
  labels <- getinfo(dtrain, "label")

  f_scale_labels <- function(p, l) {

    if (p == 1 && l==0){
      return(l*10)
    }else if (p == 0 && l==1){
      return(l*.1)
    }

    return(l)
  }

  scaled_labels <- unlist(map2(preds, labels, f_scale_labels))

  preds <- 1/(1 + exp(-preds))
  grad <- (preds - scaled_labels)
  hess <- preds * (1 - preds)
  return(list(grad = grad, hess = hess))
}

bst <- xgboost(.... objective = my_custom_obj)

It kind of works, but I am not satisfied.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.