How xgboost uses weight in the algorithm

Question

Is passing weight as a parameter to the xgb.DMatrix same as multiplying our predictor (say y) by the weight ?

In more detail, I have a dataset which has the number an accident with 3 possible values, 0, 1, 2. And I want to weight it by the number of days per year the user has been driving, which has values like 1/365, 2/365 ... 364/365, and 365/365.

y = [0, 1, 0, 0, 2, 0, 0,1] weight = [1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365 ] My question is, if I convert y to y/weight, and pass to xgboost without any weight, is it same as just passing y with weight ?

Note that my objective = count:poisson

Answer 1

It won't be the same. Check this for how XGBoost handles weights:

https://github.com/dmlc/xgboost/issues/144

Weighting means increasing the contribution of an example (or a class) to the loss function. That means the contribution of the gradient of that example will also be larger. That's why (as you will see in the discussion I linked above) xgboost multiplies the gradient and the hessian by the weights, not the target values.

Answer 2

It is not the same. If you use y/weight all examples will be equally weighted. If you want to put more emphasis on examples, you need to specify a vector with weights.

a small example based on your y and weights.

y <- c(0, 1, 0, 0, 2, 0, 0,1)
weights <- c(1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365, 165/365)
set.seed(424)
x <- matrix(sample(seq(0, 24, by = 0.1), size = 24, replace = TRUE), 8, 4)


xgboost(data = x, label = y, weight = weights,
        max_depth = 2, eta = 1, nthread = 2, nrounds = 2,
        objective = "count:poisson")

Answer 3

Example weighting is the exactly the same as replication (assuming integer weights). So in your case, if weight = [1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365 ], it's the same as if there was one copy of the first example, 31 copies of the second example and so on. Notice that doesn't affect the target value in anyway, it stays the same.