10
$\begingroup$

Is passing weight as a parameter to the xgb.DMatrix same as multiplying our predictor (say y) by the weight ?

In more detail, I have a dataset which has the number an accident with 3 possible values, 0, 1, 2. And I want to weight it by the number of days per year the user has been driving, which has values like 1/365, 2/365 ... 364/365, and 365/365.

y = [0, 1, 0, 0, 2, 0, 0,1] weight = [1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365 ] My question is, if I convert y to y/weight, and pass to xgboost without any weight, is it same as just passing y with weight ?

Note that my objective = count:poisson

$\endgroup$
13
$\begingroup$

It won't be the same. Check this for how XGBoost handles weights:

https://github.com/dmlc/xgboost/issues/144

Weighting means increasing the contribution of an example (or a class) to the loss function. That means the contribution of the gradient of that example will also be larger. That's why (as you will see in the discussion I linked above) xgboost multiplies the gradient and the hessian by the weights, not the target values.

$\endgroup$
3
$\begingroup$

It is not the same. If you use y/weight all examples will be equally weighted. If you want to put more emphasis on examples, you need to specify a vector with weights.

a small example based on your y and weights.

y <- c(0, 1, 0, 0, 2, 0, 0,1)
weights <- c(1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365, 165/365)
set.seed(424)
x <- matrix(sample(seq(0, 24, by = 0.1), size = 24, replace = TRUE), 8, 4)


xgboost(data = x, label = y, weight = weights,
        max_depth = 2, eta = 1, nthread = 2, nrounds = 2,
        objective = "count:poisson")
$\endgroup$
1
  • $\begingroup$ thanks @phiver Do you know how xgboost threats weights inside the algorithm ? Thanks $\endgroup$
    – Paba
    Feb 8 '18 at 12:06
0
$\begingroup$

Example weighting is the exactly the same as replication (assuming integer weights). So in your case, if weight = [1/365, 31/365, 60/365, 20/365, 3/365, 50/365, 32/365 ], it's the same as if there was one copy of the first example, 31 copies of the second example and so on. Notice that doesn't affect the target value in anyway, it stays the same.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.