Usually my job is to do classification but recently I have a project which requires me to do regression. That is to say, my response variable is not a binary True/False, but a continuous number. the response is a well formed Gaussian centered at zero.
I usually use XGboost for my classification tasks. To do a regression, I found the following from xgboost's manual:
this is the only difference with classification, use reg:linear to do linear regression
when labels are in [0,1] we can also use reg:logistic
objective = reg:linear
I guess it means: regression, using a linear model
However, I looked at the output and I found that the output is still a set of trees. As we know, tree is no linear. So what does reg:linear really mean? From my perspective, there isn't any "linear regression" here.
Can anyone provide any insight here?