2
$\begingroup$

I am trying to predict the processing time of a process by using xgboost regression algorithm in python. However I realised that my samples data is skewed to left and my algorithm struggles to predict longer process times (right side of histogram). I want to oversample the values I lack currently.

Histogram of Process Time

I have been looking at SMOTE algorithm for oversampling. All the examples I could find worked categorical y values. But my y values are continuous and I want to produce similarly continuous values. My current variables have continuous and binary values in them.

Variable Data

Output Data

I want to oversample by creating new process times while keeping variables constant. Do you have any suggestions to oversample such data? Thanks.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
3
$\begingroup$

Oversampling will mainly bias your predictions. This thread looks at oversampling in the context of "unbalanced" classification, but it applies to your situation, too.

Your problem appears to be that some processes run longer not necessarily systematically, but because of residual variation. (Variability may also depend on predictors.) If it is important to you to be prepared for longer processing times, then you should not predict mean process times, but "worst cases", i.e., do quantile predictions and predict (say) a number such that 95% of your processes will take less time. This will give you a safety cushion. You can train your model for quantiles using the pinball loss. See also our tag.

Improve this answer
$\endgroup$
2
  • $\begingroup$ still, it would be handy to know such a technique. I am using data from two datasets, one a study of elderly people and the other a study of the general population. My model is regressing very well for the elderly people, and somewhat reasonable on the general population. Is there nothing more sophisticated then binning my regression target and oversampling that way? $\endgroup$
    – AyamGorengPedes
    Commented Dec 3 at 16:03
  • $\begingroup$ Binning is not good practice, quite apart from oversampling or not. Whether there is something "sophisticated" you can do will depend heavily on the problem at hand, and on your domain knowledge: stats.stackexchange.com/q/222179/1352. Some problems simply are harder, or have higher residual variation, so you simply cannot improve as much as you would like to. $\endgroup$
    – Stephan Kolassa
    Commented Dec 3 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.