I'm looking for a simple approach (e.g. defining a new target label / sample weights and then using some off-the-shelf regressor with some standard objective) for the following problem:
I want to predict a 90% lower bound for taxi demand in the next hour in some location, based on many features and previous demand. E.g. if I predict 4, it means that I'm 90% certain that there would be demand for at least 4 taxi rides.
problems I'm facing:
- The data is very imbalanced: most of the time (~85%) the demand is 0. This causes my evaluation metrics to be inflated (because it's very easy to be right for the weekends and nights, just predict 0), and possibly also confuses my regressors, as maybe they focus on these dead zones - perhaps busy with discerning between 0-1 rides which won't make any difference as both their lower bounds would probably be 0.
- I don't care much about under-estimation, e.g. if I my predicted lower-bound was 1 and the true demand was 3 it's not a big deal. However I do care a lot about over-estimation, e.g. If my predicted lower-bound was 3 and the true demand was 1 it's a disaster.
How can I transform this problem to something more standard?
I took a look at quantile regressors, pinball loss, weighting/re-sampling samples for regression - but only got more confused as I'm not sure how to pose my problem and objective in a way that would enable comparing these (partial) solutions.
