I have a dataset with ~100K samples and non-negative continuous target variable. 99% of target values are zeros and the remaining 1% are right-skewed. Here are the deciles (0 and 1 correspond to min and max):
0.0 1.990
0.1 14.971
0.2 19.990
0.3 27.294
0.4 37.980
0.5 47.990
0.6 62.174
0.7 84.512
0.8 123.094
0.9 270.337
1.0 2933.610
So here's my problem. I need to choose an error function but whichever I take has certain drawbacks for my data. The dataset represents mobile apps' revenue and I believe that there are some common approaches to such data. Any ideas and recommendations will be highly helpful.
Here are my concerns:
- MSE: I suppose that it will punish right-most values;
- MAE: an error of 100 when predicting 2000 and when predicting 20 are intuitively two different errors and the second one should be punished more;
- MAPE: I'd like to use MAPE but there are two serious issues:
- it doesn't work with zero targets and I definitely need zeroes;
- it can punish overestimation much more than underestimation and I believe that both types of errors should be treated the same;
- MSLE (mean squared logarithmic loss): it seems to solve above-mentioned problems with MAPE but I feel it highly unintuitive: an error of 3 means that my prediction might be 3 times smaller or 3 times larger than the true value: how should I use it?
Whichever loss function I consider, I face either 99% of zeros or long right skew. Possibly I miss some function that nicely fits this situation? Or, may be< I should somehow separate the two cases and punish model for non-predicting zero in one way and for errors in predicting non-zero in some other? What are the best practices for such data?
Thanks in advance.