First off, apologies for any malformed parts of this question. I'm a very much a newbie and still very much learning. I've tried to do as much research as possible before asking here.
I'm building/tuning a model to solve a tabular regression problem and predict a price for a widget. Some notable properties of the data/business domain:
- The tabular data has a wide range of possible prices - from 10 dollars to 100,000+ dollars (a very small number are $1m+).
- The model is being built to predict optimal purchase price - so in this case, under-prediction is much preferred to over-prediction (paying too little is good - paying too much is bad).
I'm building this using ludwig (and may experiment with H20 or scikit-learn) but the key thing I have control over now is the loss function.
I've built the first version of this model using RMSLE (Root Mean Squared Logarithmic Error) as the loss function and achieved decent results. I chose RMSLE because it optimizes for ratio-based losses -- a prediction of 100 with actual price 110 is the same as a prediction of 10,000 with an actual price of 11,000, which fits my problem space.
However an issue I've uncovered with RMSLE are that it penalizes under-prediction more (which is the opposite of what I want) and tends to be more forgiving of large error. The curve looks like:
I would like to build a loss function whose properties are similar to RMSLE in that it's ratio-based and can handle both large and small predicted/actual prices, but with a curve that's more asymmetric and penalizes overprediction more than underprediction. I read a bit about Linear Exponental Loss and the curve looks right:
But I'm concerned it might not share the properties of being ratio-based like RMSLE does.
So I seek your collective wisdom for advice and guiance. Many many thanks in advance and again, apologies for any foolish/malformed portions of this question.