Real-world example of quantile loss used for evaluation

Question

We can use quantile loss (a.ka. tick or pinball loss) for training a model or for evaluating predictions. (It is helpful to distinguish the two clearly, e.g. as done here.) I am interested in the latter case. What would be a good example (preferably from business, economics or finance) of a prediction problem where losses of overprediction are linear in prediction errors and losses of underprediction are also linear in prediction errors, but the two lines have different slopes? In other words, what is some real-world example when we want to use quantile loss for evaluating predictions?

Answer 1

Demand forecasting: example 1

Suppose you a retailer. You have to forecast demand for the goods you are selling. Based on the forecast, you order $\hat n$ goods, and the demand turns out to be $n$. Let $e:=\hat n-n$.
From each good you sell, you make a profit of $r$. From each good you fail to sell, you incur a cost of $c$ (due to storage costs, partial depreciation or complete depreciation due to perishability).
In the ideal case, you would order $n$ goods, so that $e=0$, and then you would earn $nr$. In reality, you will likely end up with $e\neq 0$. If $e>0$, your loss (relative to the ideal scenario) is $ce$. If $e<0$, your loss is $r(-e)$. This is quantile loss.

Demand forecasting: example 2

You are a producer. You have some funds available, but you need to borrow some more to finance your production. The demand for your product, $n$, is uncertain, so you have to predict it. Your prediction is $\hat n$. Let the prediction error be $e:=\hat n-n$.
If you borrow too much, you will have paid unnecessary interest on the extra amount. With an interest rate $r$, you end up paying $re$, and that is your loss. If you borrow too little, you cannot produce as much as demanded, and you forgo a profit of $\pi$ per unit. Thus your loss is $\pi(-e)$. This is quantile loss.

Answer 2

Great question! Unfortunately I don't have a perfect answer, but hopefully this is a step in the right direction and will motivate a better answer from someone else. Mine is too simple to qualify as fully "real world", but it is intended to address the "preferably from business, economics or finance" part of your request.

Suppose you are a business holding an inventory of $N$ widgets, and imagine there are $M$ other widgets out there in the hands of others. Consider the following simple scenario:

• You are forced to announce a price $p$ per widget
• After that point, the price you announced is fixed, you cannot undo it (you can imagine this scenario describing a single day, or some fixed period of time)
• The true value $y$ per widget, previously unknown/random, is now realized and observed by all, including the holders of the $M$ other widgets
• If you announced $p > y$, you are forced to buy all $M$ widgets (you are buying at a loss)
• If you announced $p < y$, you are forced to sell your entire inventory of $N$ widgets (you are selling at at loss)

Your loss is therefore $M \left(p - y\right)$ if $p > y$, and $N \left(y - p\right)$ otherwise. This should coincide with the quantile loss function, and depending on the ratio of $M$ and $N$ (assuming these are fixed numbers you observe ahead of time), your optimal action will be to post a price $p$ equal to a specific quantile of the distribution of $y$.