Loss function/error measurement for allocation problem

Question

I'm working on a prediction rule for an allocation problem. So, it's data like this:

| group | allocation (actual)  | allocation (predicted) |
|-------|----------------------|------------------------|
| 1     | 0.2                  | 0.15                   |
| 1     | 0.5                  | 0.6                    |
| 1     | 0.3                  | 0.25                   |
| 2     | 0.1                  | 0.0                    |
| 2     | 0.2                  | 0.4                    |
| 2     | 0.4                  | 0.45                   |
| 2     | 0.3                  | 0.15                   |
| ...   | ...                  | ...                    |

The distribution of actual allocations for each group happens to be highly skewed:

I'm unsure which loss function to use to tune my prediction rule. I definitely care about discrimination of very small values from large ones, but I don't really care whether values above 0.25 are accurately predicted as long as they end up in a > 0.25 bucket.

Among the various forms of squared error (MSE, RMSE, weighted MSE, etc.) and absolute error (MAE, MAD), is there one that would make the most sense for my purposes?

Answer 1

How about some piecewise functions on Loss like Humber Loss

$$ L_\delta(y, F) = \begin{cases} \frac{1}{2}(y - F)^2 & \textrm{for } |y - F| \le \delta, \\ \delta\, |y - F| - \frac{1}{2}\delta^2 & \textrm{otherwise.} \end{cases} $$

Basically it says, if the difference between predicted value and actual value is small, then use squared loss, if it is large, then use something else. Here is the example from this tutorial.