1
$\begingroup$

I'm working on a prediction rule for an allocation problem. So, it's data like this:

| group | allocation (actual)  | allocation (predicted) |
|-------|----------------------|------------------------|
| 1     | 0.2                  | 0.15                   |
| 1     | 0.5                  | 0.6                    |
| 1     | 0.3                  | 0.25                   |
| 2     | 0.1                  | 0.0                    |
| 2     | 0.2                  | 0.4                    |
| 2     | 0.4                  | 0.45                   |
| 2     | 0.3                  | 0.15                   |
| ...   | ...                  | ...                    |

The distribution of actual allocations for each group happens to be highly skewed:

density

I'm unsure which loss function to use to tune my prediction rule. I definitely care about discrimination of very small values from large ones, but I don't really care whether values above 0.25 are accurately predicted as long as they end up in a > 0.25 bucket.

Among the various forms of squared error (MSE, RMSE, weighted MSE, etc.) and absolute error (MAE, MAD), is there one that would make the most sense for my purposes?

$\endgroup$
1
$\begingroup$

How about some piecewise functions on Loss like Humber Loss

$$ L_\delta(y, F) = \begin{cases} \frac{1}{2}(y - F)^2 & \textrm{for } |y - F| \le \delta, \\ \delta\, |y - F| - \frac{1}{2}\delta^2 & \textrm{otherwise.} \end{cases} $$

Basically it says, if the difference between predicted value and actual value is small, then use squared loss, if it is large, then use something else. Here is the example from this tutorial.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.