Timeline for Is there an error metric that decreases the weight when the target is near zero?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 13, 2023 at 16:04 | comment | added | Nuclear Hoagie | @schefflaa But the use of addition just adds a fixed value to the error term based only on the y_true values and regardless of what the model predictions actually are. Every model will see its error value increase by some constant, so it won't help you discriminate between models. Two models that have the same set of errors but distributed over different samples will get scored the same by using addition, but you want to score those models differently based on whether the errors fall on the samples with y_true=0 or not. | |
| Apr 13, 2023 at 15:25 | comment | added | schefflaa |
Yeah you're right in this. I should have clarified my question more. As were calculating the mean of the error of our samples, for example, [50, 12, 0] the error of 0 is a very good prediction and counts the same way to our mean as the other two values. Your formula would then, as i was trying to say, result in 0 for a horrendous prediction if y_true=0, which then counts the same towards the other samples. The formula I posted in the orignial question would overcome this through addition.
|
|
| Apr 13, 2023 at 13:32 | comment | added | Nuclear Hoagie | @schefflaa Yes, I thought that was the goal? The question says you want something for "situations where predicting zero values accurately is considered less important than predicting non-zero values". Here, predicting values whose true value is close to zero is less important than predicting values far from zero, and the predictions for true values that are exactly zero don't matter at all. You could add some small epsilon value to the y_true in this formula if you want zero-valued samples to contribute small amounts of error rather than none at all. | |
| Apr 13, 2023 at 11:56 | comment | added | schefflaa |
wouldn't this, for y_true=0, even for horrendous predictions, result in an error of 0 for the sample? The multiplicative part is still strange to me
|
|
| Apr 11, 2023 at 20:26 | comment | added | Nuclear Hoagie | @schefflaa Whoops, I misread the exponential there - I updated the formula, we can scale the error values by y_true itself. When y_true=0, the sample does not contribute any error no matter how bad the prediciton is. | |
| Apr 11, 2023 at 20:25 | history | edited | Nuclear Hoagie | CC BY-SA 4.0 |
deleted 16 characters in body
|
| Apr 11, 2023 at 20:01 | comment | added | schefflaa |
but wouldn't this, for a big enough y_true, essentially set the error to 0 regardless of (y_true - y_pred)?
|
|
| Apr 11, 2023 at 15:54 | history | answered | Nuclear Hoagie | CC BY-SA 4.0 |