Timeline for Importance of optimizing the correct loss function
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 14, 2018 at 18:28 | history | edited | Glen_b | CC BY-SA 4.0 |
added 2 characters in body
|
| Jan 28, 2014 at 20:37 | vote | accept | utdiscant | ||
| Jan 27, 2014 at 15:20 | comment | added | Glen_b | @Erik I didn't choose to discuss the comparison of MSE with mean absolute error 'because it is more robust', I used that comparison because it's the specific pair of criteria mentioned in the second paragraph of the question. The OP posits the case where he uses MSE and the criterion of the competition is MAE and asks about the importance of using the correct one (MAE). I show that it can potentially be very important to use MAE, because MSE can be very bad when you are being judged by how you do on MAE. | |
| Jan 27, 2014 at 14:53 | comment | added | Erik | Good point, though I read "correct" more as being the one that one is interested later on than the one that will yield a robust model for the particular usecase. Otherwise penalization is an excellent example for optimizing a different error metric than the one which one uses to measure the quality of the predictions on new data, which I felt was not was the OP was after by my reading from the original post. | |
| Jan 27, 2014 at 14:46 | comment | added | Glen_b | This suggests that it may make a huge difference. In the absence of clearer indication of what situations we might face, considering worst cases is a reasonable response. | |
| Jan 27, 2014 at 14:39 | history | edited | Glen_b | CC BY-SA 3.0 |
added 133 characters in body
|
| Jan 27, 2014 at 14:33 | comment | added | Glen_b | In respect of the question being asked, "what can be said about the importance of optimizing the "correct" loss function?" my example addresses that question very clearly and directly by showing that the two can be arbitrarily different. | |
| Jan 27, 2014 at 14:30 | comment | added | Erik | I think this goes too much into the direction of robust methods; if we can expect outliers of this magnitude in the next data set the MSE predictions may still be better when it comes to minimizing the MSE of the predictions. | |
| Jan 27, 2014 at 14:20 | history | answered | Glen_b | CC BY-SA 3.0 |