Timeline for Why do we worry about overfitting even if "all models are wrong"?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 1, 2019 at 14:42 | comment | added | gerrit | Let us continue this discussion in chat. | |
| Nov 1, 2019 at 14:21 | comment | added | James | @gerrit The noise is the signal. You overfit until you capture all the noise so you recover the original data with the specific input (of the compressed signal) and don't care that it will be of no use with a different input (and in fact prefer that). | |
| Nov 1, 2019 at 14:03 | comment | added | gerrit | @James Yes — but then you're not overfitting. I can see how models can be used for compression, but I'm not sure how overfitting fits in there. In your example, a lossless compression will only work if the remaining data points fit the model perfectly, and for a lossy compression (perhaps there is noise on the quadratic curve) one needs again a model that generalises (interpolates) well such that using too many parameters would make the fit worse, isn't it? Overfitting does not always lead to improper generalisation. | |
| Nov 1, 2019 at 13:33 | comment | added | James | @gerrit I can give you 10,000 data points from a quadratic curve, but you can describe it entirely with just 3 parameters. | |
| Nov 1, 2019 at 13:02 | comment | added | gerrit | @James I admit I don't know much about compression or what overfitting implies for it, although it seems to me that storing an n-degree polynomial instead of (n+1) data points doesn't save much or any space. | |
| Nov 1, 2019 at 12:57 | comment | added | James | @gerrit Overfitted models are useful in tasks such as data compression, where the aim is reconstruction of the test data. F1 cars reconstruct the track. That is why they change the setup for each track rather than use a general setup for the whole season. | |
| Nov 1, 2019 at 12:29 | comment | added | gerrit | @Caleth The training points are not a forecast/prediction, they are a measurement. | |
| Nov 1, 2019 at 12:26 | comment | added | Caleth | @gerrit the overfitted model predicts n+1 points exactly. It's only useless elsewhere. | |
| Nov 1, 2019 at 12:13 | comment | added | gerrit | I don't find this a very good analogy. A severely overfitted model (such as an n-degree polynomial fitted to n+1 points) isn't useful for anything. A F1 is not overfitted, it is just a highly specialised tool useful for a very specific role. The statistical analogy would be a model that is trained and useful for a very specific type of forecasting, but not useful for other roles; such a model is not overfitted, just very limited in scope. | |
| Nov 1, 2019 at 9:52 | comment | added | leftaroundabout | @Dirk actually yes, quite literally, the problem with driving an F1 over an open field is that it's such a good fit for race tracks. Namely, it fits very well to the ground of a flat raceway (low ground clearance), but therefore isn't as flexible to also fit to anything non-flat. A normal car has more flexible suspension, which means it doesn't “stick to the pavement” as well but in return manages also some other tasks. — “There may be a car which is a great fit for both open fields and racetracks” – that would require very good active suspension, probably be heavy and therefore slower. | |
| Nov 1, 2019 at 6:24 | comment | added | Dirk | Devil's advocate: The problem with driving an F1 over an open field is not that the F1 is such a good fit for race tracks, (I could imagine that there may be a car which is a great fit for both open fields and racetracks), but that the F1 is just a bad fit for open fields. | |
| Oct 31, 2019 at 14:28 | history | edited | James | CC BY-SA 4.0 |
More closely aligned the analogy with modeling
|
| Oct 31, 2019 at 14:21 | history | edited | Nick Cox | CC BY-SA 4.0 |
added 1 character in body
|
| Oct 31, 2019 at 14:19 | history | answered | James | CC BY-SA 4.0 |