Timeline for Polynomial regression underfits data when degree becomes large
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 6 at 13:58 | comment | added | whuber♦ | No "hairy math" is needed. Simply notice that in double precision arithmetic, $1^{10}$ compared to $18^{10}$ is indistinguishable from zero. Your real problem stems from attempting to use such a high-degree polynomial in the first place. | |
| Jun 3, 2022 at 0:05 | comment | added | Ammar Rashed | That's a truly eluding issue that is not directly obvious unless you delve into the hairy math behind these models. The ill-conditioned matrix is a suspect, since you could end up with more parameters than observations if you are using interaction parameters too (i.e. $x^5y^5, x^4y^6, x^3y^7$ ...and so on). Notice also that more parameters does not necessarily mean higher marginal likelihood, which will result in suddenly underfitting the data after some point. I explained this a bit in the following answer > answer | |
| Nov 6, 2019 at 17:25 | review | Close votes | |||
| Nov 7, 2019 at 12:00 | |||||
| Nov 6, 2019 at 17:09 | comment | added | Dave | I found two links on CV: stats.stackexchange.com/questions/241703/… and stats.stackexchange.com/questions/258307/…. My suspicion is that the issue is numerical instability. | |
| Nov 6, 2019 at 16:49 | comment | added | FrankieYin | @Dave Hi, I haven't used them. In fact I don't think I learned that in class. | |
| Nov 6, 2019 at 16:45 | review | First posts | |||
| Nov 6, 2019 at 17:08 | |||||
| Nov 6, 2019 at 16:43 | comment | added | Dave | Have you used orthogonal polynomials? | |
| Nov 6, 2019 at 16:42 | history | asked | FrankieYin | CC BY-SA 4.0 |