Timeline for Adding uncorrelated variables to glm increases AIC
Current License: CC BY-SA 4.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 27, 2019 at 11:15 | comment | added | Jean Paul | Here is my new question: stats.stackexchange.com/questions/428974/… | |
| Sep 27, 2019 at 9:26 | vote | accept | Jean Paul | ||
| Sep 27, 2019 at 9:26 | vote | accept | Jean Paul | ||
| Sep 27, 2019 at 9:26 | |||||
| Sep 27, 2019 at 9:26 | vote | accept | Jean Paul | ||
| Sep 27, 2019 at 9:26 | |||||
| Sep 27, 2019 at 9:04 | comment | added | mkt | @JeanPaul I don't understand your new claim about AIC, but it seems sufficiently different that it may be worth asking a new question about that as well. | |
| Sep 27, 2019 at 8:49 | comment | added | Jean Paul | My primary goal was not to compare AIC and adjusted $R^2$, I just used $R^2$ to better discern the limitation of AIC. I think the limit of AIC is the failure to be generalized outside of the current sample, so it cannot say in absolute if a model is better than another. It is nevertheless an use that I saw in the literature. | |
| Sep 27, 2019 at 4:54 | comment | added | mkt | @JeanPaul If you wish to ask a question about comparing AIC and adjusted $R^2$, you are welcome to do so. I believe my present answer has addressed the question you posed. | |
| Sep 26, 2019 at 21:55 | comment | added | Jean Paul | One limitation that I find for AIC compared to adjusted $R^2$ is that AIC is dependent on sample size in the sense that adding a variable with a very noisy but existent signal will not add useful information if the sample size is too small so the AIC will increase. But with a large enough sample size, the same variable will provide some useful information so the AIC will decrease. One does not have this issue for adjusted $R^2$: even with small sample sizes, it will be positive in mean and it will stay the same in mean when the sample size increases. | |
| Sep 26, 2019 at 15:05 | comment | added | mkt | @JeanPaul You are correct that adjusted $R^2$ also penalises models for complexity. However, this penalty is not very large and the claim that "...adjusted $R^2$ only represents the true information that is captured by the model" is not accurate. See this comment and the associated answer for more information. | |
| Sep 26, 2019 at 14:59 | comment | added | mkt | @JeanPaul "log-likelihood will increase due to overfitting but not AIC thanks to the 2k penalty". Yes, exactly. Your error is in the earlier quotation "AIC... [will] stay the same in mean, but it appears to correct more than that". It will not stay the same - it will get worse, and that is by design. If it stayed the same when you added an uncorrelated variable, there would be no use in AIC at all. | |
| Sep 26, 2019 at 14:27 | comment | added | Jean Paul | I meant it compensates overfitting in the sense of the formula $\mathit{AIC} = 2k - 2\ln(L)$: if one adds an uncorrelated random variable, log-likelihood will increase due to overfitting but not AIC thanks to the 2k penalty. In fact, adjusted $R^2$ does not increase after adding uncorrelated variables because it also corrects for overfitting. From my understanding, the difference between adjusted $R^2$ and AIC is that adjusted $R^2$ only represents the true information that is captured by the model, whereas AIC also penalizes for false information. | |
| Sep 26, 2019 at 7:12 | history | edited | mkt | CC BY-SA 4.0 |
added 131 characters in body
|
| Sep 25, 2019 at 17:50 | history | answered | mkt | CC BY-SA 4.0 |