Timeline for Enormous coefficients in logistic regression - what does it mean and what to do?
Current License: CC BY-SA 4.0
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 9, 2021 at 23:29 | history | edited | James Stanley | CC BY-SA 4.0 |
Updated broken link
|
| Feb 3, 2013 at 21:35 | history | edited | James Stanley | CC BY-SA 3.0 |
Added in some missing words on re-read
|
| Feb 3, 2013 at 21:28 | comment | added | James Stanley | I'm not very experienced in cross-validation -- have only done it once. You might have more luck getting an answer on your other question (and I think any potential answer would work better as an answer on that newer thread, to do it justice.) | |
| Feb 3, 2013 at 11:40 | comment | added | Tomas | James, one more question. Do you think that those model with quasi-complete separation would perform well in cross-validation or not? Because I tried it and the cross-validation seems to perform well (if I interpret it correctly). | |
| Feb 2, 2013 at 19:20 | comment | added | James Stanley | I'm not sure whether there is a technical term beyond quasi-complete separation. I would say "to avoid quasi-complete separation (due to sparse data in combinations of the two factors) we did not test for interactions". Obviously this is pretty much all jargon, but I think this might be the best description? | |
| Feb 2, 2013 at 18:38 | comment | added | Tomas | James, and have you found the exact term? I need to write to the paper something like: "In order to avoid [what?? overstretching/overfitting/...?] we do not test interactions." Thanks in advance! | |
| Feb 2, 2013 at 14:19 | vote | accept | Tomas | ||
| Jan 29, 2013 at 20:16 | comment | added | James Stanley | I don't know much about VIF, and have never used those methods: sorry! | |
| Jan 29, 2013 at 9:47 | comment | added | Tomas | Thanks James. What do you think about the high VIF coefficients (please see my comment above)? | |
| Jan 29, 2013 at 1:53 | history | edited | James Stanley | CC BY-SA 3.0 |
added some pragmatic options
|
| Jan 29, 2013 at 1:48 | comment | added | James Stanley | I think this is just a question of terminology/jargon -- what you are describing is still a problem, and is due to overspecification, but I don't think we would refer to this as "overfitting" in a formal sense. I'll have to go away and read some bits on the distinctions to be clearer! | |
| Jan 29, 2013 at 1:46 | comment | added | Tomas | thanks James - but this is exactly what I imagine under the term Overfitting.. BTW, I used VIF and got enormous values too, please see my edited question. Does this tell you something new about multicollinearity/overfitting issues? | |
| Jan 29, 2013 at 1:39 | comment | added | James Stanley | I don't think this is technically "overfitting", but a case of overstretching your model. See e.g. Wikipedia on what is generally meant by overfitting (and I won't pretend to be an expert on the definition): en.wikipedia.org/wiki/Overfitting -- that an overspecified model is one where the parameters estimated would likely not perform well in cross-validation, or in other words, the model you've specified will describe this sample, but would not work well on another sample from the same population. | |
| Jan 29, 2013 at 1:32 | comment | added | Tomas | thanks James. So does this actually mean overfitting? Does this mean that I should possibly not include the interactions into the model? | |
| Jan 29, 2013 at 1:25 | history | answered | James Stanley | CC BY-SA 3.0 |