Timeline for Why and what induces degeneracy in the solution space in logistic regression models with respect to the energy landscape of the logistic loss?
Current License: CC BY-SA 3.0
20 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 1, 2018 at 22:43 | comment | added | Charlie Parker | related question, might be useful: stats.stackexchange.com/questions/12492/… | |
| Feb 1, 2018 at 22:41 | comment | added | Charlie Parker | note for the limit minimizer (for the separable case), refer to the following paper: arxiv.org/abs/1710.10345 | |
| Feb 1, 2018 at 1:21 | history | edited | Charlie Parker | CC BY-SA 3.0 |
added 384 characters in body
|
| Feb 1, 2018 at 1:16 | history | edited | Charlie Parker | CC BY-SA 3.0 |
added 384 characters in body
|
| Feb 1, 2018 at 0:27 | answer | added | Sycorax♦ | timeline score: 2 | |
| Jan 31, 2018 at 22:44 | comment | added | Charlie Parker | @Sycorax well, as far as I can tell its not a linear system (there is softmax/logistic function in the way of the linearity and a $-log$ in the loss function). So its not entirely clear to me what even the nullspace is or where it comes from or how one even proves its non-empty rigorously. I'm in the search of formal answers if possible. | |
| Jan 31, 2018 at 19:14 | comment | added | Sycorax♦ | If I'm reading this correctly, you understand how a nonempty null space causes degeneracy in linear regression, but do not understand how a nonempty null space causes degeneracy in logistic regression? | |
| Jan 31, 2018 at 17:34 | history | edited | Charlie Parker | CC BY-SA 3.0 |
added 781 characters in body; edited title
|
| Jan 31, 2018 at 16:49 | comment | added | Charlie Parker | @Sycorax plus the question only talks about the data while I care to characterize the complexity of the model was well to specify the degeneracy of the solution space. | |
| Jan 31, 2018 at 16:48 | comment | added | Charlie Parker | @Sycorax I'm not sure how that question u linked is helpful. It doesn't explain anything, nor does it address any of the points I brought up well. It lacks a lot of quality. | |
| Jan 29, 2018 at 22:42 | comment | added | Charlie Parker | @Sycorax ah, that looks more promising. I will check that out later and see if it answers my Q. Thanks for taking the time to check my question btw :) | |
| Jan 29, 2018 at 22:40 | comment | added | Sycorax♦ | In that case, I think this answer stats.stackexchange.com/questions/12492/… is what you're looking for -- i.e. unique solutions require a full-rank design matrix. | |
| Jan 29, 2018 at 21:36 | history | edited | Charlie Parker | CC BY-SA 3.0 |
edited title
|
| Jan 29, 2018 at 21:32 | comment | added | Charlie Parker | @Sycorax maybe this will help. I am not interested in the classification error. I am interested in the loss, so the actual landscape we are optimizing over and the number of minimizers that arise from that. Thats why separability is not necessarily interesting to me. If we need to impose it to make the analysis easier so be it, but separability is not the goal. I also remove the tag separation because its not relevant, this is a optimization problem technically. | |
| Jan 29, 2018 at 21:31 | history | edited | Charlie Parker | CC BY-SA 3.0 |
added 488 characters in body
|
| Jan 29, 2018 at 21:00 | comment | added | Charlie Parker | @Sycorax why do you think those questions are helpful? My question is not about separation or separability. My question is about the energy landscape and how it looks like and the mathematical conditions for there to be a unique minimizer or not. Its obvious that the error can't be zero for the logistic loss because of $ log(1+e^{-yw^\top x}) $, so non-convergence is obvious too (which I point out in my question itself). In short, my question is more interested about mathematical conditions of how the landscape for logistic regression looks like when its degenerate. | |
| Jan 29, 2018 at 19:34 | review | Close votes | |||
| Jan 30, 2018 at 11:53 | |||||
| Jan 29, 2018 at 19:14 | comment | added | Sycorax♦ | Possible duplicate of Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and stats.stackexchange.com/questions/5354/… but also see the tag description separation (which I have added) for a concise explanation. | |
| Jan 29, 2018 at 19:13 | history | edited | Sycorax♦ | CC BY-SA 3.0 |
edited tags
|
| Jan 29, 2018 at 18:27 | history | asked | Charlie Parker | CC BY-SA 3.0 |