Timeline for Logistic Regression is a Convex Problem but my results show otherwise?
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 31, 2019 at 18:13 | comment | added | soloice | I tried 2 experiments: Firstly I made a toy dataset without perfect separation, the non-convex surface still exists. Secondly, I removed the special treatment for |arg| > 18 in the sigmoid function, the loss surface looks convex. Now it's clear: @Commodore's answer is correct. The non-convexity is caused by approximation of sigmoid function in (-inf, -18] and [18, inf). | |
| Mar 31, 2019 at 17:56 | comment | added | soloice | This logistic-model-maximum-likelihood webpage says the log-likelihood of LR is always concave (i.e.: the cost function is convex) as long as the design matrix is of full rank. Convex algebra also tells us the cost function of LR is convex (ln (1 + exp(x_i \beta)) is the log-sum-exp of affine functions) + convex (y_i \beta x_i is an affine function) = convex function. | |
| Mar 31, 2019 at 17:45 | comment | added | soloice | This is weird. According to is-there-any-intuitive-explanation-of-why-logistic-regression-will-not-work-for, the complete separation problem only leads to a degenerated solution (solution goes to infinity) instead of turning the loss surface from concave to non-concave. | |
| Jan 2, 2019 at 2:48 | vote | accept | RMurphy | ||
| Jan 2, 2019 at 2:47 | vote | accept | RMurphy | ||
| Jan 2, 2019 at 2:48 | |||||
| Aug 2, 2017 at 23:23 | review | First posts | |||
| Aug 3, 2017 at 0:49 | |||||
| Aug 2, 2017 at 23:21 | history | answered | Hyco | CC BY-SA 3.0 |