Timeline for Gradient descent or not for simple linear regression
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| 22 hours ago | history | edited | Sycorax♦ | CC BY-SA 4.0 |
added 142 characters in body
|
| Dec 28, 2018 at 16:05 | comment | added | Sycorax♦ | @anu Solving logistic regression in a non-iterative way requires minimizing a non-linear system of equations; in general, this is hard! This situation is analogous to the Abel-Ruffini theorem (no algebraic solution to roots of a 5th degree polynomial): we simply don't have direct computation methods to solve the system exactly. IIRC, this is discussed in Elements of Statistical Learning's chapter about logistic regression. There's probably a thread somewhere on stats.SE about it as well, but I'm having trouble finding a good one. | |
| Dec 28, 2018 at 5:17 | comment | added | Anu | @Sycorax, Can you suggest a reason why we can't use Normal equations method(or other direct solution methods) for logistic regression ( why you highlighted that it requires iterative updates! in my understanding the only difference between linear & logistic regression. is there objective functions!). Any explanation or pointing to the right resource would be helpful! | |
| Apr 27, 2018 at 2:02 | comment | added | Oliver Angelil | I asked a follow-up question: stats.stackexchange.com/questions/343069/… | |
| Apr 27, 2018 at 1:01 | comment | added | Sycorax♦ | @OliverAngelil be careful, though. In general, you might have even fewer. Without careful implementation, you could get a garbage result and never know it. | |
| Apr 27, 2018 at 0:59 | comment | added | Oliver Angelil | 6 decimal places is more than enough for me! | |
| Apr 27, 2018 at 0:41 | comment | added | Sycorax♦ | Numerically stable algorithms in double precision floating point should match the exact answer to 15 decimals. Matching to merely six implies a loss of 9 decimal digits of precision! | |
| Apr 26, 2018 at 23:45 | comment | added | Oliver Angelil | So are the "normal equations" used in statistical software when there's only 1 predictor variable? For n = 100, I get identical (to 6 decimal places) b0 and b1 coefficients when I use the normal equations vs the LinearRegression function in scikit-learn. Although I'm confused: #3 in the link states that the "normal equations" are a "TERRIBLE" idea?? | |
| Apr 26, 2018 at 21:14 | comment | added | Matthew Gunn | @OliverAngelil The "normal equations" are indeed the jargon term for the linear system of equations that are the first order conditions for the ordinary least squares optimization problem. | |
| Apr 26, 2018 at 21:10 | comment | added | Oliver Angelil | In the link you supplied, does #3: the "Normal equations", refer to the equations in my question here? If not, what is the technical term for these equations? | |
| Apr 26, 2018 at 21:03 | vote | accept | Oliver Angelil | ||
| Apr 26, 2018 at 20:30 | history | answered | Sycorax♦ | CC BY-SA 3.0 |