Timeline for Why is gradient descent required?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 20, 2016 at 7:48 | comment | added | Matthew Gunn | A point some people may find confusing is how is solving a linear system an optimization problem? The answer of course is that solving a linear system can be reframed as minimizing a quadratic objective. Some iterative methods for solving linear systems are easier to understand from the perspective that they're minimizing a quadratic objective in an iterative fashion. (Eg. the Krylov subspace method conjugate gradient's step direction is based on the gradient... it's loosely related to gradient descent.) | |
| May 20, 2016 at 7:43 | comment | added | Matthew Gunn | Inverting a matrix is a bit of a strawman here as QR decomposition with partial pivoting is more accurate and faster, but yeah, QR is still $O(n^3)$. I agree that for sufficiently large systems (eg. > 10,000 variables) that can start becoming a problem. The modern, high tech approach is then to approximate the solution with iterative Krylov subspace methods (eg. conjugate gradient, GMRES). | |
| May 20, 2016 at 5:46 | vote | accept | Niranjan Kotha | ||
| May 16, 2016 at 0:59 | history | edited | Danica | CC BY-SA 3.0 |
formatting
|
| May 16, 2016 at 0:40 | history | answered | jpmuc | CC BY-SA 3.0 |