Timeline for Do we need gradient descent to find the coefficients of a linear regression model?
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 31, 2015 at 17:41 | comment | added | Mark L. Stone | QR was one of the top 10 algorithms of the 20th century. But time marches on, and even though effective algorithms for computing SVD date back to the 1960s, you have to look at the importance of the application areas. Therefore I believe SVD is the TOP algorithm of the 21st century. Quite frankly, have you ever heard of QR being used to recommend movies? No, SVD is used for that critical appiication. SVD clearly is the algorithm of choice when Twitter sends out unsolicited recommendations to conservative old geezers as to which teenage celebrities they should follow. Let's see QR do that!!! | |
| Jul 31, 2015 at 17:03 | comment | added | Vikas Raturi | @usεr11852 yes ofcourse. That because, i wanted to keep the answer simple, so as to avoid concepts such as QR decompostion, remaining relevant to the domain of Ng's course level. | |
| Jul 31, 2015 at 14:26 | comment | added | usεr11852 |
You want to solve $Ax=b$. Given $A=QR$ where $R$ is upper triangular and $Q$ is orthogonal, $ Ax=b \Rightarrow QRx=b \Leftrightarrow Rx = Q^Tb$ which can be solved by backward substitution (ie. fast) as $R$ is triangular and $Q^T Q=I$. For very large matrices (millions of entries) this can be more expensive than an iterative solver eg. SGD. As most people do not have very large matrices the QR decomposition is better. In general QR decomposition has shaped the numerical world; SIAM selected it as one of the top10 algorithms of the 20th century.
|
|
| Jul 31, 2015 at 14:08 | comment | added | Victor | Can you elaborate what is QR decomposition? | |
| Jul 31, 2015 at 12:59 | comment | added | usεr11852 |
While not wrong I think your answer misses the bigger picture by focusing on a non-standard case. The vast majority of linear regression models are fitted by using QR decomposition employing a closed form solution. GD -gradient decent- is used as an example to introduce more advanced methods (eg. SGD - stochastic GD).
|
|
| Jul 31, 2015 at 12:08 | review | First posts | |||
| Jul 31, 2015 at 12:17 | |||||
| Jul 31, 2015 at 12:07 | history | answered | Vikas Raturi | CC BY-SA 3.0 |