Timeline for When should one use Coordinate descent vs. gradient descent?
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| S Nov 2, 2022 at 17:15 | history | suggested | user343233 | CC BY-SA 4.0 |
update URL
|
| Nov 1, 2022 at 14:16 | review | Suggested edits | |||
| S Nov 2, 2022 at 17:15 | |||||
| Sep 9, 2020 at 6:26 | comment | added | littleO | ista (the proximal gradient method) converges like $1/k$, not $1/k^2$, right? Accelerated proximal gradient methods such as FISTA converge like $1/k^2$. | |
| Apr 12, 2020 at 15:25 | comment | added | Nulik | @Bar you can just increase coordinate descent's complexity when it fails to find new minimum by combining evaluation with other coordinates. This will increase number of test cases, but you don't have such conditions often in a hyperdimensional spaces like when training an ANN. An Ann with a few layers will have > 500 parameters, the probability of getting stuck is low and it will probably be at local minimum | |
| Jun 30, 2016 at 11:49 | comment | added | Royi | How come CD is faster than GD? It seems counter logic. | |
| Apr 15, 2015 at 10:28 | vote | accept | Bar | ||
| Apr 15, 2015 at 10:21 | comment | added | Tommy L | I can't say that a specific class of functions will be faster with CD than with other methods, such as e.g. FISTA. As far as I know this depends heavily on your function, and how expensive it is to evaluate the gradient and things like that. From my experience, CD is faster than FISTA on the lasso problem when there are few variables in the model (don't remember, but less than some thousands). Note that I am only comparing CD to ISTA and FISTA here, other algorithms (such as Newton or Pseudo-Newton) will likely be way faster; but this depends entirely on the problem at hand. | |
| Apr 15, 2015 at 9:18 | comment | added | Bar | So which are the cases where coordinate descent (CD) will be faster? Are there some specific types of functions on which CD will be a better candidate? | |
| Apr 15, 2015 at 6:36 | history | answered | Tommy L | CC BY-SA 3.0 |