Timeline for Are there any "convex neural networks"?
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 26, 2023 at 21:50 | comment | added | Josiah Yoder | @CagdasOzgenc Hinton, Bengio, and LeCun have a very long talk that focuses on this question for a few slides. See, e.g. Slide 53, which points out that even with a single neuron in each layer multiplied by a constant weight, attempting to learn the identity function is not convex. Indeed, their are an infinite number of solutions lying along a hyperbola in w1,w2 space. Perhaps you mean that they are "not strictly convex in the neighborhood of the minimum"? | |
| Apr 5, 2021 at 5:01 | vote | accept | Fraïssé | ||
| Sep 3, 2023 at 18:10 | |||||
| Dec 9, 2020 at 22:12 | comment | added | Fraïssé | @CagdasOzgenc What is the meaning of "linear regression correlated regressor" | |
| Dec 9, 2020 at 13:42 | comment | added | Cagdas Ozgenc | I believe that you are wrong. MLPs with linear activations are convex, but not strictly convex, just like linear regression with correlated regressors. They can be effectively optimized. | |
| Dec 9, 2020 at 13:35 | history | answered | carlo | CC BY-SA 4.0 |