Timeline for Cost function of neural network is non-convex?
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 6 at 8:59 | comment | added | saper0 | Only strictly convex functions have a unique global minimum. Thus, this argument only rules out that our function is strictly convex. Convex functions are allowed to have multiple local minima, which are all global optima. So one would need to show that on the line segment (in parameter space) between the two found minima, our function value increases again. For an alternative answer not suffering from this issue, look at @Reza's answer. | |
| Jul 26, 2020 at 16:07 | comment | added | John Jiang | This is pretty rigorous. Your argument shows that if the columns of each weight matrix are not identical at convergence, then the objective function is nonconvex. | |
| May 30, 2020 at 17:36 | comment | added | Abhinav | This is not a proof by any means, but how I intuitively understand it. It is certainly possible to design pathological examples where the above argument doesn't work. Besides the condition you mentioned, another case when this doesn't work is when all the weights at the minima are equal. But we know this case is highly unlikely. We also know that a minima exists by examining the region close to where gradient descent terminates. | |
| May 29, 2020 at 15:56 | comment | added | Maverick Meerkat | you're assuming there is a minima that is attained. What if there isn't (think $e^{-x}$)? | |
| Mar 21, 2020 at 16:43 | comment | added | Seymour | interesting heuristic | |
| S May 4, 2017 at 9:25 | history | suggested | CommunityBot | CC BY-SA 3.0 |
Corrected grammar.
|
| May 4, 2017 at 8:55 | review | Suggested edits | |||
| S May 4, 2017 at 9:25 | |||||
| Sep 3, 2015 at 16:37 | review | Suggested edits | |||
| Sep 3, 2015 at 16:59 | |||||
| Feb 11, 2015 at 6:41 | review | Late answers | |||
| Feb 11, 2015 at 6:41 | |||||
| Feb 11, 2015 at 6:26 | review | First posts | |||
| Feb 11, 2015 at 6:52 | |||||
| Feb 11, 2015 at 6:23 | history | answered | Abhinav | CC BY-SA 3.0 |