Timeline for What does the term saturating nonlinearities mean?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 30, 2021 at 9:49 | answer | added | brazofuerte | timeline score: 2 | |
| Jan 24, 2021 at 14:03 | comment | added | BjornW | It should be noted that the main important difference is not the form of the function and neither really its squashing behaviour but that the non-saturating ones don't have vanishing gradients if the activations for some reason get out of control. And you need gradients to do gradient descent. The Rectified Linear unit (ReLU), what you refer to by the max() formula, has a decent gradient at all values (pun intended ;). | |
| S Nov 21, 2019 at 21:31 | history | suggested | user82135 | CC BY-SA 4.0 |
I've removed redundancy and I fixed a typo.
|
| Nov 21, 2019 at 18:35 | review | Suggested edits | |||
| S Nov 21, 2019 at 21:31 | |||||
| Jul 4, 2019 at 21:18 | comment | added | Nathan majicvr.com | I also found this quora answer very helpful. | |
| Oct 10, 2017 at 11:04 | answer | added | Pradi KL | timeline score: 1 | |
| Dec 28, 2015 at 17:00 | history | edited | Franck Dernoncourt |
add tags~~~~~~~~~
|
|
| Sep 28, 2015 at 15:43 | vote | accept | Charlie Parker | ||
| Sep 28, 2015 at 0:21 | answer | added | Franck Dernoncourt | timeline score: 56 | |
| Sep 26, 2015 at 20:37 | history | edited | JohnRos |
added terminology tag
|
|
| Sep 26, 2015 at 19:45 | history | asked | Charlie Parker | CC BY-SA 3.0 |