Timeline for How does ReLU deal with negative inputs? [duplicate]
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 28, 2019 at 13:13 | history | duplicates list edited | Sycorax♦ | duplicates list edited from How can a network with only ReLU nodes output negative values? to What should I do when my neural network doesn't learn?, How can a network with only ReLU nodes output negative values?, What should I do when my neural network doesn't generalize well? | |
| Aug 28, 2019 at 13:07 | comment | added | Sycorax♦ | In any case, the answer turns on whether or not the universal approximation theorem applies to ReLUs. It turns out that the answer is "yes." arxiv.org/pdf/1708.02691.pdf If we accept that the UAT is applicable to your problem (a strong assertion), then the question becomes "Why is my network not working very well?" for which we have several threads: (1) stats.stackexchange.com/questions/352036/… and (2) stats.stackexchange.com/questions/365778/… | |
| Aug 28, 2019 at 12:56 | comment | added | stevew | They are using linear output layer. I guess this is indeed very similar to the other question. Thanks for pointing out. | |
| Aug 28, 2019 at 12:23 | history | closed |
itdxer Sycorax♦ neural-networks Users with the neural-networks badge or a synonym can single-handedly close neural-networks questions as duplicates and reopen them as needed. |
Duplicate of How can a network with only ReLU nodes output negative values? | |
| Aug 28, 2019 at 12:22 | comment | added | Sycorax♦ | Are you sure they're using ReLUs for the output layer, instead of only in the hidden layers? | |
| Aug 28, 2019 at 12:11 | history | asked | stevew | CC BY-SA 4.0 |