Timeline for Does the universal approximation theorem apply to ReLu?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 21 at 22:40 | comment | added | Thomas Lumley | Hornik is interested in fairly strong approximation results over all of $\mathbb{R}^d$, not just over a single compact set. In particular, he notes that polynomials are not universal approximants in the senses he is using, because they can only approximate a few types of tail behaviour as the inputs go to infinity. As @BenReiniger says, ReLU is ok because there are combinations of ReLUs that do satisfy his conditions even though a single ReLU doesn't | |
| May 24, 2021 at 14:02 | comment | added | Dave | I didn't read the paper. It is possible that the author is discussing stronger conditions, but what I wrote applies to the standard universal approximation theorem. | |
| May 24, 2021 at 13:42 | comment | added | krvger | Are you sure about that? I think $\psi$ denotes the activation function, and the conditions refer to $\psi$ if I read it correctly... | |
| May 24, 2021 at 12:34 | history | edited | Dave | CC BY-SA 4.0 |
added 18 characters in body
|
| May 24, 2021 at 12:27 | history | edited | Dave | CC BY-SA 4.0 |
added 222 characters in body
|
| May 24, 2021 at 12:20 | history | answered | Dave | CC BY-SA 4.0 |