Timeline for Can a deep enough linear neural network (with activation functions) be able to learn anything?
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 28 at 18:59 | comment | added | Sycorax♦ | Lots of examples of functions that cannot be solved by NNs in this Q. stats.stackexchange.com/questions/563604/… I think you should edit to be more specific about the kinds of functions you are interested in | |
| Feb 28 at 18:26 | comment | added | Zohaib Hamdule | I found this resource: quora.com/… | |
| Feb 28 at 18:23 | comment | added | John Madden | It doesn't even have to be deep! One layer is enough if you have a buncha neurons | |
| Feb 28 at 18:19 | comment | added | Zohaib Hamdule | I had seen a representation of CNN as simple linear network with shared parameters. Can the linear networks also replace RNN (hypothetically)? | |
| Feb 28 at 18:15 | comment | added | Ggjj11 | Yes, it just becomes hard to train. Also note that you can directly represent a convolutional layer as a matrix multiplication ai.stackexchange.com/a/21874/41576 | |
| Feb 28 at 18:07 | history | edited | Zohaib Hamdule | CC BY-SA 4.0 |
added 115 characters in body
|
| Feb 28 at 18:05 | comment | added | Zohaib Hamdule | Yes, but does Universal Approximation Theorem also apply to functions that model temporal or pictorial data? So Hypothetically I could replace an RNN or a CNN with a deep enough enough fully connected feed forward network? | |
| Feb 28 at 18:05 | comment | added | Sextus Empiricus | Maybe some things are not learnable from data? E.g. concepts like Gödel's incompleteness theorems | |
| Feb 28 at 18:02 | comment | added | Dave | Isn't this a universal approximation theorem? | |
| Feb 28 at 18:01 | history | asked | Zohaib Hamdule | CC BY-SA 4.0 |