Timeline for Neural Network - Estimating Non-linear function
Current License: CC BY-SA 4.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 30, 2020 at 4:55 | comment | added | Anon | Don't be too wary. It's not unusual to see deep neural networks that have on the scale of thousands to millions of parameters. The full benefit of deep learning though does require multiple layer, so just try 5-7 layers of like 10-20 neurons each just to see what happens. Just make sure that your dataset is sufficiently large. Also, use Rectified Linear activations instead of sigmoid when you have deeper networks because those make gradient calculations easier. | |
| Jan 29, 2020 at 18:31 | comment | added | Alex | Thanks! So even in my simple situation where the underlying function is X1 + X2^2 + X3 + X4, would you recommend more layers and/or neurons? I have tried 3 neurons in the first layer, and 2 neurons in the second layer along with a logistic activation function, and I don't think it recognized the non-linearity very well. I'm a little wary of blindly increasing the complexity of the network when I only have four features. | |
| Jan 29, 2020 at 5:41 | history | answered | Anon | CC BY-SA 4.0 |