I implemented dropout on my neural networks. I tried to train the neural network to act as the
f(x) = sin(x) function. During normal backpropagation without dropout regularization, it literally needed less than 10 iterations to reach an extremely small error.
However, when I activated dropout, it got stuck at a reasonably high MSE error (
0.05) and there was no improvement over time any more. I used two hidden units.
But asides from that, are there any examples of dropout regularization on small neural networks. Or any papers describing the effectiveness on small neural networks? That way I can test if my implementation was done correctly.