I am playing with dropout since all state of the art results in machine learning seem to be using it (for example, see here). I am familiar with all the guidelines (train longer, increase capacity of the model, use higher learning rates), but still cannot see it working. I've tried several different examples: CNN for IMDB, CNN for MNIST, MLP for MNIST, MLP for IRIS, and turning off dropout makes all my results better even though the default configurations have dropout (taken from the Keras examples). For example, I am attaching my results for one of the models trained on the IRIS dataset. The configuration without dropout has clearly the best performance.
What am I missing?
The code for the IRIS example is here.