I'm modeling 15000 tweets for sentiment prediction using a single layer LSTM with 128 hidden units using a word2vec-like representation with 80 dimensions. I get a descent accuracy (38% with random = 20%) after 1 epoch. More training makes the validation accuracy start declining as the training accuracy starts climbing - a clear sign of overfitting.
I'm therefore thinking of ways to do regularization. I'd prefer not to reduce the number of hidden units (128 seems a bit low already). I currently use dropout with a probability 50%, but this could perhaps be increased. The optimizer is Adam with the default parameters for Keras (http://keras.io/optimizers/#adam).
What are some effective ways of reducing overfitting for this model on my dataset?