I am training an LSTM neural network for nlm on a big dataset: the model has about 100M learnable parameters and the dataset consists of about 2G characters.
Therefore it seems that overfitting should not be a problem. Should I still try to apply dropout to my model? How should I determine this? (would you please link something theoretically grounded if possible).