A simple 32x16 multi-layer perceptron is apparently overfitting my dataset. The training error goes way down while the test error balloons. Note that with smaller MLPs like 8x8 the training and test error both go down. Anything larger than 8x8 seems to result in overfitting.
Given that the dataset seems huge to me (30 million rows, 500 columns), this suggests that the training examples must be similar in some way, but I don't know how to diagnose this. How would I go about confirming that overfitting is indeed the issue? And is there anything I can do to alleviate it?