I'm training a 4 layer (2 hidden layers) Neural Net, with 10 input nodes, and 6000 output nodes.
During training, my loss function (RMS), and R2 currently looks like this:
So far it seems like the Loss function is starting to level off, even though my R2 is only at 0.2.
Is it reasonable to expect that there will be another sharp improvement? Or do Neural Networks typically go through a single "phase" of improvement?