2
$\begingroup$

I'm training a 4 layer (2 hidden layers) Neural Net, with 10 input nodes, and 6000 output nodes.

During training, my loss function (RMS), and R2 currently looks like this:

Loss and R^2

So far it seems like the Loss function is starting to level off, even though my R2 is only at 0.2.

Is it reasonable to expect that there will be another sharp improvement? Or do Neural Networks typically go through a single "phase" of improvement?

$\endgroup$

1 Answer 1

1
$\begingroup$

Judging by the way the charts look, I would say there is room for improvement for R^2. You can probably wait for more iterations until the loss function reaches a flat zone.

You can also experiment starting with a higher learning rate and apply a decay during training.

Or do Neural Networks typically go through a single "phase" of improvement?

I think this depends more on the data you are using for training. There can be cases where training might get stuck in a local optimum making the network for example to predict correctly only 3/5 classes and just randomly guess for the other 2. If the limit between classes is sensible, the training might exit the local optimum and you will see another sharp descend in the error function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.