I've built a classical backpropagation ANN using Keras for a regression problem, which has two hidden layers with a low amount of neurons (max. 8 per layer). The amount of samples for training and validating is 20000, divided 90% and 10% respectively.

Often I observed the situation when there was a dropdown in Validation Loss (MAE) in the first epochs. I have never seen such a low MAE on the testing set, observed during these dropdowns (around 3.5%). Typically I get 4%+ results.

Dropdown in Validation Loss in the first epochs

How can it be explained? Is it normal to have this kind of dropdown? Too low batch size (currently 128)? Any overfitting/underfitting?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.