I'm training a neural network using 70% of my data as training set, 20% as external test set and 10% for validation using Keras. When I evaluate the trained model the performance on the validation set is much lower than the training set, there's a high loss and low accuracy. In addition the loss x epochs plot shows overfitting is happening. However, when I evaluate this same model on the external test set the performance is similar to my training set (i.e. much higher than validation set). I'm a bit confused if my model is overfitting or not, since it seems to work on real world data (the external set).

Heres's the result from model.evaluate() from Keras using the test set.

        acc       loss      matthews_correlation  precision    recall
train   0.977796  0.069748  0.590605              0.570999     0.63658

        acc       loss      matthews_correlation  precision    recall
test    0.975806  0.072908  0.562041              0.554142     0.597151

And here's the result for training and validation sets:

 train   loss: 0.0529 - acc: 0.9783 - matthews_correlation: 0.5342 - precision: 0.6517 - recall: 0.4603 - 
 validation   val_loss: 0.2901 - val_acc: 0.9514 - val_matthews_correlation: 0.0386 - val_precision: 0.0639 - val_recall: 0.0628

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.