0
$\begingroup$

I am training a deep learning model for binary image classification using Keras and TensorFlow. My model gave the highest acc and lower loss. The other metrics I used also have higher values like precision and recall.

As I used the generator and I mentioned the validation set during fitting the model. So the precision and recall on validation data are just 50%. But the Confusion metric is not that much worse on test data. What should I assume that my model is overfitting?

Here is my few epochs logs:

Epoch 00042: LearningRateScheduler setting learning rate to 8.28633770088062e-06.
595/595 [==============================] - 1871s 3s/step - loss: 0.0162 - accuracy: 0.9952 - precision_m: 0.9958 - recall_m: 0.9936 - f1_m: 0.9942 - val_loss: 0.1693 - val_accuracy: 0.9637 - val_precision_m: 0.5000 - val_recall_m: 0.4797 - val_f1_m: 0.4891

Epoch 00042: saving model to /home/ali/Desktop/pneumonia classification/Final Dataset/RSNA/model_training_512_finetune_3.h5
Epoch 43/50

Epoch 00043: LearningRateScheduler setting learning rate to 7.497788410238852e-06.
595/595 [==============================] - 2167s 4s/step - loss: 0.0147 - accuracy: 0.9952 - precision_m: 0.9968 - recall_m: 0.9944 - f1_m: 0.9953 - val_loss: 0.1673 - val_accuracy: 0.9645 - val_precision_m: 0.5000 - val_recall_m: 0.4823 - val_f1_m: 0.4904

Epoch 00043: saving model to /home/ali/Desktop/pneumonia classification/Final Dataset/RSNA/model_training_512_finetune_3.h5
Epoch 44/50

Epoch 00044: LearningRateScheduler setting learning rate to 6.784279506100467e-06.
595/595 [==============================] - 2405s 4s/step - loss: 0.0125 - accuracy: 0.9967 - precision_m: 0.9982 - recall_m: 0.9958 - f1_m: 0.9968 - val_loss: 0.1676 - val_accuracy: 0.9671 - val_precision_m: 0.5000 - val_recall_m: 0.4831 - val_f1_m: 0.4909
Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
1
$\begingroup$

This shows that your model is overfit. It means it is learning but not applying generally on any other data than the training it self. To prevent this type of problem you have to apply some techniques to prevent from overfit.

  1. Perform Hyperparameter tunning in Nueral networks
  2. Regularization
  3. Apply Early stopping in Fit method
  4. Get more training and test data
  5. Use ensemble learning as not every problem has to be solved by neural nets

One more thing, If your data is images then use Data Augmentation before training with keras.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Try anything you havent used. It takes time but eventually you will get to better test recall and precision. as recall is the most important. $\endgroup$
    – Sarim Sikander
    Commented Feb 2, 2022 at 12:29
  • $\begingroup$ Hyperparamters is tunned for this model. ResNet50v2 already contains BN and dropout. Early stopping is applied. I have 20GB of data I think it's enough for this model. Ensemble learning is applied when you want to use multiple models for a single task to improve results. (useless for now) $\endgroup$
    – Engr Ali
    Commented Feb 2, 2022 at 12:33
  • $\begingroup$ Can you tell me how many classes you have in your dataset? $\endgroup$
    – Sarim Sikander
    Commented Feb 2, 2022 at 12:36
  • $\begingroup$ It's a binary classification problem having a balanced dataset. It's chest Xrays where data have very limited diversity in a normal and infected persons. $\endgroup$
    – Engr Ali
    Commented Feb 2, 2022 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.