Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model:

rows: 10,166, features: 1692, model used: feed forward neural networks

logs for a particular fold:

epoch: 0, train loss : 0.585782928946, train score: 0.497655085604, val score: 0.610834066686 training with 32 sized batches best validation score till now: 0.0

epoch: 1, train loss : 0.350900779755, train score: 0.509627315655, val score: 0.625354453897 training with 32 sized batches best validation score till now: 0.0

epoch: 2, train loss : 0.249989495512, train score: 0.528054031557, val score: 0.646059450474 training with 32 sized batches best validation score till now: 0.0

epoch: 3, train loss : 0.210879948519, train score: 0.553915714449, val score: 0.65603304977 training with 32 sized batches best validation score till now: 0.0

epoch: 4, train loss : 0.196106336583, train score: 0.578792288474, val score: 0.644788305466 training with 32 sized batches best validation score till now: 0.0

epoch: 5, train loss : 0.189342178014, train score: 0.601882261054, val score: 0.640632639093 training with 32 sized batches best validation score till now: 0.0

epoch: 6, train loss : 0.18660737301, train score: 0.616972833149, val score: 0.640021511685 training with 32 sized batches new best validation score: 0.629363449692 saving model...

epoch: 7, train loss : 0.183136423458, train score: 0.630294120457, val score: 0.629363449692 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 8, train loss : 0.180001894893, train score: 0.641283331582, val score: 0.611542974479 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 9, train loss : 0.179555135817, train score: 0.646623580034, val score: 0.606971741469 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 10, train loss : 0.17941892064, train score: 0.650499675163, val score: 0.603036080962

and the trend of increasing training score and decreasing validation score continues. My question is which model (after which epoch) should be selected. Currently i select by early stopping based on validation score not having increased for more than 15 epochs and under the condition that training score be greater than the validation score at that epoch. Which makes it model after epoch 7 here.

Below is a specific case but a general situation I find myself involved within my job. This question is intended at getting ideas on how to pick the best model:

rows: 10,166, features: 1692, Model used: Feed Forward Neural Networks

logs for a particular fold:

epoch: 0, train loss : 0.585782928946,  train score: 0.497655085604, val score: 0.610834066686 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 1, train loss : 0.350900779755,  train score: 0.509627315655, val score: 0.625354453897 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 2, train loss : 0.249989495512,  train score: 0.528054031557, val score: 0.646059450474 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 3, train loss : 0.210879948519,  train score: 0.553915714449, val score: 0.65603304977 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 4, train loss : 0.196106336583,  train score: 0.578792288474, val score: 0.644788305466 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 5, train loss : 0.189342178014,  train score: 0.601882261054, val score: 0.640632639093 
training  with 32 sized batches
best validation score till now: 0.0

epoch: 6, train loss : 0.18660737301,  train score: 0.616972833149, val score: 0.640021511685 
training  with 32 sized batches
new best validation score: 0.629363449692
saving model...

epoch: 7, train loss : 0.183136423458,  train score: 0.630294120457, val score: 0.629363449692 
training  with 32 sized batches
best validation score till now: 0.629363449692

epoch: 8, train loss : 0.180001894893,  train score: 0.641283331582, val score: 0.611542974479 
training  with 32 sized batches
best validation score till now: 0.629363449692

epoch: 9, train loss : 0.179555135817,  train score: 0.646623580034, val score: 0.606971741469 
training  with 32 sized batches
best validation score till now: 0.629363449692

epoch: 10, train loss : 0.17941892064,  train score: 0.650499675163, val score: 0.603036080962

and the trend of increasing training score and decreasing validation score continues. My question is which model (after which epoch) should be selected. Currently, I select by early stopping based on validation score not having increased for more than 15 epochs and under the condition that training score is greater than the validation score at that epoch. Which makes it model after epoch 7 here.

Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model:

Dataset:

rows: 10,166, features: 1692, model used: feed forward neural networks

logs for a particular fold:

epoch: 0, train loss : 0.585782928946, train score: 0.497655085604, val score: 0.610834066686 training with 32 sized batches best validation score till now: 0.0

epoch: 1, train loss : 0.350900779755, train score: 0.509627315655, val score: 0.625354453897 training with 32 sized batches best validation score till now: 0.0

epoch: 2, train loss : 0.249989495512, train score: 0.528054031557, val score: 0.646059450474 training with 32 sized batches best validation score till now: 0.0

epoch: 3, train loss : 0.210879948519, train score: 0.553915714449, val score: 0.65603304977 training with 32 sized batches best validation score till now: 0.0

epoch: 4, train loss : 0.196106336583, train score: 0.578792288474, val score: 0.644788305466 training with 32 sized batches best validation score till now: 0.0

epoch: 5, train loss : 0.189342178014, train score: 0.601882261054, val score: 0.640632639093 training with 32 sized batches best validation score till now: 0.0

epoch: 6, train loss : 0.18660737301, train score: 0.616972833149, val score: 0.640021511685 training with 32 sized batches new best validation score: 0.629363449692 saving model...

epoch: 7, train loss : 0.183136423458, train score: 0.630294120457, val score: 0.629363449692 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 8, train loss : 0.180001894893, train score: 0.641283331582, val score: 0.611542974479 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 9, train loss : 0.179555135817, train score: 0.646623580034, val score: 0.606971741469 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 10, train loss : 0.17941892064, train score: 0.650499675163, val score: 0.603036080962

and the trend of increasing training score and decreasing validation score continues. My question is which model (after which epoch) should be selected. Currently i select by early stopping based on validation score not having increased for more than 15 epochs and under the condition that training score be greater than the validation score at that epoch. Which makes it model after epoch 6 here.

Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model:

Dataset:

rows: 10,166, features: 1692, model used: feed forward neural networks

logs for a particular fold:

epoch: 0, train loss : 0.585782928946, train score: 0.497655085604, val score: 0.610834066686 training with 32 sized batches best validation score till now: 0.0

epoch: 1, train loss : 0.350900779755, train score: 0.509627315655, val score: 0.625354453897 training with 32 sized batches best validation score till now: 0.0

epoch: 2, train loss : 0.249989495512, train score: 0.528054031557, val score: 0.646059450474 training with 32 sized batches best validation score till now: 0.0

epoch: 3, train loss : 0.210879948519, train score: 0.553915714449, val score: 0.65603304977 training with 32 sized batches best validation score till now: 0.0

epoch: 4, train loss : 0.196106336583, train score: 0.578792288474, val score: 0.644788305466 training with 32 sized batches best validation score till now: 0.0

epoch: 5, train loss : 0.189342178014, train score: 0.601882261054, val score: 0.640632639093 training with 32 sized batches best validation score till now: 0.0

epoch: 6, train loss : 0.18660737301, train score: 0.616972833149, val score: 0.640021511685 training with 32 sized batches new best validation score: 0.629363449692 saving model...

epoch: 7, train loss : 0.183136423458, train score: 0.630294120457, val score: 0.629363449692 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 8, train loss : 0.180001894893, train score: 0.641283331582, val score: 0.611542974479 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 9, train loss : 0.179555135817, train score: 0.646623580034, val score: 0.606971741469 training with 32 sized batches best validation score till now: 0.629363449692

epoch: 10, train loss : 0.17941892064, train score: 0.650499675163, val score: 0.603036080962

and the trend of increasing training score and decreasing validation score continues. My question is which model (after which epoch) should be selected. Currently i select by early stopping based on validation score not having increased for more than 15 epochs and under the condition that training score be greater than the validation score at that epoch. Which makes it model after epoch 7 here.