Interview question: train/test error and "best" model

Question

I recently had a puzzling interview question and I am wondering whether anybody can tell me the intended answer.

The question shows train and test error for three models plotted against the number of iterations/epochs and asks you to rank the models in order "from best to worst".

All models have the same minimum test error but different error curves on the training set.

It is a multiple choice question and all six of the orders are given as possible answers.

I didn't know what the intention was (or what "best" means) so I ranked the models by how many epochs they took to achieve minimum test set error. But this is just a guess.

What is the intended answer to the question?

Answer 1

I would agree that 2-1-3 seems like the best model, but I think there a couple other factors that could be considered which provide additional support for that answer.

1. The fact that the training curve is less convex suggests that the model is more robust to overfitting.

2. The train curve stays closer to the test curve than for the other models, suggesting that the model is learning the "right" information from the training set.

3. In addition to reaching its minimum quickest, model 2's loss is lower already after the first epoch. Depending on the setup of this problem, this could possibly suggest the model is better able to extract useful information.