In this cs231 lection note there is a counterintuitive quote:
The gap between the training and validation accuracy indicates the amount of overfitting.
The other possible case is when the validation accuracy tracks the training accuracy fairly well. This case indicates that your model capacity is not high enough: make the model larger by increasing the number of parameters.
- Isn't the goal of training to achieve the validation accuracy as high as training accuracy or did I miss something?
- What the intuition under increasing the number of parameters if the validation accuracy tracks the training accuracy fairly well?