ML models performance can be evaluated using different scoring metrics; MSE, accuracy, precision as well as, it can be evaluated using learning curves. I am wondering how overfitting/underfitting problems can be quantitatively evaluated? for instance, how to keep a model within variance/bias as less than a specific number? Or how to get that number first?


It's a very good question, and I thought the following slide can be of help.
enter image description here

The bias can be approximately the gap between the Bayes optimal error(or irreducible error or the human performance error) and the training error; while the variance is the gap between the training error and the evaluation error(or dev error).

If the bias is big that means the model is inexpressive and we can use a bigger model with more parameters or different network architecture, or we can train it longer; while if the variance is big it means the model is overfitting(cannot generalize to unseen data) and we should train the model using more data or use regularization methods.

The bias is normally avoidable so we should focus on reducing the bias by using the right model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.