In his book, "Pattern recognition and Machine learning", Bishop talks about the influence of the bias and overfitting in the MLE framework. Here is a quote from p.28, just before he has shown that the maximum likelihood estimate of the sample variance is an biased estimator: "However, throughout this book we shall be interested in more complex models with many parameters, for which the bias problems associated with maximum likelihood will be much more severe. In fact, as we shall see, the issue of bias in maximum likelihood lies at the root of the over-fitting problem that we encountered earlier in the context of polynomial curve fitting."

So what I do not understand how biasedness can cause overfitting (or vice versa) because as I understand it, bias is connected to underfitting and too high variance is connected to overfitting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.