I'm a beginner in machine learning. I was reading http://ciml.info/ 5.9 Bias/Variance Trade-off
According to this book:
The trade-off between estimation error and approximation error is often called the bias/variance trade-off, where “approximation error” is “bias” and “estimation error” is “variance.”
However, I have done some extra research like this:
It gives a very different definition of bias/variance. According to the book, bias is the error of the best classifier. According to the blog, bias is the expected output minus the true output. I just get confused at these two definitions. How are approximation error, bias, estimation error, variance related? Why are there two very different definitions? Thanks a lot!