My question is about the 1-nearest neighbor classifier and is about a statement made in the excellent book The Elements of Statistical Learning, by Hastie, Tibshirani and Friedman. The statement is (p. 465, section 13.3):
"Because it uses only the training point closest to the query point, the bias of the 1-nearest neighbor estimate is often low, but the variance is high."
The book is available at
For starters, we can define what bias and variance are. From the question "how-can-increasing-the-dimension-increase-the-variance-without-increasing-the-bi" , we have that:
"First of all, the bias of a classifier is the discrepancy between its averaged estimated and true function, whereas the variance of a classifier is the expected divergence of the estimated prediction function from its average value (i.e. how dependent the classifier is on the random sampling made in the training set).
Hence, the presence of bias indicates something basically wrong with the model, whereas variance is also bad, but a model with high variance could at least predict well on average."
Could someone please explain why the variance is high and the bias is low for the 1-nearest neighbor classifier?