Does a non-parametric model necessarily have zero bias? For a parametric model like linear regression, the bias is often interpreted as "the parameters & architecture you chose are inappropriate for the shape of this dataset".
For (one definition of) a non-parametric model, the model is not "built on" parameters, but rather by storing a subset of the data itself [1]. The model's capacity thus is not limited by parameters, and grows with the training data.
So, does a non-parametric model necessarily have zero bias?
[1] On page 15 of Andrew Ng's notes.
 A: I may give you an example. Let's consider two feature $X_1$ and $X_2$. The feature $X_1$ gives the unbiased model. And $X_2$ is related to the $X_1$ by the relationship:
$$X_2 = X_1 + a$$
where $a$ is constant.
Let's consider we know the true model which is given by:
$$ Y = X_1^2 + \varepsilon$$
where $\varepsilon$ is noise term with $E[\varepsilon] = 0$
So we have the unbiased model with the feature $X_1$.
But if you happen to use feature $X_2$, you would get
$$
\begin{align}
Y &= {X_2}^2 + \varepsilon\\
&=X_1^2 + 2aX_1+a^2 +\varepsilon
\end{align}
$$
The term, $ 2aX_1+a^2$ give rise to bias and the basis cannot be removed if you use only the feature $X_2$.
So if you use the wrong feature $X_2$, you will get the biased model but you can remove the bias by using the correct feature $X_1$.

For a non-parametric model, we may think of kNN as your model. And let's assume the true model is linear this time:
$$ Y = X_1 + \varepsilon$$
For the data generated by the linear relationship, you will get an unbiased estimation of $Y$ with kNN.
On the other hand,  if you use $X_2$ instead of $X_1$ for your model, you have
$$
\begin{align}
Y &= {X_2} + \varepsilon\\
&=X_1 + a +\varepsilon
\end{align}
$$
That is to say, you will get the biased estimation of $Y$ with bias $a$ if you use kNN with feature $X_2$.
