I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$.
Specifically, I'm distraught with equation $(3.59)$ on page 102 of this book. This is part of the proof of Theorem 3.1 on page 62.
Here's the setup. Let $(X_1, Y_1),\ldots,(X_n,Y_n)$ be iid taking values in $\mathbb R^2,$ let $X_1$ have density $f.$ Let $m(x)=E(Y|X=x)$ and let $K$ be a symmetric kernel with bounded support, let $K_h(t) = K(t/h)/h,$ where $h$ is the bandwidth. Write $$\mathbf X=((X_i-x_0)^j)_{i=1,\ldots,n \atop j=0,\ldots,p}, \mathbf W = \operatorname{diag}(K_h(X_1-x_0),\ldots,K_h(X_n-x_0)),\\ \mathbf y=(Y_1\ldots,Y_n)^T, \mathbf m=(m(X_1),\ldots,m(X_n))^T.$$
Then the conditional bias of the local polynomial estimator $\hat\beta=(\mathbf X^T \mathbf W \mathbf X)^{-1}\mathbf X^T \mathbf W \mathbf y$ is $$\operatorname{Bias}(\hat\beta|(X_1,\ldots,X_n))=(\mathbf X^T \mathbf W \mathbf X)^{-1}\mathbf X^T \mathbf W \mathbf r =: S_n^{-1}\mathbf X^T \mathbf W \mathbf r,$$ where $\mathbf r = \mathbf m-\mathbf X \beta,\, \beta=(m(x_0),\ldots,m^{(p)}(x_0)/{p!}).$
Assume that $m^{(p+1)}(\cdot)$ is continuous in a neighborhood of $x_0.$ Fan writes on page 102:
By using the Taylor expansion the conditional bias $S_n^{-1}\mathbf X^T \mathbf W \mathbf r$ of $\hat\beta$ can be written as $$S_n^{-1}\mathbf X^T \mathbf W \Bigl[\beta_{p+1}(X_i-x_0)^{p+1}+o_P\left\{(X_i-x_0)^{p+1}\right\}\Bigr]_{1\le i\le n}$$
I don't understand what is meant by $o_P\left\{(X_i-x_0)^{p+1}\right\}$ in this context. I know that the usual definition is that it's a term which converges in probability to zero even after dividing by $(X_i-x_0)^{p+1}.$ But what converges in probability here? Is it meant that this holds as $n\to\infty$?
I tried writing everything out but failed to understand what he means: $$ \begin{align} \mathbf r &= \mathbf m-\mathbf X \beta \\ &= \Biggl[\sum_{l=1}^{p+1} \frac{m^{(l)}(x_0)}{l!} (X_i-x_0)^{l} + (X_i-x_0)^{p+1}\frac{m^{(p+1)}(\xi_i)}{(p+1)!} \text{ (using Lagrange remainder)}\\ &\quad\quad- \sum_{l=0}^{p} \frac{m^{(l)}(x_0)}{l!} (X_i-x_0)^{l}\Biggr]_{1\le i\le n}\\ &= \left[\frac{m^{(p+1)}(x_0)}{(p+1)!} (X_i-x_0)^{p+1} + (X_i-x_0)^{p+1}\frac{m^{(l)}(\xi_i)}{(p+1)!}\right]_{1\le i\le n} \end{align} $$ If I could show that in some sense $$ (X_i-x_0)^{p+1}\frac{m^{(l)}(\xi_i)}{(p+1)!} = o_P((X_i-x_0)^{p+1}) $$ I would be done, but I'm not even sure in what sense he means this..
