I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$.

Specifically, I'm distraught with equation $(3.59)$ on page 102 of this book. This is part of the proof of Theorem 3.1 on page 62.

Here's the setup. Let $(X_1, Y_1),\ldots,(X_n,Y_n)$ be iid taking values in $\mathbb R^2,$ let $X_1$ have density $f.$ Let $m(x)=E(Y|X=x)$ and let $K$ be a symmetric kernel with bounded support, let $K_h(t) = K(t/h)/h,$ where $h$ is the bandwidth. Write $$\mathbf X=((X_i-x_0)^j)_{i=1,\ldots,n \atop j=0,\ldots,p}, \mathbf W = \operatorname{diag}(K_h(X_1-x_0),\ldots,K_h(X_n-x_0)),\\ \mathbf y=(Y_1\ldots,Y_n)^T, \mathbf m=(m(X_1),\ldots,m(X_n))^T.$$

Then the conditional bias of the local polynomial estimator $\hat\beta=(\mathbf X^T \mathbf W \mathbf X)^{-1}\mathbf X^T \mathbf W \mathbf y$ is $$\operatorname{Bias}(\hat\beta|(X_1,\ldots,X_n))=(\mathbf X^T \mathbf W \mathbf X)^{-1}\mathbf X^T \mathbf W \mathbf r =: S_n^{-1}\mathbf X^T \mathbf W \mathbf r,$$ where $\mathbf r = \mathbf m-\mathbf X \beta,\, \beta=(m(x_0),\ldots,m^{(p)}(x_0)/{p!}).$

Assume that $m^{(p+1)}(\cdot)$ is continuous in a neighborhood of $x_0.$ Fan writes on page 102:

By using the Taylor expansion the conditional bias $S_n^{-1}\mathbf X^T \mathbf W \mathbf r$ of $\hat\beta$ can be written as $$S_n^{-1}\mathbf X^T \mathbf W \Bigl[\beta_{p+1}(X_i-x_0)^{p+1}+o_P\left\{(X_i-x_0)^{p+1}\right\}\Bigr]_{1\le i\le n}$$

I don't understand what is meant by $o_P\left\{(X_i-x_0)^{p+1}\right\}$ in this context. I know that the usual definition is that it's a term which converges in probability to zero even after dividing by $(X_i-x_0)^{p+1}.$ But what converges in probability here? Is it meant that this holds as $n\to\infty$?

I tried writing everything out but failed to understand what he means: $$ \begin{align} \mathbf r &= \mathbf m-\mathbf X \beta \\ &= \Biggl[\sum_{l=1}^{p+1} \frac{m^{(l)}(x_0)}{l!} (X_i-x_0)^{l} + (X_i-x_0)^{p+1}\frac{m^{(p+1)}(\xi_i)}{(p+1)!} \text{ (using Lagrange remainder)}\\ &\quad\quad- \sum_{l=0}^{p} \frac{m^{(l)}(x_0)}{l!} (X_i-x_0)^{l}\Biggr]_{1\le i\le n}\\ &= \left[\frac{m^{(p+1)}(x_0)}{(p+1)!} (X_i-x_0)^{p+1} + (X_i-x_0)^{p+1}\frac{m^{(l)}(\xi_i)}{(p+1)!}\right]_{1\le i\le n} \end{align} $$ If I could show that in some sense $$ (X_i-x_0)^{p+1}\frac{m^{(l)}(\xi_i)}{(p+1)!} = o_P((X_i-x_0)^{p+1}) $$ I would be done, but I'm not even sure in what sense he means this..


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.