Suppose we use the Kernel Regression Estimator $$\hat{m}(c)=\frac{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i}{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)}$$ where $h\to 0$ and $nh\to \infty$ as $n\to \infty$.

The true DGP has form $$y_i=\alpha +\beta x_i +\gamma z_i+\varepsilon_i$$

I assume $\{(y_i,x_i,z_i)\}$ is i.i.d. and all variables are absolutely continuous, have finite second moments, and have positive density over the whole real line. I assume strict exogeneity in both variables, i.e., $\mathbb{E}[\varepsilon_i|x_i;z_i]=0$, and that $x_i$ and $z_i$ are independent of each other.

I want to know to what object $\hat{m}(c)$ converges.

To this end, define $\hat{g}(c)=\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i$ and $\hat{f}(c)=\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)$ such that $\hat{m}(c)=\hat{g}(c)/\hat{f}(c)$. It is easy enough to show $\hat{f}(c)\xrightarrow{p} f(c)$ where $f(c)$ is the pdf of $x$. Now, letting $p(z)$ be the pdf of $z$, \begin{align*} \mathbb{E}[\hat{g}(c)] &= \mathbb{E}\left[\frac{1}{nh}\sum_{i=1}^nK\left(\frac{x_i-c}{h}\right)y_i\right] \\ &= \frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &=\frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)(\alpha +\beta x_i +\gamma z_i+\varepsilon_i)\right] \\ &=\frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)(\alpha +\beta x_i +\gamma z_i)\right] \\ &= \frac{1}{h}\int_\Omega K\left(\frac{x-c}{h}\right)(\alpha +\beta x +\gamma z) \, d\mathbb{P} \\ &=\frac{1}{h}\iint K\left(\frac{x-c}{h}\right)(\alpha +\beta x +\gamma z)f(x)p(z) \, dx \, dz \\ &=\iint K(u)(\alpha +\beta (uh+c) +\gamma z) f(uh+c)p(z) \, du \, dz \\ &= (\alpha + \beta c+\gamma \mathbb{E}[z])f(c) +o(1) \\ &\xrightarrow{n\to \infty} (\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z])f(c) \end{align*}

I can also show that $\mathbb{V}[\hat{g}(c)]\to 0$ which shows $\hat{g}(c)\xrightarrow{p} (\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z])f(c)$. Therefore, I think $$\hat{m}(c)\xrightarrow{p}\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z]$$

However, the TA for the course I'm taking mentioned that $\hat{m}(c)$ should converge to something potentially nonlinear in $c$ which I clearly am not getting. I think the reason for this could be that I am assuming $x_i$ and $z_i$ are independent. However, without this assumption I'm not sure how to find the probability limit of $\hat{m}(c)$.

Could someone either point out where I have gone wrong in my working or explain how to find the limit without assuming independence?


1 Answer 1


I spoke with my TA, and the nonlinearity comes from a $\mathbb{E}[z|x]$ term. A more general approach shows that $\hat{m}(c)\xrightarrow{p} \mathbb{E}[y|x]$.

Using the same notation as in the question and letting $r(\cdot,\cdot)$ be the joint pdf of $(x,y)$, \begin{align*} \mathbb{E}[\hat{g}(c)] &= \mathbb{E}\left[\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &= \frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &=\frac{1}{h}\int K\left(\frac{x-c}{h}\right)y ~\mathrm d\mathbb{P} \\ &=\frac{1}{h}\int K\left(\frac{x-c}{h}\right)y \ r( x,y)~\mathrm d(x,y) \\ &=\frac{1}{h}\iint K\left(\frac{x-c}{h}\right)y \ r( x,y)~\mathrm dx\mathrm dy \\ &=\iint K\left(u\right)y \ r( uh+c,y)~\mathrm du\mathrm dy \\ &=\iint K\left(u\right)y \ [r(c,y)+( r( uh+c,y)-r(c,y))]~\mathrm du\mathrm dy \\ &=\int y \ r(c,y)~\mathrm dy + \iint K(u)y[r( uh+c,y)-r(c,y)]~\mathrm du\mathrm dy \\ &=\int y \ r(c,y)~\mathrm dy + o(1) \\ &= f(c)\mathbb{E}[y|x=c] +o(1) \\ &\to f(c)\mathbb{E}[y|x=c], \end{align*}

where the little $o$ bound is shown by assuming the kernel density has bounded support, and that $r(x,y)$ is continuous.

Then, it is a standard (but longer) argument to show that $\mathbb{V}[\hat{g}(c)]\to 0$. Therefore, $\hat{g}(c)\xrightarrow{p} f(c)\mathbb{E}[y|x=c]$. In combination with $\hat{f}(c)\xrightarrow{p} f(c)$, this gives that $$\hat{m}(c)\xrightarrow{p} \mathbb{E}[y|x=c].$$

In the question, it is given that the DGP for $y_i$ is $$y_i=\alpha +\beta x_i +\gamma z_i +\varepsilon_i$$ and that $\mathbb{E}[\varepsilon_i|x_i;z_i]=0$. Therefore, by law of iterated expectations, $$\mathbb{E}[y|x=c]=\alpha +\beta c +\gamma \mathbb{E}[z|x=c]$$ and this may be nonlinear in $c$ if the mapping $c\mapsto \mathbb{E}[z|x=c]$ is nonlinear.

  • 1
    $\begingroup$ Appreciate you self-answered your query. $\endgroup$ Commented Apr 18 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.