# Non-Parametric Regression with an Omitted Variable

Suppose we use the Kernel Regression Estimator $$\hat{m}(c)=\frac{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i}{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)}$$ where $$h\to 0$$ and $$nh\to \infty$$ as $$n\to \infty$$.

The true DGP has form $$y_i=\alpha +\beta x_i +\gamma z_i+\varepsilon_i$$

I assume $$\{(y_i,x_i,z_i)\}$$ is i.i.d. and all variables are absolutely continuous, have finite second moments, and have positive density over the whole real line. I assume strict exogeneity in both variables, i.e., $$\mathbb{E}[\varepsilon_i|x_i;z_i]=0$$, and that $$x_i$$ and $$z_i$$ are independent of each other.

I want to know to what object $$\hat{m}(c)$$ converges.

To this end, define $$\hat{g}(c)=\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i$$ and $$\hat{f}(c)=\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)$$ such that $$\hat{m}(c)=\hat{g}(c)/\hat{f}(c)$$. It is easy enough to show $$\hat{f}(c)\xrightarrow{p} f(c)$$ where $$f(c)$$ is the pdf of $$x$$. Now, letting $$p(z)$$ be the pdf of $$z$$, \begin{align*} \mathbb{E}[\hat{g}(c)] &= \mathbb{E}\left[\frac{1}{nh}\sum_{i=1}^nK\left(\frac{x_i-c}{h}\right)y_i\right] \\ &= \frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &=\frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)(\alpha +\beta x_i +\gamma z_i+\varepsilon_i)\right] \\ &=\frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)(\alpha +\beta x_i +\gamma z_i)\right] \\ &= \frac{1}{h}\int_\Omega K\left(\frac{x-c}{h}\right)(\alpha +\beta x +\gamma z) \, d\mathbb{P} \\ &=\frac{1}{h}\iint K\left(\frac{x-c}{h}\right)(\alpha +\beta x +\gamma z)f(x)p(z) \, dx \, dz \\ &=\iint K(u)(\alpha +\beta (uh+c) +\gamma z) f(uh+c)p(z) \, du \, dz \\ &= (\alpha + \beta c+\gamma \mathbb{E}[z])f(c) +o(1) \\ &\xrightarrow{n\to \infty} (\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z])f(c) \end{align*}

I can also show that $$\mathbb{V}[\hat{g}(c)]\to 0$$ which shows $$\hat{g}(c)\xrightarrow{p} (\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z])f(c)$$. Therefore, I think $$\hat{m}(c)\xrightarrow{p}\alpha + \beta c+\gamma \operatorname{\mathbb{E}}[z]$$

However, the TA for the course I'm taking mentioned that $$\hat{m}(c)$$ should converge to something potentially nonlinear in $$c$$ which I clearly am not getting. I think the reason for this could be that I am assuming $$x_i$$ and $$z_i$$ are independent. However, without this assumption I'm not sure how to find the probability limit of $$\hat{m}(c)$$.

Could someone either point out where I have gone wrong in my working or explain how to find the limit without assuming independence?

I spoke with my TA, and the nonlinearity comes from a $$\mathbb{E}[z|x]$$ term. A more general approach shows that $$\hat{m}(c)\xrightarrow{p} \mathbb{E}[y|x]$$.
Using the same notation as in the question and letting $$r(\cdot,\cdot)$$ be the joint pdf of $$(x,y)$$, \begin{align*} \mathbb{E}[\hat{g}(c)] &= \mathbb{E}\left[\frac{1}{nh}\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &= \frac{1}{h}\mathbb{E}\left[K\left(\frac{x_i-c}{h}\right)y_i\right] \\ &=\frac{1}{h}\int K\left(\frac{x-c}{h}\right)y ~\mathrm d\mathbb{P} \\ &=\frac{1}{h}\int K\left(\frac{x-c}{h}\right)y \ r( x,y)~\mathrm d(x,y) \\ &=\frac{1}{h}\iint K\left(\frac{x-c}{h}\right)y \ r( x,y)~\mathrm dx\mathrm dy \\ &=\iint K\left(u\right)y \ r( uh+c,y)~\mathrm du\mathrm dy \\ &=\iint K\left(u\right)y \ [r(c,y)+( r( uh+c,y)-r(c,y))]~\mathrm du\mathrm dy \\ &=\int y \ r(c,y)~\mathrm dy + \iint K(u)y[r( uh+c,y)-r(c,y)]~\mathrm du\mathrm dy \\ &=\int y \ r(c,y)~\mathrm dy + o(1) \\ &= f(c)\mathbb{E}[y|x=c] +o(1) \\ &\to f(c)\mathbb{E}[y|x=c], \end{align*}
where the little $$o$$ bound is shown by assuming the kernel density has bounded support, and that $$r(x,y)$$ is continuous.
Then, it is a standard (but longer) argument to show that $$\mathbb{V}[\hat{g}(c)]\to 0$$. Therefore, $$\hat{g}(c)\xrightarrow{p} f(c)\mathbb{E}[y|x=c]$$. In combination with $$\hat{f}(c)\xrightarrow{p} f(c)$$, this gives that $$\hat{m}(c)\xrightarrow{p} \mathbb{E}[y|x=c].$$
In the question, it is given that the DGP for $$y_i$$ is $$y_i=\alpha +\beta x_i +\gamma z_i +\varepsilon_i$$ and that $$\mathbb{E}[\varepsilon_i|x_i;z_i]=0$$. Therefore, by law of iterated expectations, $$\mathbb{E}[y|x=c]=\alpha +\beta c +\gamma \mathbb{E}[z|x=c]$$ and this may be nonlinear in $$c$$ if the mapping $$c\mapsto \mathbb{E}[z|x=c]$$ is nonlinear.