# Show that bias term involving an indicator function convergences to zero

Assume that we have $$N$$ observations of i.i.d. data $$(Y_i,X_i)_{i=1}^{N}$$. We want to learn the model given by $$Y=f(X)+\epsilon$$. We use the data to estimate $$\hat{f}$$ using any machine learning algorithm, e.g. random forests or neural nets. We assume that our method is consistent, i.e. $$\hat{f}\xrightarrow{p}f$$, but we do not assume anything about the convergence rate. In particular, we do not assume that $$\hat{f}$$ is root-N consistent.

We consider a bias term that arises from a particular estimator (I don't want to bore you with the details). The bias term reads

$$bias=\sqrt{N}\mathbb{E}\left[Y_i\left(1_{\left\{ f\left(X_{i}\right)

where $$c$$ is a known constant.

I want to prove that $$bias=o_p(1)$$, i.e. as $$N\xrightarrow{}\infty$$, we have $$bias\xrightarrow{p}0$$. We can assume that $$X$$ has a density. If we didn't have the indicator functions, it would be impossible to show without assume root-N consistency, but I wonder whether we can show this due to the indicator function. For instance, we may assume that $$\text{Pr}\left(f\left(X\right)=c\right)=0$$.

Question: How would you prove $$bias=o_p(1)$$ (if it is at all)?

Maybe something like the following treatment:

Define the random variable

$$Z_i(N) \equiv \left(1_{\left\{ f\left(X_{i}\right)

Since the estimator is consistent, we also have that

$$Z_i(N) \to_p 0 \implies Z_i(N) \to_d 0$$

Then

$$\text{bias} = -\mathbb{E}\left[Y_i\mid \{Z_i(N) =-1\}\right]\cdot \sqrt{N} \cdot P_N(Z_i=-1) + \mathbb{E}\left[Y_i\mid\{ Z_i(N) =1\}\right]\cdot \sqrt{N}\cdot P_N(Z_i=1)$$

So one needs for zero bias at the limit, that

$$\sqrt{N} \cdot P_N(Z_i=-1) \to 0,\qquad \sqrt{N}\cdot P_N(Z_i=1) \to 0$$

or,

$$\exists\, N^*, \,\delta >0 : N> N^*\implies P_N(Z_i=-1) \leq \frac{1}{\sqrt{N}\cdot N^\delta}$$

and the same for the other probability. But this requirement will eventually hold for some finite $$N^*$$, since these probabilities will not "jump down" to zero from some strictly positive value.

PS: Evidently, we need also to assume that the conditional expected values remain finite.