Show that bias term involving an indicator function convergences to zero

Question

Assume that we have $N$ observations of i.i.d. data $(Y_i,X_i)_{i=1}^{N}$. We want to learn the model given by $Y=f(X)+\epsilon$. We use the data to estimate $\hat{f}$ using any machine learning algorithm, e.g. random forests or neural nets. We assume that our method is consistent, i.e. $\hat{f}\xrightarrow{p}f$, but we do not assume anything about the convergence rate. In particular, we do not assume that $\hat{f}$ is root-N consistent.

We consider a bias term that arises from a particular estimator (I don't want to bore you with the details). The bias term reads

$bias=\sqrt{N}\mathbb{E}\left[Y_i\left(1_{\left\{ f\left(X_{i}\right)<c\right\} }-1_{\left\{ \hat{f}\left(X_{i}\right)<c\right\} }\right)\right],$

where $c$ is a known constant.

I want to prove that $bias=o_p(1)$, i.e. as $N\xrightarrow{}\infty$, we have $bias\xrightarrow{p}0$. We can assume that $X$ has a density. If we didn't have the indicator functions, it would be impossible to show without assume root-N consistency, but I wonder whether we can show this due to the indicator function. For instance, we may assume that $\text{Pr}\left(f\left(X\right)=c\right)=0$.

Question: How would you prove $bias=o_p(1)$ (if it is at all)?

Answer 1

Maybe something like the following treatment:

Define the random variable

$$Z_i(N) \equiv \left(1_{\left\{ f\left(X_{i}\right)<c\right\} }-1_{\left\{ \hat{f}\left(X_{i}\right)<c\right\} }\right) \implies Z_i(N) \in \{-1,0,1\}$$

Since the estimator is consistent, we also have that

$$Z_i(N) \to_p 0 \implies Z_i(N) \to_d 0$$

Then

$$ \text{bias} = -\mathbb{E}\left[Y_i\mid \{Z_i(N) =-1\}\right]\cdot \sqrt{N} \cdot P_N(Z_i=-1) + \mathbb{E}\left[Y_i\mid\{ Z_i(N) =1\}\right]\cdot \sqrt{N}\cdot P_N(Z_i=1)$$

So one needs for zero bias at the limit, that

$$\sqrt{N} \cdot P_N(Z_i=-1) \to 0,\qquad \sqrt{N}\cdot P_N(Z_i=1) \to 0$$

or,

$$ \exists\, N^*, \,\delta >0 : N> N^*\implies P_N(Z_i=-1) \leq \frac{1}{\sqrt{N}\cdot N^\delta}$$

and the same for the other probability. But this requirement will eventually hold for some finite $N^*$, since these probabilities will not "jump down" to zero from some strictly positive value.

PS: Evidently, we need also to assume that the conditional expected values remain finite.