Let $(X_i, Y_i)_{i=1}^{\infty}$ be iid continuous random vectors with continuous joint density, where $X_1$ have support $\mathcal{X}$. Let $B_n\subset \mathcal{X}\subset\mathbb{R}$ be decreasing subsets (open intervals) such that $\cap B_n= x_0\in\mathcal{X}$.

Let $S = \{i\leq n: X_i\in B_n\}$. I want to show that $$ \frac{1}{|S|}\sum_{i\in S}Y_i \overset{P}{\to} \mathbb{E}[Y_1\mid X_1=x_0], \,\,as\,\,n\to\infty. $$ I assume that the necessary condition for this convergence is $|S|\to\infty$ or that $nP(X_i\in B_n)\to\infty$. Is it sufficient? Is there some theory that describes this?

Intuitively, this condition should be sufficient since, for large $n$ also $B_n$ will be large and $\frac{1}{|S|}\sum_{i\in S}Y_i \approx \mathbb{E}[Y_1\mid X_1\in B_n]\approx \mathbb{E}[Y_1\mid X_1=x_0]$. However proving this two-step intuition is harder than it looks (at least for me).

  • $\begingroup$ Source of the problem? $\endgroup$
    – Zhanxiong
    Apr 2 at 23:44
  • $\begingroup$ @Zhanxiong I made it up, I need it to prove consistency of an estimator of mine $\endgroup$ Apr 3 at 9:14

1 Answer 1


It's not sufficient. Take ${\cal X}=[-1,1]$ and let $X$ be uniform on ${\cal X}$. Suppose the distribution of $Y$ given $X=x$ is a mixture of $N(0,1)$ with probability $1-x$ and $N(1/x,1)$ with probability $x$. For any non-zero $x$, $E[Y|X=x]=1$, but $E[Y|X=x]=0$

Now take $x_0=0$ and take $B_n = (-n^{-1/2},n^{-1/2})$. Your average will converge to 1, but the true value is zero.

I now need to show that this does actually have a continuous density. It's obviously continuous except at $x=0$. Consider any sequence of points $(x_n,y_n)$ converging to a limit $(0,y)$. For any fixed $y$ and all large enough $n$, the $N(1/x_n,1)$ component of the density will be essentially zero in a neighbourhood of $Y=y$, and it is exactly zero at $x=0$, so $f(x_n,y_n)\to f(0,y)$.

You will need some sort of uniform continuity, or some sort of bound on the tails of the conditional distributions of $Y|X=x$

  • $\begingroup$ Thanks for the thought. But I still don't belive that such density is continuous. The argument '...component of the density will be essentially zero in a neighbourhood of $Y=y$...' is not valid, since this component will have less and less weight on it but will be larger and larger. I don't think that if joint density is continuous, then conditionals can have a jump. If you write your density down for $x=0.001$ and for $x=0$ and for $x=-0.001$ I think they will be shifted by a constant. $\endgroup$ Apr 3 at 9:19
  • $\begingroup$ It's not uniformly continuous, but it is continuous at every $(x,y)$ $\endgroup$ Apr 3 at 22:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.