# Weak Law of Large Numbers: Conditional Expectations in Random Subsequences

Let $$(X_i, Y_i)_{i=1}^{\infty}$$ be iid continuous random vectors with continuous joint density, where $$X_1$$ have support $$\mathcal{X}$$. Let $$B_n\subset \mathcal{X}\subset\mathbb{R}$$ be decreasing subsets (open intervals) such that $$\cap B_n= x_0\in\mathcal{X}$$.

Let $$S = \{i\leq n: X_i\in B_n\}$$. I want to show that $$\frac{1}{|S|}\sum_{i\in S}Y_i \overset{P}{\to} \mathbb{E}[Y_1\mid X_1=x_0], \,\,as\,\,n\to\infty.$$ I assume that the necessary condition for this convergence is $$|S|\to\infty$$ or that $$nP(X_i\in B_n)\to\infty$$. Is it sufficient? Is there some theory that describes this?

Intuitively, this condition should be sufficient since, for large $$n$$ also $$B_n$$ will be large and $$\frac{1}{|S|}\sum_{i\in S}Y_i \approx \mathbb{E}[Y_1\mid X_1\in B_n]\approx \mathbb{E}[Y_1\mid X_1=x_0]$$. However proving this two-step intuition is harder than it looks (at least for me).

• Source of the problem? Apr 2 at 23:44
• @Zhanxiong I made it up, I need it to prove consistency of an estimator of mine Apr 3 at 9:14

It's not sufficient. Take $${\cal X}=[-1,1]$$ and let $$X$$ be uniform on $${\cal X}$$. Suppose the distribution of $$Y$$ given $$X=x$$ is a mixture of $$N(0,1)$$ with probability $$1-x$$ and $$N(1/x,1)$$ with probability $$x$$. For any non-zero $$x$$, $$E[Y|X=x]=1$$, but $$E[Y|X=x]=0$$

Now take $$x_0=0$$ and take $$B_n = (-n^{-1/2},n^{-1/2})$$. Your average will converge to 1, but the true value is zero.

I now need to show that this does actually have a continuous density. It's obviously continuous except at $$x=0$$. Consider any sequence of points $$(x_n,y_n)$$ converging to a limit $$(0,y)$$. For any fixed $$y$$ and all large enough $$n$$, the $$N(1/x_n,1)$$ component of the density will be essentially zero in a neighbourhood of $$Y=y$$, and it is exactly zero at $$x=0$$, so $$f(x_n,y_n)\to f(0,y)$$.

You will need some sort of uniform continuity, or some sort of bound on the tails of the conditional distributions of $$Y|X=x$$

• Thanks for the thought. But I still don't belive that such density is continuous. The argument '...component of the density will be essentially zero in a neighbourhood of $Y=y$...' is not valid, since this component will have less and less weight on it but will be larger and larger. I don't think that if joint density is continuous, then conditionals can have a jump. If you write your density down for $x=0.001$ and for $x=0$ and for $x=-0.001$ I think they will be shifted by a constant. Apr 3 at 9:19
• It's not uniformly continuous, but it is continuous at every $(x,y)$ Apr 3 at 22:56