If $\theta_{n, 1}, \dots ,\theta_{n, m} \stackrel{p}{\rightarrow} \theta$, does $m^{-1}\sum_{i}\theta_{n, i}$ converge in probability to $\theta$?

Question Details

If $$\theta_{n, i} \stackrel{p}{\rightarrow} \theta$$ for $$i = 1, \dots ,m$$, where $$m$$ is fixed, then does this imply

$$\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta?$$

Context: used as a lemma for other proofs.

Attempted Solution

By triangle inequality, $$\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert \geq \lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \lvert$$, which leads to

$$\mathbb{1}\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq \mathbb{1}\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

for any $$\epsilon > 0$$. Taking expectations of both sides

$$P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

Then applying probability union bound,

$$P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right) \leq P\left(\bigcup_{i = 1}^{m}(\lvert \theta_{n, i} - \theta \lvert > \epsilon)\right) \leq \sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon).$$

Then since $$\lim_{n \rightarrow \infty}P(\lvert \theta_{n, i} - \theta\lvert > \epsilon) = 0$$ by assumption, $$\lim_{n \rightarrow \infty}\sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon) = 0$$, which implies by above

$$\lim_{n \rightarrow \infty}P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) = 0 \iff \lim_{n \rightarrow \infty}P\left(\bigg\lvert \frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} - \theta \bigg\lvert > \epsilon\right) = 0$$

Thus proving that $$\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta$$

A simple way to see that this result is true is to use the continuous mapping theorem. We have $$\theta_n \to \eta$$ in probability where $$\eta = (\underbrace{\theta, \ldots, \theta}_{\text{m times}})$$ and the mapping $$g(\theta_n) = \frac{1}{m}\sum_{i=1}^m \theta_{n,i}$$ is continuous. It follows that $$g(\theta_n) \to g(\eta)$$ in probability, i.e., $$\frac 1 m \sum_{i=1}^m \theta_{n,i} \to \frac{1}{m} \sum_{i=1}^m \theta = \theta$$. The same argument works with convergence in probability replaced with other modes of convergence.

I don't see anything wrong with the argument you presented, although you might want to be more explicit in how you are applying the union bound. Specifically, you have $$[\sum_i |\theta_{n,i} - \theta| > m\epsilon] \subseteq \bigcup_i [|\theta_{n,i} - \theta| > \epsilon]$$. And you should be more explicit in stating precisely what question you are trying to answer (it is not clear on a first read that $$m$$ is fixed). The result itself is false in general if $$m$$ is growing with $$n$$; in that case, you would need some additional assumption.

• I assumed $m$ was growing with $n$ -- if it isn't, then as you say it's straightforward. – Thomas Lumley Jul 15 at 23:38
• @ThomasLumley - yes, $m$ is fixed and this assumption is explicitly added to the question. I agree with being more explicit in applying union bound and that applying continuous mapping theorem is more straightforward. Thanks! – TheGrayGrunt Jul 15 at 23:54

The proof is not valid. You argue that $$\lim_{n\to\infty} P(|\theta_{n,i}|>\epsilon)=0$$ implies $$\lim_{n\to\infty}\sum_{i=1}^m P(|\theta_{n,i}|>\epsilon)=0$$ which would fail if, for example, $$P(|\theta_{n,i}|>\epsilon)=1/i.$$

The hypothesis is not precisely stated, but say we mean for any $$\epsilon>0$$ there exist $$M$$ and $$N$$ such that if $$i>M$$ and $$n>N$$ we have $$P(|\theta_{n,i}-\theta|>\epsilon)<\epsilon)$$, which seems a reasonable definition.

The claim is false in general. Suppose $$\theta=0$$, for tidyness. Let $$\theta_{n,i}=m$$ if $$i=1$$ and $$\theta_{n,i}=0$$ otherwise. The hypothesis is true: $$|\theta_{n,i}-\theta|=0$$ for all $$n$$ and all $$i>1$$. The conclusion is false, since $$m^{-1}\sum_{i=1}^n\theta_{n,i}=1.$$

• I do not believe this is correct because $m$ is not tending to $\infty$. Instead, $m$ is fixed. It is of course true that $\lim_{n\to\infty} \sum_{i=1}^m a_{n,i} \to 0$ if each of the $a_{n,i} \to 0$; this is a simple property of finite sums. Your counterexample is also not correct because $\theta_{n,1} \to 0$ in probability fails. – guy Jul 15 at 23:24