If $\theta_{n, 1}, \dots ,\theta_{n, m} \stackrel{p}{\rightarrow} \theta$, does $m^{-1}\sum_{i}\theta_{n, i}$ converge in probability to $\theta$?

Question

Question Details

If $\theta_{n, i} \stackrel{p}{\rightarrow} \theta$ for $i = 1, \dots ,m$, where $m$ is fixed, then does this imply

$$\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta?$$

Context: used as a lemma for other proofs.

Attempted Solution

Please point out mistakes.

By triangle inequality, $\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert \geq \lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \lvert$, which leads to

$$\mathbb{1}\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq \mathbb{1}\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

for any $\epsilon > 0$. Taking expectations of both sides

$$P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

Then applying probability union bound,

$$P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right) \leq P\left(\bigcup_{i = 1}^{m}(\lvert \theta_{n, i} - \theta \lvert > \epsilon)\right) \leq \sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon).$$

Then since $\lim_{n \rightarrow \infty}P(\lvert \theta_{n, i} - \theta\lvert > \epsilon) = 0$ by assumption, $\lim_{n \rightarrow \infty}\sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon) = 0$, which implies by above

$$\lim_{n \rightarrow \infty}P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) = 0 \iff \lim_{n \rightarrow \infty}P\left(\bigg\lvert \frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} - \theta \bigg\lvert > \epsilon\right) = 0$$

Thus proving that $\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta$

Answer 1

A simple way to see that this result is true is to use the continuous mapping theorem. We have $\theta_n \to \eta$ in probability where $\eta = (\underbrace{\theta, \ldots, \theta}_{\text{$m$ times}})$ and the mapping $g(\theta_n) = \frac{1}{m}\sum_{i=1}^m \theta_{n,i}$ is continuous. It follows that $g(\theta_n) \to g(\eta)$ in probability, i.e., $\frac 1 m \sum_{i=1}^m \theta_{n,i} \to \frac{1}{m} \sum_{i=1}^m \theta = \theta$. The same argument works with convergence in probability replaced with other modes of convergence.

I don't see anything wrong with the argument you presented, although you might want to be more explicit in how you are applying the union bound. Specifically, you have $[\sum_i |\theta_{n,i} - \theta| > m\epsilon] \subseteq \bigcup_i [|\theta_{n,i} - \theta| > \epsilon]$. And you should be more explicit in stating precisely what question you are trying to answer (it is not clear on a first read that $m$ is fixed). The result itself is false in general if $m$ is growing with $n$; in that case, you would need some additional assumption.

Answer 2

The proof is not valid. You argue that $$\lim_{n\to\infty} P(|\theta_{n,i}|>\epsilon)=0$$ implies $$\lim_{n\to\infty}\sum_{i=1}^m P(|\theta_{n,i}|>\epsilon)=0$$ which would fail if, for example, $P(|\theta_{n,i}|>\epsilon)=1/i.$

The hypothesis is not precisely stated, but say we mean for any $\epsilon>0$ there exist $M$ and $N$ such that if $i>M$ and $n>N$ we have $P(|\theta_{n,i}-\theta|>\epsilon)<\epsilon)$, which seems a reasonable definition.

The claim is false in general. Suppose $\theta=0$, for tidyness. Let $\theta_{n,i}=m$ if $i=1$ and $\theta_{n,i}=0$ otherwise. The hypothesis is true: $|\theta_{n,i}-\theta|=0$ for all $n$ and all $i>1$. The conclusion is false, since $$m^{-1}\sum_{i=1}^n\theta_{n,i}=1.$$