Question Details

If $\theta_{n, i} \stackrel{p}{\rightarrow} \theta$ for $i = 1, \dots ,m$, where $m$ is fixed, then does this imply

$$\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta?$$

Context: used as a lemma for other proofs.

Attempted Solution

Please point out mistakes.

By triangle inequality, $\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert \geq \lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \lvert$, which leads to

$$\mathbb{1}\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq \mathbb{1}\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

for any $\epsilon > 0$. Taking expectations of both sides

$$P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) \leq P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right)$$

Then applying probability union bound,

$$P\left(\sum_{i = 1}^{m}\lvert \theta_{n, i} - \theta \lvert > m\epsilon\right) \leq P\left(\bigcup_{i = 1}^{m}(\lvert \theta_{n, i} - \theta \lvert > \epsilon)\right) \leq \sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon).$$

Then since $\lim_{n \rightarrow \infty}P(\lvert \theta_{n, i} - \theta\lvert > \epsilon) = 0$ by assumption, $\lim_{n \rightarrow \infty}\sum_{i = 1}^{m}P(\lvert \theta_{n, i} - \theta \lvert > \epsilon) = 0$, which implies by above

$$\lim_{n \rightarrow \infty}P\left(\bigg\lvert \sum_{i = 1}^{m}(\theta_{n, i} - \theta) \bigg\lvert > m\epsilon\right) = 0 \iff \lim_{n \rightarrow \infty}P\left(\bigg\lvert \frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} - \theta \bigg\lvert > \epsilon\right) = 0$$

Thus proving that $\frac{1}{m}\sum_{i = 1}^{m}\theta_{n, i} \stackrel{p}{\rightarrow} \theta$


2 Answers 2


A simple way to see that this result is true is to use the continuous mapping theorem. We have $\theta_n \to \eta$ in probability where $\eta = (\underbrace{\theta, \ldots, \theta}_{\text{$m$ times}})$ and the mapping $g(\theta_n) = \frac{1}{m}\sum_{i=1}^m \theta_{n,i}$ is continuous. It follows that $g(\theta_n) \to g(\eta)$ in probability, i.e., $\frac 1 m \sum_{i=1}^m \theta_{n,i} \to \frac{1}{m} \sum_{i=1}^m \theta = \theta$. The same argument works with convergence in probability replaced with other modes of convergence.

I don't see anything wrong with the argument you presented, although you might want to be more explicit in how you are applying the union bound. Specifically, you have $[\sum_i |\theta_{n,i} - \theta| > m\epsilon] \subseteq \bigcup_i [|\theta_{n,i} - \theta| > \epsilon]$. And you should be more explicit in stating precisely what question you are trying to answer (it is not clear on a first read that $m$ is fixed). The result itself is false in general if $m$ is growing with $n$; in that case, you would need some additional assumption.

  • 1
    $\begingroup$ I assumed $m$ was growing with $n$ -- if it isn't, then as you say it's straightforward. $\endgroup$ Jul 15, 2020 at 23:38
  • 1
    $\begingroup$ @ThomasLumley - yes, $m$ is fixed and this assumption is explicitly added to the question. I agree with being more explicit in applying union bound and that applying continuous mapping theorem is more straightforward. Thanks! $\endgroup$ Jul 15, 2020 at 23:54

The proof is not valid. You argue that $$\lim_{n\to\infty} P(|\theta_{n,i}|>\epsilon)=0$$ implies $$\lim_{n\to\infty}\sum_{i=1}^m P(|\theta_{n,i}|>\epsilon)=0$$ which would fail if, for example, $P(|\theta_{n,i}|>\epsilon)=1/i.$

The hypothesis is not precisely stated, but say we mean for any $\epsilon>0$ there exist $M$ and $N$ such that if $i>M$ and $n>N$ we have $P(|\theta_{n,i}-\theta|>\epsilon)<\epsilon)$, which seems a reasonable definition.

The claim is false in general. Suppose $\theta=0$, for tidyness. Let $\theta_{n,i}=m$ if $i=1$ and $\theta_{n,i}=0$ otherwise. The hypothesis is true: $|\theta_{n,i}-\theta|=0$ for all $n$ and all $i>1$. The conclusion is false, since $$m^{-1}\sum_{i=1}^n\theta_{n,i}=1.$$

  • $\begingroup$ I do not believe this is correct because $m$ is not tending to $\infty$. Instead, $m$ is fixed. It is of course true that $\lim_{n\to\infty} \sum_{i=1}^m a_{n,i} \to 0$ if each of the $a_{n,i} \to 0$; this is a simple property of finite sums. Your counterexample is also not correct because $\theta_{n,1} \to 0$ in probability fails. $\endgroup$
    – guy
    Jul 15, 2020 at 23:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.