# convergence of an average of consistent estimators?

Let $$\frac{1}{n}\sum_{i=1}^n X_i^j \overset{\text{p}}{\to} \mu^j$$ for each $$j$$ (as $$n \to \infty$$ ). Under what conditions can we guarantee that $$\frac{1}{nm}\sum_{j=1}^m\sum_{i=1}^nX_i^j \overset{\text{p}}{\to} E[\mu]?$$

Is there a way to guarantee it that doesn't involve assuming uniform convergence in probability? The sequences are uniformly convergent in probability if $$\sup_{j} \left| \frac{1}{n}\sum_{i=1}^n X_i^j - \mu^j \right| \overset{\text{p}}{\to} 0.$$

• $E[\mu]$ is undefined. Is $m$ fixed or does $m \rightarrow \infty$? – Michael May 17 at 18:31
• @Michael I'm flexible on this, but what I had in mind was $m \to \infty$ and $\mu$ was a continuous random variable. I was thinking along the lines that $m$ was a multiple of $n$ or something – Taylor May 17 at 19:55

Assuming $$E[\mu] = \frac{1}{m} \sum_{j=1}^m \mu^j$$ and $$m \rightarrow \infty$$. If you're ok with passing to a subsequence $$n_m$$ so that $$P( | \frac{1}{n_m}\sum_{i=1}^{n_m} X_{i}^j - \mu^j | < \frac{1}{m^2} ) > 1 - \frac{1}{m^2},$$ then the result would hold for the subsequence (more precisely, sub-array).
In general, it is not true that $$\epsilon^j_n \stackrel{p}{\rightarrow} 0 \; \mbox{ as } n \rightarrow \infty, \;\; \forall j$$
implies $$\frac{1}{m} \sum_{j = 1}^m \epsilon^j_n \stackrel{p}{\rightarrow} 0 \; \mbox{ as } n, \; m \rightarrow \infty.$$
Indeed, it is not even true for deterministic sequences, which is a special case. Let $$\epsilon^j_n = j \cdot (\log (n+1) - \log(n)),$$ then $$\epsilon^j_n \rightarrow 0$$ as $$n \rightarrow \infty$$ for all $$j$$. Averaging across $$j$$ gives $$\frac{1}{m} \sum_{j = 1}^m \epsilon^j_n = O\left(m \cdot ( \log (n+1) - \log(n) ) \right)$$ which does not converge to $$0$$ as $$n, m \rightarrow 0$$.
(You can easily cook up a sequence $$X^j_n$$ so that $$\frac{1}{n}\sum_{i=1}^nX_n^j = \epsilon^j_n$$, making it a counter-example to the claim.)
\begin{align*} 0 &\le \left|\frac{1}{nm}\sum_{j=1}^m\sum_{i=1}^nX_i^j - E[\mu]\right| \\ &\le \left|\frac{1}{nm}\sum_{j=1}^m\sum_{i=1}^nX_i^j - m^{-1}\sum_{j=1}^m\mu^j \right| + \left|m^{-1}\sum_{j=1}^m\mu^j - E[\mu]\right| \\ &\le \sup_{\mu \in \Theta} \left|\frac{1}{n}\sum_{i=1}^n \bar{X}_\mu - m^{-1}\sum_{j=1}^m\mu^j \right| + \left|m^{-1}\sum_{j=1}^m\mu^j - E[\mu]\right|. \end{align*} With some nonstandard assumptions, you can invoke Theorem 2.1 of this to guarantee the first term converges to $$0$$. The second term converges to $$0$$ by the standard law of large numbers.