Asymptotic normality of random vector

Question

If each component of a random vector is asymptotically normal by the central limit theorem (CLT), can anything be said about the asymptotic distribution of the random vector? The components are not independently distributed and the (finite) covariance matrix of the random vector is known.

Answer 1

Say you have a bivariate estimator $\hat \theta = (\hat \theta_1, \hat \theta_2)$ and

$$\hat \theta_1 = \sum_{i=1}^n a_iX_i \to_d Z_1 \sim N(\theta_1,v_1),\;\;\; \hat \theta_2 = \sum_{i=1}^n b_iX_i\to_d Z_2 \sim N(\theta_1,v_2)$$

then you can apply the Cramer-Wold theorem, namely examine an arbitrary linear combination of $\hat \theta_1, \hat \theta_2$, except the zero vector,

$$W = c_1\hat \theta_1 + c_2 \hat \theta_2 = \sum_{i=1}^n (c_1a_i +c_2b_1)X_i$$

If

$$W \to_d c_1Z_1 + c_2Z_2$$

Then the bivariate vector converges to the bivariate Normal distribution.

Answer 2

Comments to the question indicate the interest focuses on random variables that are unbiased linear estimators. Despite this restriction, in order to draw conclusions about the full bivariate distribution of the two estimators (suitably standardized as in the CLT), you need some control over the coefficients in the linear combinations. In short, the answer is "no, not without additional assumptions."

To show why this is, I will construct realistic counterexamples involving unbiased estimators of parameters of a Normal distribution. This requires only a straightforward sequence of computations of expectations and variances, but I'll show the details to make it clear (as well as to expose any assumptions).

---

Consider the textbook case of estimating the mean $\mu$ from a sample $X_1, \ldots, X_n$ of a Normal$(\mu,\sigma)$ distribution. A good estimator is the sample mean

$$t_n(X_1, \ldots, X_n) = \sum_{i=1}^n \frac{1}{n} X_i$$

which I have written explicitly as a linear combination, using notation that highlights that the coefficients depend on the sample size $n$ (of course!).

In the same circumstance we may construct a nontrivial linear estimator of zero by choosing a sequence of coefficients $c_{n ,i},$ $i=1,2,\ldots, n$ that (a) sum to zero: $$\sum_{i=1}^n c_{n,i}=0$$ and (b) are not all zero so that $$\sum_{i=1}^n c_{n,i}^2 \ne 0.$$ Given a family of such sequences (indexed by the sample size $n$), let

$$u_n(X_1, \ldots, X_n) = \sum_{i=1}^n c_{n,i} X_i.$$

Because the $X_i$ are independent, the random vector $(X_1,\ldots, X_n)$ has a multivariate Normal distribution, whence so does any linear combination like $t_n$ and $u_n.$ Consequently, for each $n,$ $(t_n,u_n)$ has a bivariate Normal distribution. It is determined by its mean

$$E(t_n,u_n) = E\left(\sum_{i=1}^n \frac{1}{n}X_i, \sum_{j=1}^n c_{n,j} X_j\right) = \left(\sum_{i=1}^n \frac{1}{n}\mu, \sum_{j=1}^n c_{n,j} \mu\right) = (\mu, 0)$$

(thereby demonstrating $u_n$ is an unbiased linear estimator of zero) and by its covariance matrix, which we may compute as

$$\operatorname{Var}(t_n) = \sum_{i=1}^n \sum_{j=1}^n \frac{1}{n}\frac{1}{n} \operatorname{Cov}(X_i,X_j) = \left(\frac{1}{n}\right)^2\sum_{i,j=1}^n \delta_{ij}\sigma^2 = \sigma^2\frac{1}{n};$$

$$\operatorname{Var}(u_n) = \sum_{i=1}^n \sum_{j=1}^n c_{n,i}c_{n,j}\operatorname{Cov}(X_i,X_j) = \sum_{i,j=1}^n \delta_{ij}c_{n,i}c_{n,j}\sigma^2 = \sigma^2\sum_{i=1}^n c_{n,i}^2 \ne 0;$$

and

$$\operatorname{Cov}(t_n, u_n) = \sum_{i=1}^n \sum_{j=1}^n \frac{1}{n}c_{n,j} \operatorname{Cov}(X_i,X_j) = \frac{1}{n}\sum_{i,j=1}^n \delta_{ij}c_{n,j}\sigma^2 = \frac{\sigma^2}{n}\sum_{i=1}^n c_{n,i} = 0.$$

Because the variances are nonzero and finite and the covariance is zero, $(t_n,u_n)$ are independent bivariate Normal variables.

We are going to construct more estimators of $\mu.$ To this end, let $(\alpha_n)$ be any sequence of real numbers. From this form the sequence

$$\lambda_n = \frac{\alpha_n}{\sqrt{n\sum_{i=1}^n c_{n,i}^2}}$$

so that $$\operatorname{Var}(\lambda_n u_n) = \lambda^2\operatorname{Var}(u_n) = \alpha_n^2/n.$$ Define the estimator

$$s_n = t_n + \lambda_n u_n = \sum_{i=1}^n \left(\frac{1}{n} + \lambda_n c_{n,i}\right)X_i,$$

which is manifestly a linear combination of the $X_i.$ From the foregoing its expectation is $E(s_n) = E(t_n) + \lambda_n E(u_n) = \mu,$ showing it is an unbiased estimator of $\mu,$ and the relevant second moments are

$$\operatorname{Var}(s_n) = \sigma^2 \frac{1}{n}\left(1 + \alpha_n^2\right);\quad \operatorname{Cov}(t_n,s_n) = \sigma^2 \frac{1}{n}.$$

The correlation of $(t_n,s_n)$ therefore is

$$\operatorname{\rho}(t_n,s_n) = \frac{\operatorname{Cov}(t_n,s_n)}{\sqrt{\operatorname{Var}(t_n)\operatorname{Var}(s_n)}} = \frac{1}{\sqrt{1+ \alpha_n^2}}.$$

Consequently,

When we choose a sequence $(\alpha_n)$ where $\alpha_n^2$ has no limit, the sequence of bivariate Normal distributions of the standardized estimators $(t_n, s_n)$ cannot possibly have a limit, either.

The reason is clear: this sequence of bivariate Normal distributions is a sequence of distributions of mean $(0,0)$ (from the standardization), unit variances (also from the standardization), and correlation coefficients $(1+\alpha_n^2)^{-1/2},$ which do not converge.

An explicit example of such a sequence is $\alpha_n=0$ for $n$ even (this corresponds to using the estimator $t_n$ when the sample size is even) and $\alpha_n=1$ for $n$ odd (which uses a modified estimator when $n$ is odd). The distribution of $(t_n, s_n)$ oscillates between having perfect correlation when $n$ is even--a singular distribution supported on the diagonal line in the plane--and having correlation $\rho = (1+1^2)^{-1/2}=\sqrt{1/2}$ when $n$ is odd, which is supported throughout the plane.