The title sums up my question, but for clarity consider the following simple example. Let $X_i \overset{iid}{\backsim} \mathcal{N}(0, 1)$, $i = 1, ..., n$. Define: \begin{equation} S_n = \frac{1}{n} \sum_{i=1}^n X_i \end{equation} and \begin{equation} T_n = \frac{1}{n} \sum_{i=1}^n (X_i^2 - 1) \end{equation} My question: Even though $S_n$ and $T_n$ are perfectly dependent when $n = 1$, do $\sqrt{n} S_n$ and $\sqrt{n} T_n$ converge to a joint normal distribution as $n \rightarrow \infty$?

The motivation: My motivation for the question stems from the fact that it feels odd (but wonderful) that $S_n$ and $T_n$ are perfectly dependent when $n = 1$, yet the implication of the multivariate CLT is that they approach independence as $n \rightarrow \infty$ (this would follow since $S_n$ and $T_n$ are uncorrelated for all $n$, hence if they are asymptotically joint normal, then they must also be asymptotically independent).

Thanks in advance for any answers or comments!

ps, If you can provide any references etc then all the better!

  • $\begingroup$ No answer, but a comment. I don't find this very surprising. The dependence you note for n = 1 quickly decreases as n goes up. $\endgroup$
    – Erik
    Commented Nov 19, 2012 at 11:25
  • $\begingroup$ @egbutter has provided a fine answer. If you are still looking for some alternative or some additional intuition, ping me and I will see about writing up something a little bit different. $\endgroup$
    – cardinal
    Commented Nov 29, 2012 at 1:04
  • $\begingroup$ @cardinal Thanks very much for the offer, but I'm fairly happy at this point - I awarded the bounty to egbutter. I think I've got the intuition. My main purpose in posting was to see if someone jumped in and said "No no no you've got it all wrong because of..." :-) Cheers. $\endgroup$ Commented Nov 29, 2012 at 3:52

2 Answers 2


The short answer as I understand your q is "yes, but ..." the rates of convergence on S, T, and any other moments are not necessarily the same -- check out determining bounds with the Berry-Esseen Theorem.

In case I misunderstand your q, Sn and Tn even hold to the CLT under conditions of weak dependence (mixing): check out Wikipedia's CLT for dependent processes.

CLT is such a general theorem -- the basic proof requires nothing more than the characteristic function of Sn and Tn converges to the characteristic function of the standard normal, then Levy Continuity Theorem says the convergence of the characteristic function implies convergence of the distribution.

John Cook provides a great explanation of CLT error here.

  • $\begingroup$ Thanks for the answer. I'm not really concerned by the rate of convergence as far as this question is concerned, nor whether the CLT will hold under more general conditions, eg dependence. What I was really hoping for is a reference or statement that justifies the use of the multivariate CLT when the ith component of each sum exhibit perfect contemporaneous dependence. I've subsequently found a reference in Davidson's "Stochastic Limit Theory" stating the multivariate CLT holds given arbitrary contemporaneous dependence, but am still looking for a little rigor around that statement. $\endgroup$ Commented Nov 27, 2012 at 7:10
  • $\begingroup$ It sounds like you are over-thinking this. Are your i in [1,n] the "contemporaneous" components you are referring to? If so, then the important point is that your Sn and Tn will still converge (you can prove this to yourself using the same method as the "old-school" CLT proof mentioned above) -- but for a given i, their errors will be different. That does not change the fact that CLT holds. The multi/univariate distinction is not important. $\endgroup$
    – egbutter
    Commented Nov 27, 2012 at 18:12
  • $\begingroup$ Yes, the i are the contemporaneous components. Good suggestion regarding running the example through a proof. I had actually done this, and didn't find any problems, which paradoxically made me more nervous. Perhaps I am over-thinking things at this point :-) Thanks again for the response. If no-one else has a crack at an answer by the end of the day, I'll mark your response the answer. Cheers. $\endgroup$ Commented Nov 28, 2012 at 0:02
  • $\begingroup$ I can certainly empathize -- I often do the same thing! :) $\endgroup$
    – egbutter
    Commented Nov 28, 2012 at 14:29

This doesn't prove anything, of course, but I always find doing simulations and plotting graphs to be very handy for making sense of theoretical results.

This is a particularly simple case. We generate $n$ random normal variates and compute $S_n$ and $T_n$; repeat $m$ times. Plotted are the graphs for $n = 1, 10, 100$ and $1000$. It's easy to see the dependence weakening as $n$ increases; at $n = 100$ the graph is almost indistinguishable from independence.

test <- function (m, n) 
    r <- matrix(rnorm(m * n), nrow = m)
    cbind(rowMeans(r), rowSums(r^2 - 1)/n)

plot(test(100, 1))
plot(test(100, 2))
plot(test(100, 5))
plot(test(100, 100))

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.