Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

The title sums up my question, but for clarity consider the following simple example. Let $X_i \overset{iid}{\backsim} \mathcal{N}(0, 1)$, $i = 1, ..., n$. Define: \begin{equation} S_n = \frac{1}{n} \sum_{i=1}^n X_i \end{equation} and \begin{equation} T_n = \frac{1}{n} \sum_{i=1}^n (X_i^2 - 1) \end{equation} My question: Even though $S_n$ and $T_n$ are perfectly dependent when $n = 1$, do $\sqrt{n} S_n$ and $\sqrt{n} T_n$ converge to a joint normal distribution as $n \rightarrow \infty$?

The motivation: My motivation for the question stems from the fact that it feels odd (but wonderful) that $S_n$ and $T_n$ are perfectly dependent when $n = 1$, yet the implication of the multivariate CLT is that they approach independence as $n \rightarrow \infty$ (this would follow since $S_n$ and $T_n$ are uncorrelated for all $n$, hence if they are asymptotically joint normal, then they must also be asymptotically independent).

Thanks in advance for any answers or comments!

ps, If you can provide any references etc then all the better!

share|improve this question
    
No answer, but a comment. I don't find this very surprising. The dependence you note for n = 1 quickly decreases as n goes up. –  Erik Nov 19 '12 at 11:25
    
@egbutter has provided a fine answer. If you are still looking for some alternative or some additional intuition, ping me and I will see about writing up something a little bit different. –  cardinal Nov 29 '12 at 1:04
    
@cardinal Thanks very much for the offer, but I'm fairly happy at this point - I awarded the bounty to egbutter. I think I've got the intuition. My main purpose in posting was to see if someone jumped in and said "No no no you've got it all wrong because of..." :-) Cheers. –  Colin T Bowers Nov 29 '12 at 3:52

2 Answers 2

up vote 6 down vote accepted
+50

The short answer as I understand your q is "yes, but ..." the rates of convergence on S, T, and any other moments are not necessarily the same -- check out determining bounds with the Berry-Esseen Theorem.

In case I misunderstand your q, Sn and Tn even hold to the CLT under conditions of weak dependence (mixing): check out Wikipedia's CLT for dependent processes.

CLT is such a general theorem -- the basic proof requires nothing more than the characteristic function of Sn and Tn converges to the characteristic function of the standard normal, then Levy Continuity Theorem says the convergence of the characteristic function implies convergence of the distribution.

John Cook provides a great explanation of CLT error here.

share|improve this answer
    
Thanks for the answer. I'm not really concerned by the rate of convergence as far as this question is concerned, nor whether the CLT will hold under more general conditions, eg dependence. What I was really hoping for is a reference or statement that justifies the use of the multivariate CLT when the ith component of each sum exhibit perfect contemporaneous dependence. I've subsequently found a reference in Davidson's "Stochastic Limit Theory" stating the multivariate CLT holds given arbitrary contemporaneous dependence, but am still looking for a little rigor around that statement. –  Colin T Bowers Nov 27 '12 at 7:10
    
It sounds like you are over-thinking this. Are your i in [1,n] the "contemporaneous" components you are referring to? If so, then the important point is that your Sn and Tn will still converge (you can prove this to yourself using the same method as the "old-school" CLT proof mentioned above) -- but for a given i, their errors will be different. That does not change the fact that CLT holds. The multi/univariate distinction is not important. –  egbutter Nov 27 '12 at 18:12
    
Yes, the i are the contemporaneous components. Good suggestion regarding running the example through a proof. I had actually done this, and didn't find any problems, which paradoxically made me more nervous. Perhaps I am over-thinking things at this point :-) Thanks again for the response. If no-one else has a crack at an answer by the end of the day, I'll mark your response the answer. Cheers. –  Colin T Bowers Nov 28 '12 at 0:02
    
I can certainly empathize -- I often do the same thing! :) –  egbutter Nov 28 '12 at 14:29

This doesn't prove anything, of course, but I always find doing simulations and plotting graphs to be very handy for making sense of theoretical results.

This is a particularly simple case. We generate $n$ random normal variates and compute $S_n$ and $T_n$; repeat $m$ times. Plotted are the graphs for $n = 1, 10, 100$ and $1000$. It's easy to see the dependence weakening as $n$ increases; at $n = 100$ the graph is almost indistinguishable from independence.

test <- function (m, n) 
{
    r <- matrix(rnorm(m * n), nrow = m)
    cbind(rowMeans(r), rowSums(r^2 - 1)/n)
}

par(mfrow=c(2,2))
plot(test(100, 1))
plot(test(100, 2))
plot(test(100, 5))
plot(test(100, 100))

enter image description here

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.