Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have often read that thanks to the CLT, the residuals of a model are asymptotically normal. This argument always seemed odd to me since CLT states that
The sum of a number of independent and identically distributed random variables are asymptotically distributed.
Since a residual is generally a difference and not a sum of iid random variables, how can we can claim that the CLT applies?
What i think you are referring to is that estimators are often asymptotically normal with increase of the sample size. It's a mathematically correct statement that follows from more general formulation of CLT, Lyapunov's CLT. It doesn't require having identical distribution of variables, just independence. For example, OLS estimator is $\hat{\beta}=\beta+P_{x}\epsilon$, where projection operator $P_{x}=(X^{T}X)^{-1}X^{T}$, vector of errors is $\epsilon$. In other words the difference from the actual $\beta$ is linear combination of errors, which are independently distributed. Hence by Lyapunov CLT this difference is asympyotically normally distributed.
In terms of what you literally wrote, i.e. "residuals of a model are asymptotically normal" it is not clear what asymptotic limit we are talking about here. There is a sense in which errors themselves could be approximately normal, i.e. when they are modeled as a sum of very large number of unknown factors. When this assumption is applicable, once again Lyapunov CLT suggests they are approximately normal. Again, i am not sure that is what you are asking, because strictly there is no asymptotical equality here. Also, i suspect there is another confused notion about squared residuals being distributed normally. It would be more optimal to ask the question more specifically.
You are right to be skeptical - The central limit theorem (CLT) says nothing about the distribution of the residuals in a statistical model.$^\dagger$ They might be normal, or not, depending on the underlying distribution of the error terms. The CLT is a theorem that concerns the asymptotic distribution of the standardised sample mean, and other standardised quantities formed by summation of variables. It is unfortunately misinterpreted in such an expansive way that it is attributed to all sorts of quantities it does not apply to. Misinterpretation of the CLT as an assertion of asymptotic normality of the underlying values used in model is unfortunately a ubiquitous error.
$^\dagger$ The one exception to this would be if the error terms in the model are themselves formed as a sum of lots of other IID quantities, etc., such that the CLT would apply to the distribution of the error terms. Although theoretically possible, this is not how statistical models are generally framed.
the residuals of a model are asymptotically normal.
The residuals cannot be asymptotically normal. The phrasing doesn't make a sense. What can be asymptotically normal is some function of those residuals such as the coefficient estimate, so that when the sample size increases the function starts to act like a normal distributed variable.
When you increase the sample size residuals do not suddenly turn into normal. It would be like saying that under CLT in a large sample i.i.d. random variables become normally distributed, they do not! A properly chosen linear combination of random variables will become normal under the conditions of CLT, but not the random variables themselves
