# Distribution of the “error term” in the central limit theorem

The classical central limit theorem states that if $X_1, X_2, \ldots, X_n$ are IID random variables with mean $\mu$ and standard deviation $\sigma$, then

${\lim}_{n \rightarrow \infty} \frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}} \rightarrow^{d} N(0, 1)$.

What can be said in general about the distribution, for fixed $n$, of the "error term"?

I'm not entirely sure how to formulate it, but I suppose I am interested in the following situation:

Suppose $A \sim N(0, 1)$ and $B \sim \frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}}$. What can be said about distributions $\epsilon$ such that $A \sim B + \epsilon$.

One way to do this is of course to let $\epsilon = A - B$. I.e. let $B$ be a random variable distributed as $\frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}}$ and let $\epsilon$ by a standard normal minus this particular $B$.

Perhaps this is the only way you can always do it. Suppose $n = 1$ and $X_1$ is a Bernoulli variable, then it seems that the only way to get to obtain a standard normal variable by translation is to subtract $X_1$ and add a standard normal variable.

My initial hope was to find a "small" $\epsilon$ that is independent of $B$, i.e. that $B$ is distributed as a standard normal up to "noise" of "small magnitude". Can one say anything about whether this is ever possible for certain $X$?

• See en.wikipedia.org/wiki/Berry%E2%80%93Esseen_theorem and en.wikipedia.org/wiki/Edgeworth_series#The_Edgeworth_series and mathoverflow.net/questions/141464/… . There are whole books on the subject - here's one books.google.com/… .. These mainly deal with asymptotic results as n increases, not for some fixed "small" n. – Mark L. Stone Sep 23 '16 at 23:33
• Please note that $A\sim B+\epsilon$ does not imply that $\epsilon=A-B$ (as random variables, which apparently is how "$A$", "$B$", and "$\epsilon$" must be understood). For instance, suppose $B$ and $\epsilon$ are iid standard Normal. Then the variance of $A$ is $2$. Since "$\sim$" appears intended to mean "has the same distribution as" rather than is equal to, it looks like we are to assume $A$ is independent of $B$ and $\epsilon$. But then $\epsilon=A-B$ would have a variance of $3$, which is impossible. Could you therefore explain your notation and assumptions? – whuber Sep 24 '16 at 15:46
• Thank you for the question, whuber. I realize I expressed myself unclearly. I have a random variable $B$, and what I am seeking is another random variable $\epsilon$ such that $B + \epsilon$ is distributed as a standard normal. What I meant above was that one (uninteresting) way to obtain this would be to choose $\epsilon$ to be equal to a standard normal minus (the specific variable) $B$. – Christian Sep 24 '16 at 18:42
• One interesting way is to look at the saddlepoint approximation, see stats.stackexchange.com/questions/191492/… and search this site! – kjetil b halvorsen Oct 12 '20 at 23:21