The classical central limit theorem states that if $X_1, X_2, \ldots, X_n$ are IID random variables with mean $\mu$ and standard deviation $\sigma$, then
${\lim}_{n \rightarrow \infty} \frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}} \rightarrow^{d} N(0, 1)$.
What can be said in general about the distribution, for fixed $n$, of the "error term"?
I'm not entirely sure how to formulate it, but I suppose I am interested in the following situation:
Suppose $A \sim N(0, 1)$ and $B \sim \frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}}$. What can be said about distributions $\epsilon$ such that $A \sim B + \epsilon$.
One way to do this is of course to let $\epsilon = A - B$. I.e. let $B$ be a random variable distributed as $\frac{\frac{1}{n}\sum_{i=1}^n X_i - \mu}{\sqrt{\frac{\sigma^2}{n}}}$ and let $\epsilon$ by a standard normal minus this particular $B$.
Perhaps this is the only way you can always do it. Suppose $n = 1$ and $X_1$ is a Bernoulli variable, then it seems that the only way to get to obtain a standard normal variable by translation is to subtract $X_1$ and add a standard normal variable.
My initial hope was to find a "small" $\epsilon$ that is independent of $B$, i.e. that $B$ is distributed as a standard normal up to "noise" of "small magnitude". Can one say anything about whether this is ever possible for certain $X$?
