Sum of random variables without normalization approaches gaussian

Question

The central limit theorem states that the limiting distribution of a centered and normalized sum of independent random variables with mean $\mu$ and finite variance $\sigma^2$ is Gaussian.

$$ \frac{\sum_{i=1}^n(X_i-\mu)}{\sigma\sqrt{n}}\xrightarrow{d}N(0,1) $$

However in practice, we may not be working with sums of centered and normalized random variables. Still, if we run experiments where we sum without normalization, the distribution of the sum can look increasingly Gaussian with increasing mean and variance. The statement

$$ \sum_{i=1}^nX_i\xrightarrow{d}N(n\mu,n\sigma^2) $$

would capture this intuition, but doesn't make sense because the "$\xrightarrow{d}$" is a claim in the limit as $n\rightarrow\infty$, and it doesn't make sense to talk about a Gaussian with infinite mean and variance.

Is there a theorem capturing the notion that the distribution of an uncentered and unnormalized sum still approaches a Gaussian? Or is this simply a corollary of the CLT? I'm looking for a proof of something like the following statement:

For a given $\delta > 0$, there exists an $N>0$ such that

$$ \text{distance}\left(\sum_i^nX_i, N(n\mu,n\sigma^2)\right)<\delta $$

for $n>N$ and some distance function of the distributions. That is, if we sum enough random variables, we can get as close to a Gaussian as we like.

Answer 1

If you want to talk about behavior in finite samples, you'd need to go to something like the Berry-Esséen inequality.

While the inequality is stated with the variables in standardized form, the bound on the difference in cdf isn't affected by the horizontal scaling factor nor by a shift.

This isn't specifically convergence because we're dealing with some particular $n$ but it may do what you need for a particular sense of "close to Gaussian"

Note, however, that a close-to-Gaussian cdf implied by a small bound on the difference in cdf doesn't imply Gaussian-like behavior of the variable. For example, it's possible for a variable that has at most a miniscule absolute deviation from a Gaussian (e.g.bounded by some fixed but small $\epsilon >0$) to have infinite variance and no mean.