Does the normal distribution converge to a certain distribution if the standard deviation grows without bounds? it appears to me that the pdf starts looking like a uniform distribution with bounds given by $[-2 \sigma, 2 \sigma]$. Is this true?

  • 2
    No, but in order to answer your question properly, we need to know what is your definition of convergence. Keep in mind that formal discussion is only possible when the right hand side is not changing. So, you can't establish convergence to Unifrom[$-\sigma,\sigma$] because your $\sigma$ is changing. Look up the formulation of CLT to see what I mean – Aksakal Jul 10 at 19:50
  • only if you wrap or truncate it to something $o(\sigma)$. – enthdegree Jul 12 at 0:13

A common mistake in probability is to think that a distribution is uniform because it looks visually flat when all its values are near zero. This is because we tend to see that $f(x)=0.001 \approx 0.000001=f(y)$ and yet $f(x)/f(y)=0.001/0.000001=1000$, i.e. a small interval around $x$ is 1000 times more likely than a small interval around $y$.

It's definitely not uniform on the entire real line in the limit, as there is no uniform distribution on $(-\infty,\infty)$. It's also not even approximately uniform on $[-2\sigma,2\sigma]$.

You can see the latter from the 68-95-99.7 rule you seem to be familiar with. If it were approximately uniform on $[-2\sigma,2\sigma]$, then the probability of being in $[0,\sigma]$ and $[\sigma,2\sigma]$ should be the same, as the two intervals are the same length. But this is not the case: $P([0,\sigma])\approx 0.68/2= 0.34$, yet $P([\sigma,2\sigma])\approx (0.95-0.68)/2 = 0.135$.

When viewed over the entire real line, this sequence of normal distributions doesn't converge to any probability distribution. There are a few ways to see this. As an example, the cdf of a normal with standard deviation $\sigma$ is $F_\sigma(x) = (1/2)(1+\mbox{erf}(x/\sqrt{2}\sigma)$, and $\lim_{\sigma\rightarrow\infty} F_\sigma(x) = 1/2$ for all $x$, which is not the cdf of any random variable. In fact, it's not a cdf at all.

The reason for this non-convergence boils down to "mass loss" is the limit. The limiting function of the normal distribution has actually "lost" probability (i.e. it has escaped to infinity). This is related to the concept of tightness of measures, which gives necessary conditions for a sequence of random variables to converge to another random variable.

  • 1
    The incorrect "it's" was "all it's values are near zero". The "It's" in "It's a common mistake" was correct. – Acccumulation Jul 11 at 16:09

Your statement the pdf starts looking like a uniform distribution with bounds given by $[−2σ,2σ]$ is not correct if you adjust $\sigma$ to match the wider standard deviation.

Consider this chart of two normal densities centred on zero. The red curve corresponds to a standard deviation of $1$ and the blue curve to a standard deviation of $10$, and it is indeed the case that the blue curve is almost flat on $[-2,2]$

enter image description here

but for the blue curve with $\sigma=10$, we should actually be looking at its shape on $[-20,20]$. Rescaling both the $x$-axis and $y$-axis by factors of $10$ gives this next plot, and you get exactly the same shape for the blue density in this later plot as the red density in the earlier plot

enter image description here

The other answers already here do a great job of explaining why Gaussian RVs don't converge to anything as the variance increases without bound, but I want to point out a seemingly-uniform property that such a collection of Gaussians does satisfy that I think might be enough for someone to guess that they are becoming uniform, but that turns out to not be strong enough to conclude that. $\newcommand{\len}{\text{len}}$

Consider a collection of random variables $\{X_1,X_2,\dots\}$ where $X_n \sim \mathcal N(0, n^2)$. Let $A = [a_1,a_2]$ be a fixed interval of finite length, and for some $c \in \mathbb R$ define $B = A +c$, i.e. $B$ is $A$ but just shifted over by $c$. For an interval $I = [i_1,i_2]$ define $\len (I) = i_2-i_1$ to be the length of $I$, and note that $\len(A) = \len(B)$.

I'll now prove the following result:

Result: $\vert P(X_n \in A) - P(x_n\in B)\vert \to 0$ as $n \to \infty$.

I call this uniform-like because it says that the distribution of $X_n$ increasingly has two fixed intervals of equal length having equal probability, no matter how far apart they may be. That's definitely a very uniform feature, but as we'll see this doesn't say anything about the actual distribution of the $X_n$ converging to a uniform one.

Pf: note that $X_n = n X_1$ where $X_1 \sim \mathcal N(0, 1)$ so $$ P(X_n \in A) = P(a_1 \leq n X_1 \leq a_2) = P\left(\frac{a_1}{n} \leq X_1 \leq \frac{a_2}n\right) $$ $$ = \frac{1}{\sqrt{2\pi}}\int_{a_1/n}^{a_2/n} e^{-x^2/2}\,\text dx. $$ I can use the (very rough) bound that $e^{-x^2/2} \leq 1$ to get $$ \frac{1}{\sqrt{2\pi}}\int_{a_1/n}^{a_2/n} e^{-x^2/2}\,\text dx \leq \frac{1}{\sqrt{2\pi}}\int_{a_1/n}^{a_2/n} 1\,\text dx $$ $$ = \frac{\text{len}(A)}{n\sqrt{2\pi}}. $$

I can do the same thing for $B$ to get $$ P(X_n \in B) \leq \frac{\text{len}(B)}{n\sqrt{2\pi}}. $$

Putting these together I have $$ \left\vert P(X_n \in A) - P(X_n \in B)\right\vert \leq \frac{\sqrt 2 \text{len}(A) }{n\sqrt{\pi}} \to 0 $$ as $n\to\infty$ (I'm using the triangle inequality here).

$\square$

How is this different from $X_n$ converging on a uniform distribution? I just proved that the probabilities given to any two fixed intervals of the same finite length get closer and closer, and intuitively that makes sense that as the densities are "flattening out" from $A$ and $B$'s perspectives.

But in order for $X_n$ to converge on a uniform distribution, I'd need $P(X_n \in I)$ to head towards being proportional to $\text{len}(I)$ for any interval $I$, and that is a very different thing because this needs to apply to any $I$, not just one fixed in advance (and as mentioned elsewhere, this is also not even possible for a distribution with unbounded support).

  • Right, you could almost say they converged in distribution, except that the limit of what they converge to is an improper distribution. One type of convergence that would be well defined is I would guess you could show the Wasserstein metric will approach zero as $\sigma \rightarrow \infty$? – Cliff AB Jul 11 at 18:25

Your question is fundamentally flawed. The standard normal distribution is scaled so that $\sigma = 1$. So for some other Gaussian distribution ($\mu = 0, \sigma = \sigma^*$) then the curve between bounds $[-2\sigma^*, 2\sigma^*]$ has the same shape as the standard normal distribution. The only difference is the scaling factor. So if you rescale the Gaussian by dividing by $\sigma^*$, then you end up with the standard normal distribution.

Now if you have a Gaussian distribution ($\mu = 0, \sigma = \sigma^*$) then yes as $\sigma^* \rightarrow \infty$, the region between $[-2, 2]$ becomes increasing flatter.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.