6
$\begingroup$

Wikipedia states the following:
(https://en.wikipedia.org/wiki/Beta_distribution#Special_and_limiting_cases)
$$\lim_{n \to \infty} nB(1,n) = \operatorname{Exponential}(1).$$

I'm having trouble deriving this result. This is how far I get: $$B(1,n) = \frac{1}{\operatorname{Beta}(1,n)} (1-x)^{n-1} = \frac{\Gamma(1+n)}{\Gamma(1)\Gamma(n)} (1-x)^{n-1} = n(1-x)^{n-1} $$

I think I'm supposed to make use of the fact that $\lim_{n \to \infty} (1-\frac{x}{n})^{n} = e^{-x}$, but I don't see how.

$\endgroup$
7
  • 3
    $\begingroup$ I doubt Wikipedia states this, because it (obviously) isn't true: after all, Beta distributions have no probability beyond the interval $[0,1]$ but the Exponential does. You have to standardize the Beta distribution appropriately for this limit to be anything other than the distribution of a constant $0$. $\endgroup$
    – whuber
    Nov 20, 2017 at 20:57
  • 5
    $\begingroup$ The actual statement is $\lim_{n \to \infty}nB(1,n) = \dots$, slightly (but importantly!) different to yours. I assume this means $n$ times a $B(1,n)$ variate. $\endgroup$
    – jbowman
    Nov 20, 2017 at 20:59
  • 3
    $\begingroup$ The assertion you're trying to prove is one about the distribution of a random variable--it's not directly about the density function. Thus you should consider starting with what you know about a variable which is $n$ times one that has a Beta distribution and work from that as a starting point. $\endgroup$
    – whuber
    Nov 20, 2017 at 21:54
  • 6
    $\begingroup$ In short, let $X\sim \text{Beta}(1,n)$ and let $Y=nX$. Write down the pdf for $Y$. ... it should be clear where to go from there. It's possible the confusion was partly caused by some sloppiness in the form of the original statement from Wikipedia. $\endgroup$
    – Glen_b
    Nov 20, 2017 at 23:30
  • 3
    $\begingroup$ This link may help: en.wikibooks.org/wiki/Probability/… $\endgroup$ Nov 21, 2017 at 9:06

2 Answers 2

13
$\begingroup$

First, let's get a sense why this should be true. The density of a Beta$(1,n)$ variable which has been multiplied by $n$ should be proportional to

$$\left(\frac{x}{n}\right)^{1-1}\left(1 - \frac{x}{n}\right)^{n-1} \propto \left(1 - \frac{x}{n}\right)^{n-1} \approx e^{-x}$$

(for large $n$, anyway), so it surely looks exponential.


A rigorous and relatively elementary way to demonstrate the result is to work with the distribution function (CDF) $F_n$. That is, suppose $Y_n$ has a Beta$(1,n)$ distribution (for $n \gt 0$) and let $X_n=nY_n$. By definition,

$$F_n(x) = \Pr(X_n \le x) = \Pr(nY_n \le x) = \Pr\left(Y_n \le \frac{x}{n}\right).$$

When $0 \lt x \lt n$, this probability is given by the Beta integral, proportional to

$$\Pr\left(Y_n \le \frac{x}{n}\right) \propto \int_0^{x/n} (1-y)^{n-1} dy = -\frac{1}{n} (1-y)^n|_0^{x/n} \propto 1 - \left(1 - \frac{x}{n}\right)^n.\tag{*}$$

When $x \ge n$, this probability is $1$ (it's no longer given by the integral).

(Notice how freely we may drop any multiplicative constants, like that factor of $1/n$, that do not depend on $y$ or $x$: in the end we only need to establish that the limiting function rises from $0$ to a finite value in the limit as $x\to\infty$. The function then can be divided by that limiting value to produce a genuine distribution.)

The right hand side is often used to define the exponential in the sense that

$$e^{-x} = \lim_{n\to \infty} \left(1 - \frac{x}{n}\right)^n.$$

Since for any $x \gt 0$ and $n\to \infty$ eventually $x\lt n$, the limiting value of $F_n(x)$ is the limiting value of $(*)$: we don't have to worry about the fact that $F_n(x)=1$ when $n$ is small. Furthermore, for $x\le 0$, $F_n(x)=0$ always and so its limiting value obviously is $0$ in such cases.

Figure showing some of the distributions and their limit

To illustrate the analysis, this figure plots $F_1$ (blue), $F_2$ (red), $F_4$ (gold), $F_8$ (green), and the limiting distribution (dashed, in gray). Evidently the distributions $F_n$ converge down to their limiting value everywhere $x \gt 0$.

The normalizing constant turns out to be unity, because the limiting value of $1 - e^{-x}$ as $x\to\infty$ is $1$: it already is a valid distribution function.

This shows that $F_n(x)$ approaches $F(x)=1 - e^{-x}$ arbitrarily closely for any $x\gt 0$ for sufficiently large $n$ and otherwise is $0$. This is the standard Exponential distribution. Therefore whenever $(Y_n)$ is a sequence of random variables with Beta$(1,n)$ distributions, the distributions of the random variables $(nY_n)$ converge to the standard Exponential distribution, QED.


Addendum

It might be worthwhile to show what can go wrong with an analysis of densities (PDFs).

For any $n=1,2,\ldots,$ define a "uniform $n$-distribution" to be an equally-weighted mixture of Normal distributions, each with standard deviation $2^{-2n}$ and located at the odd multiples of $2^{-n}$: that is, at $k2^{-n}$ for $k=1, 3, \ldots, 2^n-1$. Here is a plot of densities of the uniform $n$-distributions for $n=1,2,3,4$:

Figure 2

The uniform $n$-distribution has $2^{n-1}$ spikes and those spikes are occupying exponentially narrower portions of the gaps between their peaks. Because this is a finite mixture of Normal distributions it has a very nice density which is bounded, nonzero, and infinitely differentiable everywhere--one could scarcely complain of any mathematical "pathology." However, this family has been constructed to ensure that the limit of these densities is almost everywhere zero. (This is not hard to prove, but the details might be distracting here, so I will rely on the figure to make the point.) Note that zero itself is a nice function, too: bounded and infinitely differentiable. It's just impossible to normalize it to unit area!

Nevertheless, this sequence of distribution functions does have a limiting distribution function: it is the (usual) Uniform$(0,1)$ distribution. Here is a picture corresponding to the previous one, showing their distribution functions in the same colors:

Figure 3

The limit is a uniform distribution because between any $0 \le a \lt b \le 1$ there are approximately $(b-a)2^{n-1}$ spikes, each with almost all its probability (totaling $2^{1-n}$) concentrated between $a$ and $b$, for a total probability close to $b-a$: that is nearly uniform. In the picture you see a sequence of staircase-like graphs with smaller and smaller steps squeezing down to the slanted ramp (gray dots): that's the Uniform$(0,1)$ CDF.

The problem isn't restricted to densities that converge to $0$. Pick $0 \lt p\lt 1$ and let $(X_n)$ be a sequence of random variables with distributions that are a mixture of $p$ times any absolutely continuous distribution $F$ you want and $1-p$ times the uniform $n$-distributions. This sequence of densities converges to $p$ times the density of $F$. Although that is a nonzero function, it's not a valid density because it integrates to $p$, not to $1$.

The moral is that even when a sequence of very nice density functions converges, it doesn't necessarily converge to a density function.

$\endgroup$
5
  • 1
    $\begingroup$ +1. The Addendum is very perceptive. But when you say "converges" there (e.g. in the last sentence), what exactly do you mean? I assume you mean convergence in distribution. I am wondering if some stronger notions of convergence would imply that densities are converging to the correct density too. $\endgroup$
    – amoeba
    Nov 22, 2017 at 15:30
  • 1
    $\begingroup$ @amoeba Thank you. I think I've been clear throughout that this is a convergence of distributions, not of random variables. (After all, that's what the question seems to be about.) In order to invoke stronger notions of convergence it seems we would have to add some assumptions about the original sequence of random variables $(X_n)$ or $(Y_n)$. I lack the imagination (or experience) to see how such a sequence of inter-related $B(1,n)$ variables would arise in a statistical setting. Do you have an application in mind? $\endgroup$
    – whuber
    Nov 22, 2017 at 15:36
  • $\begingroup$ No. To be honest, that was idle curiosity. $\endgroup$
    – amoeba
    Nov 22, 2017 at 15:51
  • 1
    $\begingroup$ Thanks whuber, that clarifies some things. If I understand correctly, for the limit of a sequence of cdfs to again be a cdf, we only have to check if the limiting function rises from 0 to a finite value in the limit as x→∞. We do not have to worry about the function integrating to 1, because of the normalization constant introduced by integration. The latter can be a problem when working with pdfs, though. $\endgroup$
    – Gitte
    Nov 24, 2017 at 21:42
  • 1
    $\begingroup$ That's the right idea. There's really two ideas in there. The most important is that CDFs behave more nicely than PDFs under limiting operations. The secondary idea is that (in part because of the first idea) we usually don't need to worry about tracking the normalizing constants in our calculations; we can figure out the constant at the end. $\endgroup$
    – whuber
    Nov 24, 2017 at 21:53
0
$\begingroup$

Thanks to the comments I can now formulate the answer myself, I think:

We know that Y=nX, so we can use a general formula for the pdf of a transformed variable (link of Christoph Hanck):

$\rho_Y(y)=\frac{\rho_X(x)}{|f'(x)|}$

$f'(x)=\frac{d(nx)}{dx}=n$

$\rho_Y(y) = \frac{n(1-y/n)^{n-1}}{n}$

$\lim_{n \to \infty} (1-y/n)^{n-1}=e^{y}$

$\endgroup$
1
  • 3
    $\begingroup$ This is useful as a heuristic to help appreciate why the limiting PDF should be exponential. However, it's not really correct. The problem is that exactly the same argument can be used to prove some obviously false results in very similar-looking cases. There is no general theorem that says that when a sequence of PDFs converges to some function, then that function should be the PDF of a limiting variable--or even of any random variable at all. To keep from making such mistakes, you need to analyze the characteristic function or the CDF (which is fairly easy to do in this case). $\endgroup$
    – whuber
    Nov 21, 2017 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.