4
$\begingroup$

Is there a meaningful sense in which a sequence of PMFs (of a corresponding sequence of real-valued random variables) can uniformly converge to a PDF? Intuitively, it seems like a strange question to consider (and that the answer would be no) since a PMF is a function $f: \mathbb{R} \mapsto [0, 1]$, while a PDF is a function $f: \mathbb{R} \mapsto [0, \infty)$.

However, what about the following case: Say that we have a sequence of standardized random variables, which are denoted by $\left\{X_n\right\}_{n \in \mathbb{N}}$. Denote the corresponding sequence of CDFs by $\left\{F_n(x)\right\}_{n \in \mathbb{N}}$ and let this sequence of CDFs uniformly converge in distribution to $F(x)$, where $F(x)$ is the standard Normal CDF. That is,

$$\text{For all } \epsilon > 0, \text{ there exists an } N \text{ such that } \left\lvert F_n(x) - F(x) \right\rvert < \epsilon \text{ for all } n > N \text{ and for all } x.$$

Also, denote the corresponding sequence of PMFs for the sequence of standardized random variables by $\left\{f_n(x)\right\}_{n \in \mathbb{N}}$ and denote the PDF of the aforementioned limiting standard Normal distribution by $f(x)$. The maximum density of a standard Normal PDF is $\frac{1}{\sqrt{2\pi}} \approx 0.3989$.

Therefore, in this particular case, where the density of the standard Normal has an upper bound of $\frac{1}{\sqrt{2\pi}}$, it seems that it might make sense to consider the conditions under which the sequence of PMFs $\left\{f_n(x)\right\}_{n \in \mathbb{N}}$ uniformly converges to $f(x)$. On the other hand, the statement that a sequence of standardized random variables, $\left\{X_n\right\}_{n \in \mathbb{N}}$, uniformly converges to a standard Normal can usually be equivalently stated as the sequence of centered random variables scaled by $\sqrt{n}$ uniformly converges in distribution to a Normal distribution with mean $0$ and variance equal to $\nu > 0$, where now the Normal density is back to being unbounded.

So my question is: Would it be sensible in the case of a sequence of standardized random variables to give conditions under which the sequence of PMFs uniformly converges to a PDF? And if so, what might those conditions be? Or is convergence of a sequence of PMFs to a PDF a nonsensical thing to consider in general?

$\endgroup$
3
  • $\begingroup$ @Xi'an, my understanding is that the Glivenko–Cantelli theorem pertains to convergence of distribution functions, but does not say anything about convergence of discrete mass functions to a continuous density. Or a I mistaken? $\endgroup$ Commented Jun 20, 2022 at 13:49
  • $\begingroup$ You asked a related question recently in which this convergence was already made use of in a meaningful way stats.stackexchange.com/questions/579246 $\endgroup$ Commented Jun 20, 2022 at 23:04
  • 1
    $\begingroup$ Or do you want, in this question, to have the especially convergence of the PMF towards the PDF and not just the CDFs of the two? For example how the binomial distribution can be estimated by a curve that is the function of the normal distribution (multiplied by some scale factor). $\endgroup$ Commented Jun 20, 2022 at 23:09

1 Answer 1

1
$\begingroup$

It is possible, but the big question is when you can interchange limits

Yes, this kind of convergence can definitely happen. For example, consider the sequence of discrete uniform distributions $\{ F_n |n \in \mathbb{N} \}$ where each $F_n$ is the CDF for a discrete uniform distribution on $\mathcal{U}_n$, given by:

$$\mathcal{U}_n \equiv \{ 0, \tfrac{1}{n},..., \tfrac{n-1}{n}, 1 \}.$$

It is simple to demonstrate that $F_n$ converges uniformly to the CDF $F$ of a continuous uniform distribution on the unit interval. The density function for the continuous uniform distribution can be written in terms of the sequence of discrete mass functions as:

$$f(x) = \frac{dF}{dx}(x) = \frac{d}{dx} \lim_{n \rightarrow \infty} F_n(x) = \frac{d}{dx} \lim_{n \rightarrow \infty} \sum_{x \leqslant r} f_n(r).$$

You will notice here that I have not interchanged the derivative and limit, so I do not assert that the density function is a limit of a transformation of the mass functions. If you were to investigate the conditions under which these can be interchanged in this type of case, this would go a long way to showing when the density function can be written as a limit of a transformation of the mass functions.


Alternatively, you can relate these functions as:

$$f(x) = \lim_{\epsilon \downarrow 0} \lim_{n \rightarrow \infty} \frac{\sum_{x \in \mathscr{X}_n(x, \epsilon)} f_n(x)}{2\epsilon} \quad \quad \quad \quad \quad \mathscr{X}_n(x, \epsilon) \equiv \Big\{ r \in \text{supp} f_n \Big| |x-r| < \epsilon \Big\}.$$

The main thing to bear in mind here is that the probabiltiy density is a limiting measure that shows the limiting ratio of the probability of falling in a small interval, divided by the length of that interval (taking limits as the interval size goes to zero). For the discrete case, the probability of falling within any interval can be written in terms of the probability mass function, which allows the type of form above in more general cases. Again, you will notice that I have not interchanged the limits here, so I do not assert that the density function is a limit of a transformation of the mass functions. Unfortunately you can't interchange these limits here, or you will get a zero density at any irrational value $x$. Nevertheless, the above form shows that it is possible to write the density function in terms of the sequence of mass functions.


On this matter, it is also worth noting some philosophical issues relating to the use of limits, infinite quantities, and continuums in mathematics. Not only is it possible that convergence can occur from the discrete case to the continuous, but many mathematicians regard the limit of the discrete case as the best illustration of the meaning of the continuous case. Indeed, there are some mathematicians who only accept the existence of finite objects and they regard continuums and functions over continuums as ---at best--- an approximation to functions over large discrete spaces.

$\endgroup$
7
  • 1
    $\begingroup$ Re "consequently:" that does not seems to follow. Convergence of a sequence of functions does not guarantee anything about convergence of their derivatives. Do you perhaps have some kind of generalization in mind? $\endgroup$
    – whuber
    Commented Jun 20, 2022 at 13:01
  • 1
    $\begingroup$ @Ben, thank you for that very helpful response. I don't follow how uniform convergence of F_n to F leads to convergence of a sequence of discrete mass functions to the density of F. Can you please explain? To make this argument, would we need to impose certain conditions? Sweeting's 1986 article "On a Converse to Scheffe's Theorem" imposes boundedness and asymptotic uniform equicontinuity on sequence of densities to ensure convergence of densities given convergence in distributions. But what would it mean for a sequence of discrete mass functions to be asymptotically uniformly equicontinuous? $\endgroup$ Commented Jun 20, 2022 at 13:45
  • 1
    $\begingroup$ I make no claims here about the general case; only that it is possible for this kind of convergence to occur in some cases. It appears to me to hold in the uniform case I give here, which should provide a starting point for a more general analysis. I would recommend you start by recognising that a density is just a limiting ratio of the probability of a small interval divided by the length of that interval (taking the limit as the interval size goes to zero). It is clearly possible to write such an interval probability using the underlying discrete mass function, as in the example above. $\endgroup$
    – Ben
    Commented Jun 20, 2022 at 22:16
  • 2
    $\begingroup$ Yes--but what you are really doing is taking limits of the distribution functions. Wouldn't your answer be far clearer, then, if it were couched in terms of the CDFs rather than mass functions and density functions? $\endgroup$
    – whuber
    Commented Jun 20, 2022 at 22:28
  • 1
    $\begingroup$ Fair enough: but isn't their question about uniform convergence? $\endgroup$
    – whuber
    Commented Jun 21, 2022 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.