Can a sequence of PMFs converge to a PDF?

Question

Is there a meaningful sense in which a sequence of PMFs (of a corresponding sequence of real-valued random variables) can uniformly converge to a PDF? Intuitively, it seems like a strange question to consider (and that the answer would be no) since a PMF is a function $f: \mathbb{R} \mapsto [0, 1]$, while a PDF is a function $f: \mathbb{R} \mapsto [0, \infty)$.

However, what about the following case: Say that we have a sequence of standardized random variables, which are denoted by $\left\{X_n\right\}_{n \in \mathbb{N}}$. Denote the corresponding sequence of CDFs by $\left\{F_n(x)\right\}_{n \in \mathbb{N}}$ and let this sequence of CDFs uniformly converge in distribution to $F(x)$, where $F(x)$ is the standard Normal CDF. That is,

$$\text{For all } \epsilon > 0, \text{ there exists an } N \text{ such that } \left\lvert F_n(x) - F(x) \right\rvert < \epsilon \text{ for all } n > N \text{ and for all } x.$$

Also, denote the corresponding sequence of PMFs for the sequence of standardized random variables by $\left\{f_n(x)\right\}_{n \in \mathbb{N}}$ and denote the PDF of the aforementioned limiting standard Normal distribution by $f(x)$. The maximum density of a standard Normal PDF is $\frac{1}{\sqrt{2\pi}} \approx 0.3989$.

Therefore, in this particular case, where the density of the standard Normal has an upper bound of $\frac{1}{\sqrt{2\pi}}$, it seems that it might make sense to consider the conditions under which the sequence of PMFs $\left\{f_n(x)\right\}_{n \in \mathbb{N}}$ uniformly converges to $f(x)$. On the other hand, the statement that a sequence of standardized random variables, $\left\{X_n\right\}_{n \in \mathbb{N}}$, uniformly converges to a standard Normal can usually be equivalently stated as the sequence of centered random variables scaled by $\sqrt{n}$ uniformly converges in distribution to a Normal distribution with mean $0$ and variance equal to $\nu > 0$, where now the Normal density is back to being unbounded.

So my question is: Would it be sensible in the case of a sequence of standardized random variables to give conditions under which the sequence of PMFs uniformly converges to a PDF? And if so, what might those conditions be? Or is convergence of a sequence of PMFs to a PDF a nonsensical thing to consider in general?

Answer 1

It is possible, but the big question is when you can interchange limits

Yes, this kind of convergence can definitely happen. For example, consider the sequence of discrete uniform distributions $\{ F_n |n \in \mathbb{N} \}$ where each $F_n$ is the CDF for a discrete uniform distribution on $\mathcal{U}_n$, given by:

$$\mathcal{U}_n \equiv \{ 0, \tfrac{1}{n},..., \tfrac{n-1}{n}, 1 \}.$$

It is simple to demonstrate that $F_n$ converges uniformly to the CDF $F$ of a continuous uniform distribution on the unit interval. The density function for the continuous uniform distribution can be written in terms of the sequence of discrete mass functions as:

$$f(x) = \frac{dF}{dx}(x) = \frac{d}{dx} \lim_{n \rightarrow \infty} F_n(x) = \frac{d}{dx} \lim_{n \rightarrow \infty} \sum_{x \leqslant r} f_n(r).$$

You will notice here that I have not interchanged the derivative and limit, so I do not assert that the density function is a limit of a transformation of the mass functions. If you were to investigate the conditions under which these can be interchanged in this type of case, this would go a long way to showing when the density function can be written as a limit of a transformation of the mass functions.

---

Alternatively, you can relate these functions as:

$$f(x) = \lim_{\epsilon \downarrow 0} \lim_{n \rightarrow \infty} \frac{\sum_{x \in \mathscr{X}_n(x, \epsilon)} f_n(x)}{2\epsilon} \quad \quad \quad \quad \quad \mathscr{X}_n(x, \epsilon) \equiv \Big\{ r \in \text{supp} f_n \Big| |x-r| < \epsilon \Big\}.$$

The main thing to bear in mind here is that the probabiltiy density is a limiting measure that shows the limiting ratio of the probability of falling in a small interval, divided by the length of that interval (taking limits as the interval size goes to zero). For the discrete case, the probability of falling within any interval can be written in terms of the probability mass function, which allows the type of form above in more general cases. Again, you will notice that I have not interchanged the limits here, so I do not assert that the density function is a limit of a transformation of the mass functions. Unfortunately you can't interchange these limits here, or you will get a zero density at any irrational value $x$. Nevertheless, the above form shows that it is possible to write the density function in terms of the sequence of mass functions.

---

On this matter, it is also worth noting some philosophical issues relating to the use of limits, infinite quantities, and continuums in mathematics. Not only is it possible that convergence can occur from the discrete case to the continuous, but many mathematicians regard the limit of the discrete case as the best illustration of the meaning of the continuous case. Indeed, there are some mathematicians who only accept the existence of finite objects and they regard continuums and functions over continuums as ---at best--- an approximation to functions over large discrete spaces.