# Should Dirac's delta function be regarded as a subclass of the Gaussian distribution?

In Wikidata it is possible to link probability distributions (like everything else) in an ontology, e.g., that the t-distribution is a subclass of the noncentral t-distribution, see, e.g.,

https://angryloki.github.io/wikidata-graph-builder/?property=P279&item=Q209675&iterations=3&limit=3

There are various limiting cases, e.g., when the degrees of freedom in the t-distribution goes to infinity or when the variance approaches zero for the normal distribution (Gaussian distribution). In the latter case the distribution will go towards Dirac's delta function.

I note that on the English Wikipedia the variance parameter is presently stated as larger than zero, so with a strict interpretation one would not say that the Dirac's delta function is a subclass of the normal distribution. However, to me it seems quite ok, as I would say that the exponential distribution is a superclass of the Dirac's delta function.

Are there any problems with stating that Dirac's delta function is a subclass of the Gaussian distribution?

• IF dirac delta is a subclass of gaussian then its kurtosis must be 3, right? – Aksakal Sep 7 '16 at 22:57
• I guess that if we regard the Dirac delta as a subclass of several probability distributions, then the kurtosis is inconsistent for the Dirac Delta. It speaks against regarding the Dirac delta as a subclass of any of these distributions. – Finn Årup Nielsen Sep 8 '16 at 8:46
• In probability context delta is described as a generalized function. It's not am ordinary function – Aksakal Sep 8 '16 at 11:16

Dirac's delta is regarded as a Gaussian distribution when it is convenient to do so, and not so regarded when this viewpoint requires us to make exceptions.

For example, $(X_1, X_2, \ldots, X_n)$ are said to enjoy a multivariate Gaussian distribution if $\sum_i a_iX_i$ is a Gaussian random variable for all choices of real numbers $a_1, a_2, \ldots, a_n$. (Note: this is a standard definition in "advanced" statistics). Since one choice is $a_1=a_2=\cdots=a_n=0$, the standard definition treats the constant $0$ (a degenerate random variable) as a Gaussian random variable (with mean and variance $0$). On the other hand, we ignore our regard for the Dirac delta as a Gaussian distribution when we are considering something like

"The cumulative probability distribution function (CDF) of a zero-mean Gaussian random variable with standard deviation $\sigma$ is $$F_X(x) = P\{X \leq x\} = \Phi\left(\frac{x}{\sigma}\right)$$ where $\Phi(\cdot)$ is the CDF of a standard Gaussian random variable."

Note that this statement is almost right but not quite right if we regard the Dirac delta as the limiting case of a sequence of zero-mean Gaussian random variables whose standard deviation approaches $0$ (and hence as a Gaussian random variable). The CDF of the Dirac delta has value $1$ for $x \geq 0$ whereas $$\lim_{\sigma\to 0}\Phi\left(\frac{x}{\sigma}\right) = \begin{cases} 0, & x < 0,\\ \frac 12, & x = 0,\\ 1, & x > 0.\end{cases}$$ But, lots of people will tell you that regarding a Dirac delta as a Gaussian distribution is sheer nonsense since their book says that the variance of a Gaussian random variable must be a positive number (and some of them will down-vote this answer to show their displeasure). There was a very vigorous and illuminating discussion of this point a few years ago on stats.SE but unfortunately it was only in the comments on an answer (by @Macro, I believe) and not as individual answers, and I cannot find it again.

• +1. I'm not sure there's a problem concerning the CDF, because I believe the limiting value of a sequence of CDFs at any jump of the limit doesn't matter. There are two ways to see that. One is to note that your limiting formula is not a valid CDF (it is not cadlag). Another is to note that you obtain a Dirac distribution at $0$ when you let $(\mu,\sigma)\to(0,0)$ simultaneously, but you can contrive to have the limiting value of $\Phi_{\mu,\sigma}(0)$ be anything between $0$ and $1$ (or not to have a limit at all). – whuber Feb 16 '16 at 20:30
• The conversation you reference happened in the comments of this answer, though I sincerely hope that to most readers the discussion will not appear too vigorous. (+1) – cardinal Feb 17 '16 at 0:50
• @cardinal Deep knowledge of our community. Well done! – Matthew Drury Feb 17 '16 at 1:21

The delta functions fit into a mathematical theory of distributions (which is quite distinct from the theory of probability distributions, terminology here could not be more confusing).

Essentially, distributions are generalized functions. They cannot be evaluated like a function can, but then can be integrated. More precisely, a distribution $D$ is defined as follows

Let $T$ be the collection of test functions. A test function $\theta$ is a true, honest to god function, smooth, with compact support. A distribution is a linear mapping $D: T \rightarrow \mathbb{R}$

An honest function $f$ determines a distribution by the integration operator

$$T(\theta) = \int_{-\infty}^{+\infty} f(x)\theta(x) dx$$

There are distributions that are not associated to true functions, the dirac operator is one of them

$$\delta(\theta) = \theta(0)$$

In this sense, you can consider the dirac a limiting case of the normal distributions. If $N_t$ is the family of pdf's of normal distributions with mean zero and variance $t$, then for any test function $\theta$

$$\theta(0) = \lim_{t \rightarrow 0} \int_{-\infty}^{+\infty} N_t(x) \theta(x) dx$$

This is probably more commonly expressed as

$$\theta(0) = \int_{-\infty}^{+\infty} \delta(x) \theta(x) dx = \lim_{t \rightarrow 0} \int_{-\infty}^{+\infty} N_t(x) \theta(x) dx$$

which a mathematician would consider an abuse of notation, because the expression $\delta(x)$ does not actually make any sense. But then again, who am I to criticize Dirac, who is the best.

Of course, whether this makes the dirac a member of the family of normal distributions is a cultural question. Here I'm just giving a reason why it may make sense to consider it so.

• Whilst I agree with your statements, I think this implies the opposite. A delta function is not a subset of gaussians. Just as a limit of continuous functions need not be a continuous function. – seanv507 Feb 16 '16 at 22:38
• @seanv507 I did my best to not state a conclusion either way! – Matthew Drury Feb 17 '16 at 1:24
• I thought distributions are very much like probability distributions, with a Dirac delta (probability) distribution indicating a deterministic variable... – Mehrdad Feb 17 '16 at 6:37
• If you don't write the limits of the integrals they might be confused indefinite integrals. Also, this sentence doesn't make sense: "A test function θ is a true, honest to god function, smooth, with compact support". – jkabrg Sep 7 '16 at 20:09
• @jkabrg Why doesn't it make sense? Since I wrote it, it's hard for me to see it not making sense. – Matthew Drury Sep 7 '16 at 20:25

No. It's not a subclass of normal distribution.

I think the confusion comes from one of the representations of Dirac function. Remember that it's defined as follows:

$$\int_{-\infty}^\infty\delta(x)dx=1$$ $$\delta(x)=0,\forall x\ne 0$$

It's defined as an integral, which is great but sometime you need to operationalize it by a function representation rather than an integral. So, people came up with all kinds of alternatives, one of them looks like Gaussian density: $$\delta(x)=\lim_{\sigma\to 0} \frac{e^{\frac{-x^2}{2\sigma^2}}}{\sqrt{2\pi}\sigma}$$

However, this is not the only representation, e.g. there's this one: $$\delta(x)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty e^{ikx}, \forall x\in (-\pi,\pi)$$

Hence, it's best to think of Dirac function in terms of its integral definition, and take the function representations, such as Gaussian, as tools of convenience.

UPDATE To @whuber's point, a better even example is this representation of Dirac's delta:

$$\delta(x)=\lim_{\sigma\to 0} \frac{e^{-\frac{|x|}{\sigma}}}{2\sigma}$$

Does this look like Laplacian distribution to you? Shouldn't we consider then Dirac's delta as a subclass of Laplacian distribution?

• At some point in this answer you seem to switch from discussing distributions to discussing "functions." The question refers explicitly to "probability distributions." Those are not generally given by density functions, but can always be given by their distribution function. The distribution of an atom--the "Dirac delta"--fits beautifully in with all the other Gaussian distributions as a limiting case. (In Matthew Drury's setup, it is defined as that limit!) Your argument seems similar to claiming that, say, circles are not ellipses. Enforcing such exceptions doesn't seem constructive. – whuber Sep 7 '16 at 18:19
• @whuber, what's "distribution of an atom"? – Aksakal Sep 7 '16 at 18:35
• An "atom" is a lump of probability at a single point. Equivalently, it is the distribution of any random variable that is constant almost everywhere. – whuber Sep 7 '16 at 18:36
• @whuber, Oh, I was thinking of a physical atom. No, my point is that Dirac's delta is not a subclass of Gaussian, because it can be represented also by Laplacian like distros – Aksakal Sep 7 '16 at 18:45
• Re: your point about Laplace distributions. Just as a square is both a rectangle and a rhombus, and the Uniform$(0,1)$ distribution is both a special case of a Uniform$(0,\theta)$ distribution and a Beta$(\alpha,\beta)$ distribution, a distribution can belong to multiple named families of distributions. The delta distributions in fact belong to every location-scale family and at least one delta distribution belongs to every scale family. Geometrically, families are curves in a space of distributions; a given distribution is a point; and (obviously) any point may belong to many curves. – whuber Sep 7 '16 at 19:21