Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In Wikidata it is possible to link probability distributions (like everything else) in an ontology, e.g., that the t-distribution is a subclass of the noncentral t-distribution, see, e.g.,

https://angryloki.github.io/wikidata-graph-builder/?property=P279&item=Q209675&iterations=3&limit=3

There are various limiting cases, e.g., when the degrees of freedom in the t-distribution goes to infinity or when the variance approaches zero for the normal distribution (Gaussian distribution). In the latter case the distribution will go towards Dirac's delta function.

I note that on the English Wikipedia the variance parameter is presently stated as larger than zero, so with a strict interpretation one would not say that the Dirac's delta function is a subclass of the normal distribution. However, to me it seems quite ok, as I would say that the exponential distribution is a superclass of the Dirac's delta function.

Are there any problems with stating that Dirac's delta function is a subclass of the Gaussian distribution?

share|improve this question

The delta functions fit into a mathematical theory of distributions (which is quite distinct from the theory of probability distributions, terminology here could not be more confusing).

Essentially, distributions are generalized functions. They cannot be evaluated like a function can, but then can be integrated. More precisely, a distribution $D$ is defined as follows

Let $T$ be the collection of test functions. A test function $\theta$ is a true, honest to god function, smooth, with compact support. A distribution is a linear mapping $D: T \rightarrow \mathbb{R}$

An honest function $f$ determines a distribution by the integration operator

$$ T(\theta) = \int f(x)\theta(x) dx $$

There are distributions that are not associated to true functions, the dirac operator is one of them

$$ \delta(\theta) = \theta(0) $$

In this sense, you can consider the dirac a limiting case of the normal distributions. If $N_t$ is the family of pdf's of normal distributions with mean zero and variance $t$, then for any test function $\theta$

$$ \theta(0) = \lim_{t \rightarrow 0} \int N_t(x) \theta(x) dx $$

This is probably more commonly expressed as

$$ \theta(0) = \int \delta(x) \theta(x) dx = \lim_{t \rightarrow 0} \int N_t(x) \theta(x) dx $$

which a mathematician would consider an abuse of notation, because the expression $\delta(x)$ does not actually make any sense. But then again, who am I to criticize Dirac, who is the best.

Of course, whether this makes the dirac a member of the family of normal distributions is a cultural question. Here I'm just giving a reason why it may make sense to consider it so.

share|improve this answer
    
Whilst I agree with your statements, I think this implies the opposite. A delta function is not a subset of gaussians. Just as a limit of continuous functions need not be a continuous function. – seanv507 Feb 16 at 22:38
    
@seanv507 I did my best to not state a conclusion either way! – Matthew Drury Feb 17 at 1:24
    
I thought distributions are very much like probability distributions, with a Dirac delta (probability) distribution indicating a deterministic variable... – Mehrdad Feb 17 at 6:37

Dirac's delta is regarded as a Gaussian distribution when it is convenient to do so, and not so regarded when this viewpoint requires us to make exceptions.

For example, $(X_1, X_2, \ldots, X_n)$ are said to enjoy a multivariate Gaussian distribution if $\sum_i a_iX_i$ is a Gaussian random variable for all choices of real numbers $a_1, a_2, \ldots, a_n$. (Note: this is a standard definition in "advanced" statistics). Since one choice is $a_1=a_2=\cdots=a_n=0$, the standard definition treats the constant $0$ (a degenerate random variable) as a Gaussian random variable (with mean and variance $0$). On the other hand, we ignore our regard for the Dirac delta as a Gaussian distribution when we are considering something like

"The cumulative probability distribution function (CDF) of a zero-mean Gaussian random variable with standard deviation $\sigma$ is $$F_X(x) = P\{X \leq x\} = \Phi\left(\frac{x}{\sigma}\right)$$ where $\Phi(\cdot)$ is the CDF of a standard Gaussian random variable."

Note that this statement is almost right but not quite right if we regard the Dirac delta as the limiting case of a sequence of zero-mean Gaussian random variables whose standard deviation approaches $0$ (and hence as a Gaussian random variable). The CDF of the Dirac delta has value $1$ for $x \geq 0$ whereas $$\lim_{\sigma\to 0}\Phi\left(\frac{x}{\sigma}\right) = \begin{cases} 0, & x < 0,\\ \frac 12, & x = 0,\\ 1, & x > 0.\end{cases}$$ But, lots of people will tell you that regarding a Dirac delta as a Gaussian distribution is sheer nonsense since their book says that the variance of a Gaussian random variable must be a positive number (and some of them will down-vote this answer to show their displeasure). There was a very vigorous and illuminating discussion of this point a few years ago on stats.SE but unfortunately it was only in the comments on an answer (by @Macro, I believe) and not as individual answers, and I cannot find it again.

share|improve this answer
2  
+1. I'm not sure there's a problem concerning the CDF, because I believe the limiting value of a sequence of CDFs at any jump of the limit doesn't matter. There are two ways to see that. One is to note that your limiting formula is not a valid CDF (it is not cadlag). Another is to note that you obtain a Dirac distribution at $0$ when you let $(\mu,\sigma)\to(0,0)$ simultaneously, but you can contrive to have the limiting value of $\Phi_{\mu,\sigma}(0)$ be anything between $0$ and $1$ (or not to have a limit at all). – whuber Feb 16 at 20:30
5  
The conversation you reference happened in the comments of this answer, though I sincerely hope that to most readers the discussion will not appear too vigorous. (+1) – cardinal Feb 17 at 0:50
1  
@cardinal Deep knowledge of our community. Well done! – Matthew Drury Feb 17 at 1:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.