8
$\begingroup$

Like the title says, are the terms probability density function and probability distribution (or just "distribution") interchangeable? If not, what is the difference?

$\endgroup$
  • 2
    $\begingroup$ Of the two, I actually think this is the better-posed question in many ways. But as the latter of the two it is probably the one that should be closed. $\endgroup$ – Silverfish Sep 28 '15 at 13:51
  • $\begingroup$ @Silverfish Not only is this question better-posed than the other one, it is, in my opinion, asking something different. Indeed, the (sole and accepted) answer to the other question does not answer this question at all except perhaps in the very last sentence in it. I vote to reopen it; perhaps you can join me in this. I will confess that I do have an ulterior motive. Questions closed as duplicates are rarely viewed by most people, and I do not want to have wasted my time in writing an answer here. Besides, it is a shame to deprive people of the pleasure of downvoting my polemical answer. $\endgroup$ – Dilip Sarwate Sep 28 '15 at 18:14
  • $\begingroup$ @Dilip If the threads were truly duplicates, we would merge them, resulting in your contribution becoming part of the original thread. In this case, though, I agree with your contention that the question differs sufficiently to warrant re-opening this thread. $\endgroup$ – whuber Sep 28 '15 at 19:34
  • $\begingroup$ @Dilip If this were to have remained closed, one approach to increase visibility of related but not identical answers is to link back here via a comment in the question it would be closed as a duplicate of. $\endgroup$ – Glen_b Sep 29 '15 at 1:00
  • $\begingroup$ Can somebody please post here a link to the formerly-proposed dup? $\endgroup$ – kjetil b halvorsen Dec 5 '18 at 9:56
5
$\begingroup$

The phrase probability density function (pdf) means a specific thing: a function $f_X(\cdot)$ for a specific random variable $X$ (that's what that subscript there is for, to distinguish this function from the pdfs of other random variables) with the property that for all real numbers $a$ and $b$ such that $a < b$, $$P\{a < X \leq b\} = \int_a^b f_X(u)\,\mathrm du = \int_a^b f_X(v)\,\mathrm dv = \int_a^b f_X(t)\,\mathrm dt.$$ The different integrals are intended to serve as a reminder that it does not matter in the least what symbol we use as the argument of $f_X(\cdot)$ and that it is not the case (as is regrettably far too often believed by those starting on this subject) that the argument must be the lower-case letter corresponding to the upper-case letter that denotes the random variable. We also insist that $$\int_{-\infty}^\infty f_X(u)\,\mathrm du = 1.$$ If $P\{X = \alpha\} > 0$ for some real number $\alpha$, then $X$ does not have a pdf except for those who incorporate Dirac deltas into their probability calculus.

The cumulative probability distribution function (cdf or CDF) $F_X(\cdot)$ of $X$ is the function defined as $$F_X(\alpha) = P\{X \leq \alpha\}, -\infty < \alpha < \infty.$$ It is related to the pdf (for functions that do have a pdf) through $$F_X(\alpha) = \int_{-\infty}^\alpha f_X(u)\,\mathrm du.$$

=======

While there might be a very restrictive definition of the phrase probability distribution that some people insist on, the colloquial use of the term broadly encompasses the pdf and the CDF and the pmf (probability mass function which is also called the ddf or discrete density function) and whatever else we might want to include as descriptive of the probabilistic behavior of a random variable. For example, the phrase

the probability distribution of $X$ is uniform on $(a,b)$

will hardly ever be interpreted as meaning that the CDF of $X$ has constant value on $(a,b)~$!! Although it is the distribution which is alleged to be uniform, everyone in his/her right mind will take that as meaning that the density of $X$ has constant value $(b-a)^{-1}$ on the interval $(a,b)$ (and has value $0$ elsewhere). Similarly, for "$X$ is uniformly distributed on $(a,b)$" when what is meant is that the pdf of $X$ has constant value on $(a,b)$.

As another instance of colloquial usage of distribution to mean density, consider this quote from a recently posted answer by Moderator Glen_b.

"Saying the mode implies that the distribution has one and only one."

A density might possess a unique mode but a CDF cannot have a unique mode (in the unextended reals). However, no one reading that quote is likely to think that Glen_b meant the CDF when he wrote "distribution".

$\endgroup$
  • $\begingroup$ +1, but I have some reservations about the emphases of the various points in this answer. The final point that a PDF is not uniquely defined--it's actually an $L^1$ equivalence class of functions--supports a different contention that "probability distribution" does not even colloquially refer to the PDF (except as an abuse of terminology). Clearly the CDF, which (subject to the cadlag restriction) is uniquely defined for all variables, is a far better candidate to be the referent of "the distribution" of any random variable. $\endgroup$ – whuber Sep 28 '15 at 19:40
  • 1
    $\begingroup$ @whuber Thanks for re-opening the question and the upvote. I have deleted the normal random variable stuff and replaced it with a better illustration of why "distribution" does not always mean CDF but can be a stand-in for density or pdf. $\endgroup$ – Dilip Sarwate Sep 28 '15 at 20:05
  • 1
    $\begingroup$ In the edit, you make a very good case on behalf of your interpretation of colloquial usage. $\endgroup$ – whuber Sep 28 '15 at 20:18
2
$\begingroup$

In terms of common usage, consider parsing the terminology used in R. The Description on the Distributions {stats} help page says:

Density, cumulative distribution function, quantile function and random variate generation for many standard probability distributions are available in the stats package.

For each of the built-in Distributions, it provides (according to the individual help pages) the "density" (e.g. dnorm for Normal, dbinom for Binomial) and the "distribution function" (e.g., pnorm, pbinom; called the "cumulative distribution function" on the main Distributions page, as quoted above).

So one might interpret that "probability distribution" describes (perhaps a member of) a family of distributions, "density" can be used for discrete distributions like the binomial, and the phrase "distribution function" might be preferred over "distribution" when the cumulative distribution function is what is intended.

Alternatively, one might argue that common usage even among the experienced often depends on context for clarity.

$\endgroup$
0
$\begingroup$

No.

  1. "probability density function" is used only for continuous distributions. A discrete distribution can't have a pdf (though it can be approximated with a pdf). "probability distribution" is often used for discrete distributions, e.g., the binomial distribution.

  2. "probability distribution" has a meaning for both discrete and continuous distributions, but a probability distribution is directly applicable only for discrete distributions. When the word is used with continuous distributions, it refers to an underlying mathematical construct such as the normal distribution, which must for most purposes be instantiated in a function, typically a probability density function or a cumulative density function, before it can be applied.

$\endgroup$
  • 1
    $\begingroup$ "'probability distribution' is usually used for, and only well-defined for, discrete distribution" Do you mean that, e.g., a normal distribution is not a probability distribution? $\endgroup$ – Juho Kokkala Dec 5 '18 at 7:58
  • $\begingroup$ @Kokkala: If I'd meant that, I would have simply said "'probability distribution' should only be used for discrete distributions." Those extra words were to allow for cases where people call e.g. a normal distribution a probability distribution. But you raise a good point: in continuous domains, "probability distribution" is still used, but in order to apply a probability distribution, we have to use some more-specific instantiation of it, such as a pdf or cdf. So I'll amend my answer if I can. $\endgroup$ – Phil Goetz Dec 5 '18 at 16:05
  • 1
    $\begingroup$ There is a common formal definition of "probability distribution" that conflicts with your point (2); namely, the distribution of any real-valued random variable $X$ is given by the function $x\to \Pr(X\le x).$ This is clearly "directly applicable" to all distributions, not just discrete ones. $\endgroup$ – whuber Dec 5 '18 at 16:37
  • 1
    $\begingroup$ Fashions change on this. The expression probability mass function was introduced to make the discrete case utterly distinct, but conversely many writers explain that density can be density defined in terms of counting measure. It's all a question of the underlying measure. So, no; competent writers can write of density functions and have a broad definition in mind, not just applicability to continuous variables. Peter Whittle's intermediate probability text is a case in point. $\endgroup$ – Nick Cox Dec 5 '18 at 16:59

protected by kjetil b halvorsen Dec 5 '18 at 10:00

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.