17
$\begingroup$

Do the pdf and the pmf and the cdf contain the same information?

For me the pdf gives the whole probability to a certain point(basically the area under the probability).

The pmf give the probability of a certain point.

The cdf give the probability under a certain point.

So to me the pdf and cdf have the same information, but the pmf does not because it gives the probability for a point x on the distribution.

|cite|improve this question
$\endgroup$
24
$\begingroup$

Where a distinction is made between probability function and density*, the pmf applies only to discrete random variables, while the pdf applies to continuous random variables.

* formal approaches can encompass both and use a single term for them

The cdf applies to any random variables, including ones that have neither a pdf nor pmf.

enter image description here

(A mixed distribution is not the only case of a distribution that doesn't have a pdf or pmf, but it's a reasonably common situation - for example, consider the amount of rain in a day, or the amount of money paid in claims on a property insurance policy, either of which might be modelled by a zero-inflated continuous distribution)

The cdf for a random variable $X$ gives $P(X\leq x)$

The pmf for a discrete random variable $X$, gives $P(X=x)$.

The pdf doesn't itself give probabilities, but relative probabilities; continuous distributions don't have point probabilities. To get probabilities from pdfs you need to integrate over some interval - or take a difference of two cdf values.

It's difficult to answer the question 'do they contain the same information' because it depends on what you mean. You can go from pdf to cdf (via integration), and from pmf to cdf (via summation), and from cdf to pdf (via differentiation) and from cdf to pmf (via differencing), so if a pmf or a pdf exists, it contains the same information as the cdf.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Glen, could you help by providing some reference where I could read about "pdf giving relative probabilities"? It's very interesting and I don't remember to have seen it in my books. Thanks. $\endgroup$ – Alecos Papadopoulos Aug 9 '14 at 22:01
  • $\begingroup$ @Alecos It's simply a (perhaps poorly worded) explanation of the fact that while $f(x)$ is not a probability, since $f(x)\,dx$ is the probability of being in $(x,x+dx)$, then $f(x)/g(x)$ can be thought of as the ratio of the probability that a variable with density $f$ is within a very small distance of $x$ to the ratio that a variable with density $g$ is in the same interval. In that sense it expresses 'relative probability'. $\endgroup$ – Glen_b♦ Aug 10 '14 at 0:05
  • $\begingroup$ I see. It is certainly valid as an approximation of the ratio of probabilities, and certainly present in empirical density functions, where things are discrete by necessity. $\endgroup$ – Alecos Papadopoulos Aug 10 '14 at 1:42
  • $\begingroup$ @user929304 I'm not sure what transition you're talking about there $\endgroup$ – Glen_b♦ Jun 25 '16 at 1:04
10
$\begingroup$

PMFs are associated with discrete random variables, PDFs with continuous random variables. For any type of random of random variable, the CDF always exists (and is unique), defined as $$F_X(x) = P\{X \leq x\}.$$ Now, depending on the support set of the random variable $X$, the density (or mass function) need not exist. (Consider the Cantor Set and Cantor Function, the set is recursively defined by removing the center 1/3 of the unit interval, then repeating the procedure for the intervals (0, 1/3) and (2/3, 1), etc. The function is defined as $C(x) = x$, if $x$ is in the Cantor set, and the greatest lower bound in the Cantor Set if $x$ is not a member.) The Cantor Function is a perfectly good distribution function, if you tack on $C(x)= 0$ if $x < 0$ and $C(x) = 1$ if $1 < x$. But this cdf has no density: $C(x)$ is continuous everywhere but its derivative is 0 almost everywhere. No density with respect to any useful measure.

So, the answer to your question is, if a density or mass function exists, then it is a derivative of the CDF with respect to some measure. In that sense, they carry the "the same" information. BUT, PDFs and PMFs don't have to exist. CDFs must exist.

|cite|improve this answer
$\endgroup$
  • 2
    $\begingroup$ Dennis, can you clarify what you mean by the phrase "No density with respect to any measure at all"? Certainly it has a density (uniform!) with respect to itself. $\endgroup$ – cardinal Jul 12 '14 at 1:07
  • $\begingroup$ @cardinal: I'll try, but I don't know that it will make sense unless you've studied some real analysis. If you look at some older books on mathematical statistics (e.g., Freund's Mathematical Statistics), you'll see PMFs referred to as "densities". The name "density" is justified by the probability measure $\mu$ on the measureable space $(\Omega, \sigma(\Omega), \mu)$ is the basis of the CDF (see Joel's comment). The density is the Radon-Nikodym derivative of $\mu$ with respect to some measure (usually Lesbesgue measure or Counting measure). In this case, $C(x)$ has no R-N derivative. $\endgroup$ – Dennis Jul 12 '14 at 1:56
  • 3
    $\begingroup$ @cardinal (continued): The probability measure is uniform on the Cantor Set, but this is such a strange beastie that I'm not even sure what the $\sigma$-algebra looks like. Perhaps I should have said, "No density with respect to any useful measure." $\endgroup$ – Dennis Jul 12 '14 at 2:03
2
$\begingroup$

The other answers point to the fact that CDFs are fundamental and must exist, whereas PDFs and PMFs are not and do not necessarily exist.

This confused and intrigued me (being a non-statistician), as I did not know how to interpret a CDF (or how it might exist) when the sample space was not ordered; think, for example, of the circle $S^1$.

It seems to me that the answer is that the fundamental function is the probability measure, which maps each (considered) subset of the sample space to a probability. Then, when they exist, the CDF, PDF and PMF arise from the probability measure.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ The way I've seen it, most text books define "random variable" to be a mapping from a sample space to the real numbers. Essentially, a "random variable" is real-valued. $\endgroup$ – Neil G Jul 11 '14 at 10:41
  • 1
    $\begingroup$ We use random variables to get into the probability space $(\mathbb{R}, \frak{B}, \mathrm{F})$ and away from $(\Omega, \sigma(\Omega), \mu)$. $\Omega$ may or may not be well-ordered, and that makes it a pain to deal with. I suppose you're right that $\mu$ is more fundamental: after all, $$F_X(x) = \mu\{\omega\, |\, X(\omega) \leq x\}.$$ But it's difficult to do much interesting with the abstract measure space. On top of that, my students have enough problems with PxFs and CDFs. I don't care to try to teach them measure theory. $\endgroup$ – Dennis Jul 11 '14 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.