Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Do the pdf and the pmf and the cdf contain the same information?

For me the pdf gives the whole probability to a certain point(basically the area under the probability).

The pmf give the probability of a certain point.

The cdf give the probability under a certain point.

So to me the pdf and cdf have the same information, but the pmf does not because it gives the probability for a point x on the distribution.

share|improve this question
3  
Your question is very confusingly worded. I don't think you're correctly characterizing the pdf there. –  Glen_b Jul 11 at 7:38
    
@Glen_b Will "refactor" my question soon. Btw I am looking for the general case to understand these 3 functions better... –  Kare Jul 11 at 7:43
1  
What do you mean "the general case"? The pmf and pdf never exist at the same time, and in the most general case neither the pmf nor the pdf exist. Nevertheless, the cdf always exists. –  Glen_b Jul 11 at 8:12
add comment

3 Answers 3

up vote 13 down vote accepted

The pmf applies only to discrete random variables.

The pdf applies only to continuous random variables.

The cdf applies to any random variables, including ones that have neither a pdf nor pmf.

enter image description here

(A mixed distribution is not the only case of a distribution that doesn't have a pdf or pmf, but it's a reasonably common situation - for example, consider the amount of rain in a day, or the amount of money paid in claims on a property insurance policy, either of which might be modelled by a zero-inflated continuous distribution)

The cdf for a random variable $X$ gives $P(X\leq x)$

The pmf for a discrete random variable $X$, gives $P(X=x)$.

The pdf doesn't itself give probabilities, but relative probabilities; continuous distributions don't have point probabilities. To get probabilities from pdfs you need to integrate over some interval - or take a difference of two cdf values.

It's difficult to answer the question 'do they contain the same information' because it depends on what you mean. You can go from pdf to cdf (via integration), and from pmf to cdf (via summation), and from cdf to pdf (via differentiation) and from cdf to pmf (via differencing), so if a pmf or a pdf exists, it contains the same information as the cdf.

share|improve this answer
add comment

PMFs are associated with discrete random variables, PDFs with continuous random variables. For any type of random of random variable, the CDF always exists (and is unique), defined as $$F_X(x) = P\{X \leq x\}.$$ Now, depending on the support set of the random variable $X$, the density (or mass function) need not exist. (Consider the Cantor Set and Cantor Function, the set is recursively defined by removing the center 1/3 of the unit interval, then repeating the procedure for the intervals (0, 1/3) and (2/3, 1), etc. The function is defined as $C(x) = x$, if $x$ is in the Cantor set, and the greatest lower bound in the Cantor Set if $x$ is not a member.) The Cantor Function is a perfectly good distribution function, if you tack on $C(x)= 0$ if $x < 0$ and $C(x) = 1$ if $1 < x$. But this cdf has no density: $C(x)$ is continuous everywhere but its derivative is 0 almost everywhere. No density with respect to any useful measure.

So, the answer to your question is, if a density or mass function exists, then it is a derivative of the CDF with respect to some measure. In that sense, they carry the "the same" information. BUT, PDFs and PMFs don't have to exist. CDFs must exist.

share|improve this answer
1  
Dennis, can you clarify what you mean by the phrase "No density with respect to any measure at all"? Certainly it has a density (uniform!) with respect to itself. –  cardinal Jul 12 at 1:07
    
@cardinal: I'll try, but I don't know that it will make sense unless you've studied some real analysis. If you look at some older books on mathematical statistics (e.g., Freund's Mathematical Statistics), you'll see PMFs referred to as "densities". The name "density" is justified by the probability measure $\mu$ on the measureable space $(\Omega, \sigma(\Omega), \mu)$ is the basis of the CDF (see Joel's comment). The density is the Radon-Nikodym derivative of $\mu$ with respect to some measure (usually Lesbesgue measure or Counting measure). In this case, $C(x)$ has no R-N derivative. –  Dennis Jul 12 at 1:56
2  
@cardinal (continued): The probability measure is uniform on the Cantor Set, but this is such a strange beastie that I'm not even sure what the $\sigma$-algebra looks like. Perhaps I should have said, "No density with respect to any useful measure." –  Dennis Jul 12 at 2:03
add comment

The other answers point to the fact that CDFs are fundamental and must exist, whereas PDFs and PMFs are not and do not necessarily exist.

This confused and intrigued me (being a non-statistician), as I did not know how to interpret a CDF (or how it might exist) when the sample space was not ordered; think, for example, of the circle $S^1$.

It seems to me that the answer is that the fundamental function is the probability measure, which maps each (considered) subset of the sample space to a probability. Then, when they exist, the CDF, PDF and PMF arise from the probability measure.

share|improve this answer
1  
The way I've seen it, most text books define "random variable" to be a mapping from a sample space to the real numbers. Essentially, a "random variable" is real-valued. –  Neil G Jul 11 at 10:41
    
We use random variables to get into the probability space $(\mathbb{R}, \frak{B}, \mathrm{F})$ and away from $(\Omega, \sigma(\Omega), \mu)$. $\Omega$ may or may not be well-ordered, and that makes it a pain to deal with. I suppose you're right that $\mu$ is more fundamental: after all, $$F_X(x) = \mu\{\omega\, |\, X(\omega) \leq x\}.$$ But it's difficult to do much interesting with the abstract measure space. On top of that, my students have enough problems with PxFs and CDFs. I don't care to try to teach them measure theory. –  Dennis Jul 11 at 16:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.