Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I know that the PDF is the first derivative of the CDF for a continuous random variable, and the difference for a discrete random variable. However, I would like to know why this is, why are there two different cases for discrete and continuous?

share|improve this question
add comment

4 Answers 4

up vote 6 down vote accepted

I am gonna be a bit imprecise, but hopefully intuitive.

Discrete and continuous probability distributions must be treated differently. For any value in a discrete distribution there is a finite probability. With a fair coin, the probability of heads is 0.5, with a fair six sided die, the probability of a 1 is one sixth, etc. However, the probability of any specific value in a continuous distribution is zero, because one specific value is only one value out of an infinite number of possible values, and if specific values had a >0 probability, then they would not sum up to 1. Hence, with continuous distributions we talk about the probability of ranges of values.

"Sum up to" is key in answering your question. If you are at all familiar with calculus and its history, you understand that the integral sign—that elongated 'S'—is a special kind of summation: one describing the limiting case as we approach summing an infinite number of vanishingly small values between points $a$ and $b$ on some function. If that function is a CDF, we can integrate it (sum up) to produce a PDF, and conversely differentiate (difference) the PDF to obtain the CDF.

In the discrete case, we can simply perform standard arithmetic summation (hence, big '$\Sigma$', rather than the tall 'S' notation) and arithmetic differencing.

share|improve this answer
add comment

The difference is for the convenience and understanding of people who have not had to endure Ph.D. level theory courses where you derive and prove "Integral with respect to Counting Measure". Which shows there really is no difference between discrete and continuous distributions, that a sum is really an integral (and as @Alexis already mentioned, an integral is essentially a sum) and a difference is really a derivative (it is a little simpler to see that a derivative is a difference scaled appropriately).

Textbooks and courses will treat them different because it is simpler to teach/understand early on rather than requiring the math that shows there is not a difference.

share|improve this answer
add comment

Actually, you can treat continuous and discrete distributions similarly, but in order to do this you have introduce Dirac's delta functions, left limits and other "advanced" concepts.

So, the easy way to answer your question is that discrete CDF jumps, it's discontinuous. You can't differentiate it everywhere because of that.

Again, if you know delta function, all is possible!

share|improve this answer
add comment

The term density refers only to continuous random variables.

Discrete random variables have a probability mass function, sometimes called a probability function (pmf or pf, not pdf). This doesn't return density but actual probability.

Some random variables don't have either (but they still have a cdf).

Think about what the definition of a cdf is ($F_X(x) = P(X\leq x)$), and then what happens as $x$ moves a little bit in both cases.

Now consider that any jumps in the cdf imply that a particular value has non-zero probability (that $P(X\leq x)>P(X<x)$ and the difference is $P(X=x)$). That non-zero probability for particular $x$-values is what the pmf records $p_X(x)=P(X=x)$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.