8
$\begingroup$

I know that the PDF is the first derivative of the CDF for a continuous random variable, and the difference for a discrete random variable. However, I would like to know why this is, why are there two different cases for discrete and continuous?

$\endgroup$

4 Answers 4

8
$\begingroup$

I am gonna be a bit imprecise, but hopefully intuitive.

Discrete and continuous probability distributions must be treated differently. For any value in a discrete distribution there is a finite probability. With a fair coin, the probability of heads is 0.5, with a fair six sided die, the probability of a 1 is one sixth, etc. However, the probability of any specific value in a continuous distribution is zero, because one specific value is only one value out of an infinite number of possible values, and if specific values had a >0 probability, then they would not sum up to 1. Hence, with continuous distributions we talk about the probability of ranges of values.

"Sum up to" is key in answering your question. If you are at all familiar with calculus and its history, you understand that the integral sign—that elongated 'S': $\int$—is a special kind of summation: one describing the limiting case as we approach summing an infinite number of vanishingly small values between points $a$ and $b$ on some function. If that function is a PDF, we can integrate it (sum up) to produce a CDF, and conversely differentiate (difference) the CDF to obtain the PDF.

In the discrete case, we can simply perform standard arithmetic summation (hence, big '$\Sigma$', rather than the tall 'S' notation) and arithmetic differencing.

$\endgroup$
2
  • $\begingroup$ "If that function is a CDF, we can integrate it (sum up) to produce a PDF" You have got the ordering wrong, this is confusing. I edited to correct. $\endgroup$
    – Zhubarb
    Sep 11, 2014 at 7:36
  • $\begingroup$ @Zhubarb Thanks for the correct! I must have been undercaffeinated. ;) $\endgroup$
    – Alexis
    Sep 11, 2014 at 16:03
6
$\begingroup$

The difference is for the convenience and understanding of people who have not had to endure Ph.D. level theory courses where you derive and prove "Integral with respect to Counting Measure". Which shows there really is no difference between discrete and continuous distributions, that a sum is really an integral (and as @Alexis already mentioned, an integral is essentially a sum) and a difference is really a derivative (it is a little simpler to see that a derivative is a difference scaled appropriately).

Textbooks and courses will treat them different because it is simpler to teach/understand early on rather than requiring the math that shows there is not a difference.

$\endgroup$
2
$\begingroup$

(At least at introductory levels) the term density refers only to continuous random variables.

Discrete random variables have a probability mass function, sometimes called a probability function (pmf or pf, not pdf). This doesn't return density but actual probability.

Some random variables don't have either (but they still have a cdf).

Think about what the definition of a cdf is ($F_X(x) = P(X\leq x)$), and then what happens as $x$ moves a little bit in both cases.

Now consider that any jumps in the cdf imply that a particular value has non-zero probability (that $P(X\leq x)>P(X<x)$ and the difference is $P(X=x)$). That non-zero probability for particular $x$-values is what the pmf records $p_X(x)=P(X=x)$.

(In more advanced treatments, the distinction disappears.)

$\endgroup$
1
$\begingroup$

Actually, you can treat continuous and discrete distributions similarly, but in order to do this you have introduce Dirac's delta functions, left limits and other "advanced" concepts.

So, the easy way to answer your question is that discrete CDF jumps, it's discontinuous. You can't differentiate it everywhere because of that.

Again, if you know delta function, all is possible!

$\endgroup$
5
  • $\begingroup$ Delta "functions" are unnecessary to put discrete and continuous random variables on the same footing. See e.g., the development in Billingsley's "Probability and Measure". $\endgroup$
    – Zen
    Sep 5, 2014 at 17:00
  • $\begingroup$ And what do you think is pdf of discrete rv in this text? $\endgroup$
    – Aksakal
    Sep 5, 2014 at 17:02
  • $\begingroup$ The Radon-Nikodym derivative with respect to counting measure. Sorry to nitpick, but you said that we "have to introduce" Dirac's delta. We don't, and there is a zillion students accessing this forum. Check Billingsley. Great mathematician, wonderful book. $\endgroup$
    – Zen
    Sep 5, 2014 at 17:07
  • $\begingroup$ And what is rn derivative for Bernoulli distribution? $\endgroup$
    – Aksakal
    Sep 5, 2014 at 17:17
  • $\begingroup$ It's not a Dirac's delta! $\endgroup$
    – Zen
    Sep 5, 2014 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.