9
$\begingroup$

Probability density function

The probability density function of a uniform distribution (continuous) is shown above. The area under the curve is 1 - which makes sense since the sum of all the probabilities in a probability distribution is 1.

Formally, the above probability function (f(x)) can be defined as

1/(b-a) for x in [a,b]

and 0 otherwise

Consider that I have to choose a real number between a (say, 2) and b (say, 6). This makes the uniform probability = 0.25. However, since there are an infinite number of numbers in that interval, shouldn't the sum of all the probabilities sum up to infinity? What am I overlooking?

Is f(x) not the probability of the number x occurring?

$\endgroup$
18
$\begingroup$

$f(x)$ describes the probability density rather than a probability mass in your example. In general, for continuous distributions the events—the things we get probabilities for—are ranges of values, such as for the area under the curve from $a$ to $a+.1$, or from $a$ to $b$ (although such ranges need not be contiguous). For continuous distributions, the probability of any single value occurring is generally 0.

$\endgroup$
  • $\begingroup$ Is there a more technically accurate way to say what you're trying to say? I'm worried the "range" thing will throw people off, considering continuous distributions can have Dirac deltas... $\endgroup$ – Mehrdad Aug 28 '18 at 9:03
  • 3
    $\begingroup$ @Mehrdad: The dirac delta does not have a continuous distribution. The proper way of assigning probabilities would be via $P(A) = \int_A 1dF$. $\endgroup$ – Alex R. Aug 28 '18 at 10:39
  • 1
    $\begingroup$ @AlexR.: Oof, I assumed by "continuous distribution" you just meant a distribution over a continuous domain, since that's what people refer to when they say the Dirac delta is the continuous analog of the Kronecker delta. Thanks for clarifying. $\endgroup$ – Mehrdad Aug 28 '18 at 11:24
  • $\begingroup$ @Mehrdad I was thinking precisely of Dirac's delta, but I hope you will notice the term "in general", and also the apparent level of statistical literacy of the OP. $\endgroup$ – Alexis Aug 28 '18 at 13:57
  • $\begingroup$ @Mehrdad The technical formulation of a random variable is in terms of a measure: there is a function from the power set of the event space to the interval [0,1]. A probability density function can be used as a measure (the measure of a set is simply the integral of the PDF over that set), but there are measures, such as the Dirac delta (a set has measure 1 if it contains $x_0$, and is zero otherwise) that are, strictly speaking, not function in the traditional sense. $\endgroup$ – Acccumulation Aug 29 '18 at 14:51
11
$\begingroup$

Because each term in the summation is weighted by the infinitesimal d$x$. The importance of this is probably most easily understood by carefully walking through a very basic example.

Consider using Riemann summation to compute the area under the following rectangular region (a rectangle was chosen to remove the approximation aspect of Riemann summation, which is not the focus here): rectangular region] We can compute the area using 2 subregions, or by using 4 subregions. In the case of the 2 subregions (denoted $A_i$), the areas are given by $$A_1=A_2=5\times 2 = 10$$ whereas in the case of 4 subregions (denoted $B_i$), the areas are given by $$B_1=B_2=B_3=B_4=5\times 1 = 5$$ The total area in both cases correspond to $$\sum_{i=1}^2 A_i = \sum_{i=1}^4B_i = 20$$ Now, this is all fairly obvious, but it raises a subtly important question which is: why do these two answers agree? Intuitively it should be clear that it works because we've reduced the width of the second set of subregions. We could consider doing the same thing with 8 subregions each with a width of $0.5$, and again with 16... and we could continue this process until we have an infinite number of subregions, each with a tiny width of d$x$. As long as everything is always correctly weighted, the answers should always agree. Without the correct weighting, the summation would indeed simply be $\infty$.

This is why I always make sure to point out to students that an integral is not simply the symbol $\int$, but the pair of symbols $\int \text dx$.

$\endgroup$
5
$\begingroup$

You're interpreting the probability distribution the wrong way - it is an infinite number of infinitely divided probabilities, so you can't say that "the probability of drawing the value 0.5 from a (0, 1) uniform distribution" because that probability is zero - there are an infinite number of possible values you could get, and all of them are equally likely, so clearly the probability of any individual outcome is $\frac{1}{\infty} = 0$[1].

Instead, you can look at the probability for a range of outcomes, and measure that using areas (and hence integrals). For example, if you draw from the (0, 1) uniform distribution (with pdf $f(x) = 1$ for $x \in \left[0, 1\right]$ and $f(x) = 0$ otherwise), then the probability that your result lies between $0.2$ and $0.3$ is

$\int_{0.2}^{0.3} f(x)\ dx = \int_{0.2}^{0.3} 1\ dx = \left[x \right]_{0.2}^{0.3} = 0.3 - 0.2 = 0.1$

i.e. you have a 10% chance of getting a result in that range.

[1]Sorry for all the people having heart attacks at my over-simplification of the calculation.

$\endgroup$
0
$\begingroup$

In general your reasoning fails in this assumption:

However, since there are an infinite number of numbers in that interval, shouldn't the sum of all the probabilities sum up to infinity?

It's a mathematical problem, known since the Zeno of Elea Paradoxes.

Two of his claims were that

  1. An arrow can never reach its target
  2. Achilles will never overtake a turtle

Both of them were based on the claim that you can build an infinite sequence of positive numbers (in the former case by saying that an arrow has to fly infinitely times half of the remaining way to the target, in the latter by saying that Achilles has to reach position where the turtle was previously, and in the meantime the turtle moves to a new position that becomes our next reference base point).

Fast forward, this led to a discovery of infinite sums.

So in general sum of infinite many positive numbers does not necessarily have to be infinite; however, it may not be infinite only if (an extreme oversimplification, sorry about that) almost all of the numbers in the sequence are very close to 0, regardless how close to zero you request them to be.

Infinity plays even more tricks. The order in which you add elements of the sequence is important too and might lead to a situation that reordering gives different results!

Explore a bit more about paradoxes of infinity. You might be astonished.

$\endgroup$
  • $\begingroup$ I don't see a way to interpret the question such that OP thinks of countable sums. $\endgroup$ – JiK Aug 28 '18 at 15:13
0
$\begingroup$

$f(x)$ describes the probability density, and has the unit $\frac{p}{x}$. Hence for a given x you get $f(x) = \frac{1}{b-a}$ in $\frac{p}{x}$ units, and not p, as you are looking for. If you want p, you need the distribution function for a given range, that is the probability p of x being within a and b.

Hope this makes sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.