# How to estimate a probability?

Let's assume that we have a random variable $X$ that takes integer values from the range $[0, b]$, the left boundary $b$ is not precisely defined, but on practice $b<10,000$.

x <- c(rep(7, 2), rep(9, 163), rep(11,231), rep(13,343), rep(15, 211),rep(17, 159),rep(19,  27))


I need to estimate a probability $Pr(X<a)$, where $a>0$ and $a << b$.

In order to estimate the probability, I propose to plot a probability densities, so that the histogram has a total area of one.

hist(x, freq = FALSE, label = TRUE, breaks = c(seq(7, 19, 1)))


Than I can find the sum of corresponded densities, for example, for the random variable $X$ generated above: $Pr(X<a=10)=0.002+0.143$.

Question: Is proposed approach correct?

• If your variable is not continues there is no densities, but you have probabilities. – Tim Sep 14 '16 at 16:01
• @Tim In a measure-theoretic treatment of probability, probability mass functions are just a special class of probability density functions that are integrated with respect to the counting measure, as opposed to the Lebesgue measure used for continuous random variables. – Kodiologist Sep 14 '16 at 16:41

## 2 Answers

Using a histogram is unnecessarily coarse. A better approach is to use the empirical CDF, by simply counting the number of values in your sample that are below $a$ and dividing this count by the sample size. In R, this is sum(x < a)/length(x) or mean(x < a).

• i have tried sum(x <= a)/length(x) and sum(x < a)/length(x). Results are same. What is correct '<=' or '<'? – Nick Sep 14 '16 at 16:06
• @Nick Depends on what you want. Using <= estimates $p(X ≤ a)$ whereas using < estimates $p(X < a)$. – Kodiologist Sep 14 '16 at 16:37

The easiest approach is to calculate the so-called empirical distribution function $\hat{F}$ of your sample. This function is calculated as

$$\hat{F}(s) = \frac{1}{n} \sum_{i=1}^{n} I(x_i \leq s)$$

where $I(x_i \leq s) = 1$ if $x_i \leq s$ and otherwise $I(x_i \leq s) = 0$, so for each value $s$ the function $\hat{F}(s)$ tells you the proportion of the sample that is less than or equal to $s$, and we use this to estimate $P(X \leq s)$

In R you could create your own function to do this:

edf <- function(sample, s) {
mean(sample <= s)
}


Now just insert your data into the sample argument and specify the value for s and you have an estimate of the probability you want.