Are theoretical probability and relative frequency comparable?

Question

Speaking of probability measure (a number from 0 to 1), textbooks usually describe several interpretations of this number. E.g., Practical Business Statistics by Andrew F. Siegel reads (p. 132):

Probability numbers can come from previous experience (relative frequency as a percentage), from a mathematical statement (theoretical probability), or they can be simply made up (subjective probability, yes it is permitted to use someone’s opinion here).

For my question, the third interpretation is redundant, so I will consider the first two. The question itself is: Do these two interpretations correspond to any single thing at all?

I will try to explain my doubts in detail. Suppose we have two events: the first event’s probability was calculated theoretically (let the event be something like random card picking), and the second event’s probability was assessed on a frequency basis—like probability, say, of having a specific number of offspring in a litter, estimated by numerous observations of this species’ reproduction in the past. So, is there any evidence that these two probabilities belong to the same category? They are both measures of uncertainty of some kind, but what makes us believe that it is a single kind and not two different ones? E.g. the luminous intensity and the luminance both measure the light in one way or another, but they are two different measures, and you cannot directly compare what is bigger—5 candela or 10 lux.

So why are we sure that theoretical probability and frequency both refer to the same quantity? What are the reasons for comparing like “to pick the queen of spades is twice as likely as to have eight offspring in a litter”? For me, it sounds rather like “the height of Mount Fuji is twice the average speed of a snail”.

What am I missing, or what do I misunderstand?

Answer 1

For the comparison of experimental results with a theoretical probability distribution to make sense, both need to refer to the same thing measured the same way. Suppose you're talking about getting at least three Aces when fairly dealt five cards from a well-shuffled standard 52-card deck.

Theoretical: Let $X$ be the number of Aces obtained: $$P(X \ge 3) = P(X=3)+P(X=4) \\ = \left[{4\choose 3}{48 \choose 2} +{4\choose 4}{48\choose 1}\right]/{52 \choose 5}\\ \approx 0.00175,$$

which can be computed in R by evaluating the binomial coefficients or by using a hypergeometric PDF as follows:

dhyper(3, 4, 48, 5) + dhyper(4, 4, 48, 5)
[1] 0.001754548

Data-based. You could get data on many hands dealt at well-run poker competitions, to see how frequently this result occurs in practice. However, such a small probability is difficult to estimate empirically, so you would probably find out only that it is "pretty rare" to get 3 or more Aces is a 5-card hand. [I suggested getting data from professional sources because decks of cards are usually nowhere near sufficiently shuffled in most casual card games between friends. Truly randomizing a deck takes something like 7 shuffles.]

Alternatively, you might simulate many such draws using software such as R. In the simulation below, 1 represents an Ace; x is a vector of a million Ace counts, x >= 3 is a logical vector of a million TRUEs and FALSEs, and its mean is the proportion of its TRUEs. A million iterations should give about three-place accuracy. The simulated probability is $0.00185 \pm 0.00009,$ which is consistent with the theoretical computation.

set.seed(728)      # for reproducibility
deck = rep(1:13, 4)
x = replicate(10^6, sum(sample(deck,5)==1) )
mean(x >= 3)
[1] 0.001847
2*sd(x >= 3)/10^3
[1] 8.58741e-05    # 95% margin of sim error is +/- 0.00009