The short answer:
Basically it's more convincing to have 600 out of 1000 than six out of 10 because, given equal preferences it's far more likely for 6 out of 10 to occur by random chance.
Let's make an assumption - that the proportion who preferred oranges and apples are actually equal (so, 50% each). Call this a null hypothesis. Given these equal probabilities the likelihood of the two results are:
- Given a sample of 10 people, there is a 38% chance of randomly getting a sample of 6 or more people who prefer oranges (which is not all that unlikely).
- With a sample of 1000 people there is less than 1 in a billion chance of having 600 or more out of 1000 people prefer oranges.
(For simplicity I am assuming an infinite population from which to draw an unlimited number of samples).
A simple derivation
One way to derive this result is to simply list out the potential ways in which people can combine in our samples:
For ten people it's easy:
Consider drawing samples of 10 people at random from an infinite population of people with equal preferences for apples or oranges. With equal preferences it's easy to simply list all the potential combinations of 10 people:
Here's the full list.
r C (n=10) p
10 1 0.09766%
9 10 0.97656%
8 45 4.39453%
7 120 11.71875%
6 210 20.50781%
5 252 24.60938%
4 210 20.50781%
3 120 11.71875%
2 45 4.39453%
1 10 0.97656%
0 1 0.09766%
r is the number of results (people who prefer oranges), C is the number of possible ways of that many people preferring oranges, and p is the resulting discrete probability of that many people preferring oranges in our sample.
(p is just C divided by the total number of combinations. Note that there are 1024 ways of arranging these two preferences in total (i.e. 2 to the power of 10).
- For instance there is only one way (one sample) for 10 people (r=10) to all prefer oranges. The same is true for all people preferring apples (r=0).
- There are 10 different combinations resulting in nine of them preferring oranges. (One different person prefers apples in each sample).
- There are 45 samples (combinations) where 2 people prefer apples, etc, etc.
(In general we talk about n C r combinations of results r from a sample of n people. There are online calculators you can use to verify
This list allows us to gives us the probabilities above using just division. There is a 21% chance of getting 6 people in the sample that prefer oranges (210 of 1024 of the combinations). The chance of getting six or more people in our sample is 38% (the sum of all the samples with six or more people, or 386 out of 1024 combinations).
Graphically, the probabilities look like this:
With larger numbers, the number of potential combinations grows rapidly.
For a samples of just 20 people there are 1,048,576 possible
samples, all with equal likelihood. (Note: I have only shown every second combination below).
r C (n=20) p
20 1 0.00010%
18 190 0.01812%
16 4,845 0.46206%
14 38,760 3.69644%
12 125,970 12.01344%
10 184,756 17.61971%
8 125,970 12.01344%
6 38,760 3.69644%
4 4,845 0.46206%
2 190 0.01812%
0 1 0.00010%
There is still only one sample where all 20 people prefer oranges. Combinations that feature mixed results are much more likely, simply because there are many more ways that the people in the samples can be combined.
Samples that are biased are much more unlikely, just because there are fewer combinations of people that can result in those samples:
With just 20 people in each sample, the cumulative probability of having 60% or more (12 or more) people in our sample preferring oranges
drops to just 25%.
The probability distribution can be seen to become thinner and taller:
With 1000 people the numbers are huge
We can extend the above examples to larger samples (but the numbers grow too rapidly for it to be feasible to list all the combinations),
instead I have calculated the probabilities in R:
r p (n=1000)
The cumulative probability of having 600 or more out of 1000 people prefer oranges is just 1.364232e-10.
The probability distribution is now much more concentrated around the center:
(For example to calculate probability of exactly 600 out of 1000 people preferring oranges in R use
dbinom(600, 1000, prob=0.5) which equals 4.633908e-11, and the probability of 600 or more people is
1-pbinom(599, 1000, prob=0.5), which equals 1.364232e-10 (less than 1 in a billion).