Comparing the spread of data from differently-sized populations

Question

I have what seems like a simple but fundamental question. Some posts on CV seem to be along similar lines, but not quite address what I'm trying to understand (for instance Comparing percentages from different sample sizes).

Suppose I rolled six dice and counted the number of 6s that came up in each roll. I wouldn't be too surprised to see the total number of 6s vary a lot between zero and six, and over the course of many rolls I'd expect to get a mean of one 6 per roll. If I rolled sixty dice, I'd expect to get a mean of ten 6s per roll, however I'd be really surprised if I rolled sixty 6s, or zero sixes. Next I divide the results for each group of dice by the number of dice in the group (and the number of rolls). In each case I should find that the probability of any particular die rolling a 6 is 1/6. But I'd expect the spread (measured say by the standard deviation) to be much tighter in the group of sixty dice than the group of six dice, since the extreme tails of the distribution (all zeros or all sixes) are much less likely to occur in the group of sixty dice than in the group of six dice. Am I right so far? So if you showed the data for both groups of dice to someone who didn't know that the two groups were different sizes, it will look like the dice in the smaller set are behaving more randomly than the dice in the larger set.

Now apply this to a comparison between real-world data like traffic accidents in a small town vs a large city. Suppose you wanted to find the probability of someone in each case being injured in a car accident. Injuries per capita would give a value for the mean probability of being injured. But it seems to me that the spread of the data would be narrower in the large city than the small town, and hence make the drivers in the small town seem more erratic than they truly are.

Anyway, I'm pretty sure I've just rediscovered a fairly basic problem in statistics, but I'm not sure what the name of the problem is or what to google to find out more about it. I'm don't think this is a t-test or ANOVA question, because I'm not explicitly trying to compare two samples to see if they're drawn from the same population (in fact, in the traffic accident example I explicitly know they're not).

So is there a way to rescale or represent the spread of the data which doesn't bias the spread from large populations to look better (more tightly-constrained) than the data from small populations? In other words, a population-independent measure of the spread of data, which is robust to having the data rescaled so as to interpret the mean as a probability (i.e. a value between 0 and 1)?

Thanks in advance

Answer 1

In your car accident problem it might be appropriate to model the number of accidents per year as Poisson for both a town and a city.

The mean would likely be larger for the city over a year. For a Poisson distribution the mean and variance are numerically the same, so the variance would also tend to be larger for the city.

Here is an example of a city with $\lambda_c = 700$ accidents per year and a town with $\lambda_t = 100.$

set.seed(2022)
city = rpois(10, 700)
mean(city); var(city)
[1] 703.8
[1] 1023.067
town = rpois(10, 100)
mean(town); var(town)
[1] 108.2     # less than 704
[1] 69.51111  # less than 1023

For only ten years you can't expect the sample mean to be a close estimate to the population mean or the sample variance to be close to the population for either city or town. But in my example, the sample mean and variance are both larger for the city.

It is easy to do a simulation to show that the sample variance for the town is indeed almost always smaller than the sample variance for the city. In the simulation, the town had the smaller variance $9997$ out of $10/,000$ comparisons.

set.seed(609)
ratio = replicate(10000, var(rpois(10, 100))/
                        var(rpois(20, 700)))
mean(ratio < 1) 
[1] 0.9997                        

Addendum: You don't say anything about doing a test to see if you can distinguish the accident rates between 25 smaller cities (maybe $\lambda_1 = 300)$ and five larger ones (maybe $\lambda_2 = 350).$

Then the Poisson counts will be nearly normal, but the variances may be larger for larger cities. So, a Welch t test will be OK (not assuming equal population variances. For example, comparing 25 cities (where $\lambda_1 =300)$ with 5 larger cities (where $\lambda_2 =350)$ will typically give a significant result, as in the example below:

t.test(rpois(25,300), rpois(5, 350), alt="less")

        Welch Two Sample t-test

data:  rpois(25, 300) and rpois(5, 350)
t = -5.1896, df = 4.8313, p-value = 0.001931
alternative hypothesis: 
true difference in means is less than 0
95 percent confidence interval:
      -Inf -29.68011
sample estimates:
mean of x mean of y 
   300.64    349.40 

If we simulate this scenario $100\,000$ times, we find that a significant difference (5% or less) is found most of the time--the power of such a test is above 99%.

set.seed(609)
pv = replicate(10000, 
      t.test(rpois(25,300), rpois(5, 350), 
       alt="less")$p.val)
mean(pv <= 0.05)
[1] 0.9983