It is common to use, explore and test differences in attributes between groups. While often people are interested in significant differences and large effect sizes, I was wondering about the effect of slight differences between groups with theoretically similar distributions. Specifically, I am interested in how slight changes to a mean impact the number of extreme values (however decided upon).
Ex. Lets assume hypothetically that are measuring the IQ of two groups, A ($M=100, s.d.=15$) and B ($M=102, s.d.=15$). Suppose each group has $N=250$. We can simulate this easily:
set.seed(1) group_a <- rnorm(250, 100, 15) group_b <- rnorm(250, 102, 15) > sum(group_a>145)  0 > sum(group_b>145)  1 > t.test(group_a, group_b) Welch Two Sample t-test data: group_a and group_b t = -1.4821, df = 493.35, p-value = 0.1389 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -4.6843958 0.6559131 sample estimates: mean of x mean of y 100.3325 102.3468
We can see that the difference is non-significant and that group B have one person with an IQ higher than 145. This may be important as somethings correlate with very high attribute values (being a CEO, math prodigy, whathaveyou). If I wish to estimate the difference in the population in very high values of an attribute (e.g., IQ) based on a sample, how do I do it?
To make this clearer, repeating the same code above, but with group sizes of 250,000, the first group has 321 observations above 145, and the second 499 - a whopping $55.4$% more.
- What is a statistical explanation of how small differences in means between similar distributions cause massive differences in the far tail?
- How will this conclusion change when the standard deviation changes as well?
- What can be done to estimate this effect from small sample sizes (e.g., as in the example)?
- How will the effect change when the distribution is not normal (lets say with a long right tail)?