2
$\begingroup$

Is it possible to have a non statistically significant difference for all sub sets but a statistically significant one for the total? If so, how should this be interpreted?

Context I was reading an article (in Italian) about school results improvements in different regions in Italy, and for one of the disciplines the difference in results from two different years was non statistically significant when looking at each region, but statistically significant when looking at Italy as a whole.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This strongly reminds me on the Simpson paradox. Have a look en.wikipedia.org/wiki/Simpson%27s_paradox $\endgroup$
    – Semoi
    Commented Jul 28, 2019 at 19:08
  • $\begingroup$ @Semoi Interesting, I hadn't though about the Simpson paradox! Thanks! $\endgroup$
    – barbara
    Commented Jul 28, 2019 at 19:47

1 Answer 1

Reset to default
3
$\begingroup$

There are two simple explanations that occur to me:

1) Sample size is an important factor in p-value calculations. Since your sample size is by definition larger for all of Italy than it is for any constituent region, you have greater power to detect smaller effects and may therefore get significant p-values even with identical coefficient values.

2) As @Semoi points out, patterns within regions may be absent or even opposite to patterns across regions (Simpson's paradox).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.