2
$\begingroup$

Someone I know told me that he had performed $\chi^2$ test of independence with Yates's correction on a $2\times n$ contingency table, seen that $p\lt10\%$ (good enough for his purposes), and calculated adjusted residuals as described by Sharpe, q.v., only to have none of the categories show up with sufficiently high residuals to stand out. He asked me what that means. I told him that I didn't think that that was possible but that he should probably look to the higher (absolute) values of the adjusted residuals.

  • Am I right that that's not possible (so there was some error in the calculation)?
  • If it is possible, what's the answer to his question?
$\endgroup$
4
  • 3
    $\begingroup$ This happens all the time, especially with such relaxed standards of significance. The underlying intuition is that a group of tiny deviations from independence can suffice, in the aggregate, as evidence of lack of independence while none of the deviations alone is adequate evidence. It is exactly the same phenomenon as having a significant overall regression (F) statistic without any individual coefficient being significant. Indeed, that's a duplicate thread. $\endgroup$
    – whuber
    Jan 16, 2019 at 13:00
  • 1
    $\begingroup$ Minor point: Yate's correction makes Pearson's $\chi^2$ act more like Fisher's "exact" test, which is conservative. Don't use continuity correction here. And instead of residuals, look at the raw stratified proportions. More important: don't put such emphasis on statistical significance, as Peter stated below. $\endgroup$ Jan 16, 2019 at 13:04
  • $\begingroup$ @whuber many thanks. Those answers all point to multicollinearity, which seems irrelevant here. $\endgroup$
    – msh210
    Jan 16, 2019 at 13:16
  • $\begingroup$ My answer points out that multicollinearity indeed is irrelevant. It is, however, almost always relevant in ANOVA unless you have used a carefully designed experimental arrangement of factors (to achieve orthogonality). $\endgroup$
    – whuber
    Jan 16, 2019 at 13:28

1 Answer 1

2
$\begingroup$

No, you are not right and yes, this is possible.

One way it is possible is that N is large. With a large enough N, even tiny differences are statistically significant.

There's no "answer" except to not rely so much on statistical significance.

$\endgroup$
2
  • $\begingroup$ The residuals are divided by population size. $\endgroup$
    – msh210
    Jan 16, 2019 at 12:25
  • $\begingroup$ @msh210 what do you mean? $\endgroup$
    – Glen_b
    Jan 17, 2019 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.