3
$\begingroup$

SPSS provides these three statistics as output for the crosstabs command STATISTICS=CMH(1):

  1. Tests of Homogeneity of the Odds Ratio
  2. Tests of Conditional Independence
  3. Mantel-Haenszel Common Odds Ratio Estimate

I am looking for confirmation or correction of my understanding of the first two and information on the third.

I think the first output is a test of the assumption of homogeneity (which is needed for M-H test).

Also, how might oe decide whether to use the M-H statistics or the Cochran's test of independence?

I think the second output is the M-H test of the null hypothesis that 2 variables are independent, controlling for level of a 3rd variable

Why is the third output different from the second? What does it add to the analysis insofar as it provides a p value?

Finally, When the Cochran and M-H test of conditional independence yield different p values, what method should be used to decide which to report? (In once analysis, the p values were .007 and .017, for example.)

$\endgroup$
  • $\begingroup$ Have you looked into SPSS Statistics Algorithms document? $\endgroup$ – ttnphns Oct 24 '14 at 20:17
  • $\begingroup$ Thank you for pointing me to this document as I was unaware of it (pic.dhe.ibm.com/infocenter/spssstat/v22r0m0/… ). However, it seems to have nothing on the question I posed. $\endgroup$ – Joel W. Oct 24 '14 at 21:25
1
+50
$\begingroup$

Tests of Homogeneity of the Odds Ratio. Breslow-Day and Tarone. They test H0 that odds ratios are the same (of whatever magnitude) across the strata in the population. In other words, that association between the dichotomous factor and the dichotomous response - as measured by odds ratios - is identical in the strata. Tarone's statistic is B-D statistic adjusted specifically to Mantel-Haenszel estimator (see common odds ratio below).

Tests of Conditional Independence. Cochran and Mantel-Haenszel. They test H0 that odds ratios are the same and equal 1 across the strata in the population. That is, logically these tests are simply tighter versions of the former. However statistically they are not "direct particular case" of the former and are a bit different because they were proposed independently by other authors. The tests of Homogeneity are implicitly logit-based (Agresti. Categorical Data Analysis): recall logit is log of odds; whereas Conditional Independence tests are model-free, they are not test of specifically odds values. (If there is no stratification variable, Cochran's test becomes the usual pearsonian chi-square test, and Mantel-Haenszel becomes very similar to the continuety-corrected chi-square test.)

In general, if your hypothesis is odds ratio = 1 so that both types of tests are applicable, Conditional Independence tests might be regarded as preferable on the grounds of their model-free nature. Also, they are better for small strata - Norusis, SPSS Procedures companion: "Breslow-Day requires large sample sizes in each stratum. Mantel-Haenszel conditional test requires merely a large overall sample size".

Regarding your question on the difference between the two Conditional Independence tests. SPSS says: "M-H test is similar to Cochran's, but makes corrections for small sample sizes. Note that the two statistics are not necessarily equivalent [i.e. may still quite differ] for large samples!". Also: "Of Cochran: When the number of strata is fixed as the sample sizes within each stratum increase, the statistic square is asymptotically distributed as a chi-squared distribution with 1 d.f. Of Mantel-Haenszel: When the number of strata is fixed as the sample sizes within each stratum increase, or when the sample sizes within each strata are fixed as the number of strata increases, this statistic is square is asymptotically distributed as a chi-squared distribution with 1 d.f.". Agresti: "The Mantel-Haenszel approach... is more general in that it also applies to some cases in which the rows [factor variable] are not independent binomial samples from two populations". "M-H with continuety correction approximates an exact conditional test but it tends to be conservative". Norusis: "Both tests of conditional independence do not perform well if the pattern of the association differs across the strata, particularly if they are opposite.".

Common Odds Ratio Estimate. Mantel-Haenszel estimator and its test. If strata are homogeneous in respect to odds ratio in population, a better than simply marginal measure of that common odds ratio can be derived, combining strata-conditional values in one estimate. This Mantel-Haenszel estimator is not ideal though: SPSS calls it "consistent but inefficient". A t-test of H0 that the estimator's value is equal to 1 or another test-value of your choice is also performed. If the test value is 1, one is tempted to expect p-value equal to at least one of the two Conditional Independence tests, because they test a similar thing that the odds ratio is 1. This is not the case. Conditional Independence tests bypass computation of any specific estimator of the odds ratio; they ask if the factor-response association is nonzero at least in some of the strata. On the other hand, test of Mantel-Haenszel estimator is a test of a specific estimator having a value, the homogeneity of strata is already assumed at the background and the equality of the estimate value to the hypothetical value is asked. Mantel-Haenszel estimator is, by the way, implied in tests of Homogeneity, so they are tied indeed; and Coditional Independence tests stand some way from (Mantel-Haenszel conditional statistic/test has nothing to do with Mantel-Haenszel estimator, don't mix them!).

$\endgroup$
  • $\begingroup$ Thank you for the explanation. What do you suggest I do when the Cochran and M-H test of conditional independence yield different p values? What method should be used to decide which to report? Or, how do I decide which to use beforehand? Also, pls provide links or citations for your references. $\endgroup$ – Joel W. Oct 31 '14 at 11:33
  • $\begingroup$ Joel, The citations I made are quoted and the texts are named, aren't they? For SPSS citations, these are from various places of SPSS documentation (SPSS Algorithms, SPSS Help, SPSS Case Studies). I'm myself not an expert on the topic, so you should decide yourself whether you take my reasoning or not. Regarding reporting, I'd choose to report both Cochran and M-H tests, with some commenting on their differences, if they happen to give very different p-values. $\endgroup$ – ttnphns Oct 31 '14 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.