I'm currently reading this study about the adherence to health checkups in Austria. Table 2 displays some Odds Ratios for attending health checkups between groups of different educational statuses. I am confused about the row in the women subgroup. The OR is 1.187 with an 95% CI of (0.981, 1.436) but the p-value is given as < 0.001. Since the confidence interval contains the OR under the null (OR = 1) I'd expect an p-value of at most 5%. Is this an error in the paper or am I misunderstanding something?
$\begingroup$
$\endgroup$
4
-
$\begingroup$ Could this be due to reporting p-values from a Chi-Square test that assumes normality (inaccurate at low sample sizes) and then giving the CI for the Odds Ratio based on a different probability distribution that SPSS uses for estimates of multivariate logistic regression? $\endgroup$– Michael RoswellCommented Feb 16, 2021 at 21:51
-
$\begingroup$ @Michael This would make sense, since I assume that the normality assumption of the Chi-Square makes the test more powerful. Still, I'm not sure and I'd be happy to get more insight. $\endgroup$– s1624210Commented Feb 17, 2021 at 12:42
-
$\begingroup$ I'm not familiar with SPSS but this seems to suggest that it may use Wald Chi-square tests for some things and likelihood profile Chi-squared test for others: stats.idre.ucla.edu/spss/output/logistic-regression. $\endgroup$– Michael RoswellCommented Feb 18, 2021 at 19:41
-
$\begingroup$ I'd be careful about whether a normality assumption can increase power. You have discrete values that a proportion can take (and associated discrete probability distributions), and the discreteness gets to be big deal with small sample sizes. As sample sizes get huge, the probability around a your estimated proportion gets more symmetrical and smoother (basically continuous) and more normal in shape, by the central limit theorem. But with modest sample sizes, if you perceive greater power with an incorrect assumption of normality, beware of being overconfident, not more sensitive. $\endgroup$– Michael RoswellCommented Feb 18, 2021 at 19:42
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I strongly suspect that this particular part of the table, explicitly based on a multi-predictor logistic regression model stratified by sex, has an error in the p-value entry. The odds ratios are presented with respect to a reference category for each of the predictors, so that the p-values should represent the p-value corresponding to a null hypothesis of an odds ratio of 1 (equivalently, a logistic regression coefficient of 0). The width of the confidence interval is similar to those of other coefficients, leading to my suspicion that the error is in the p value.
The posted odds ratio and confidence interval are equivalent to a logistic regression coefficient estimate of 0.171 with a standard error of 0.097. The z-value of 1.76 doesn't pass the usual significance criterion, it's about p = 0.08 for a two-sided test. The error in the table is unfortunate, but having put together many tables like this I have some sympathy for the authors on that account; proofreading such tables is hard.
What's perhaps more unfortunate is this type of presentation of "significance" for multi-leveled categorical variables. Do you really care whether women with "tertiary education" have significantly different odds of getting checkups than do women with only primary education? Or do you care that the level of education is more generally associated with the odds of getting checkups? The display in that table only shows the former 2-level comparison, without an overall test of the significance of educational level. For that (in my opinion more useful) analysis of overall significance, you need to look at a likelihood-ratio test between models that include and exclude education as a predictor, or perform a Wald test on the multiple parameter estimates for the coefficients associated with education.
