1
$\begingroup$

I calculated both confidence interval and p-value on the same model using R, and the two tests of significance disagreed with one another.

I am comparing how 3 groups differ in terms of an ordinal variable, once a separate numeric predictor was taken into account. The model indicated that group was a significant predictor, and when I graphed the probabilities, I noticed that two of the groups were very similar, so I created a subset of the data only containing these groups to see if group was still a statistically significant predictor.

The p-value indicated that it was not (p=0.64), but the confidence interval indicated that it was a significant predictor (CI 2.5% = 0.68, CI 95% = 1.28).

I'm not sure where to go from here. I did a couple of other tests (which make a few extra assumptions and so aren't ideal), and all of them indicated non-significance. I've been using confidence intervals for this entire project, so suddenly switching to p-values is a bit jarring and I would prefer not to turn to tests & models that are less ideal.

I wish to know why this happened, what it means, and, if possible, whether this is due to a mistake in my code.

here's the code for the model & statistical tests I used:

library(MASS)
#model
DvS<-polr(ordinal~numeric+group, data=DvSata, Hess = TRUE)
summary(DvS)

#confidence intervals & odds ratios
ci<-confint(DvS)
exp(coef(DvS))
exp(cbind((OR = coef(DvS)), ci))

#p-value
st <- coef(summary(DvS))
pval <- pnorm(abs(st[, "t value"]),lower.tail = FALSE)* 2
st <- cbind(st, "p value" = round(pval,3))
st 
```
$\endgroup$

1 Answer 1

2
$\begingroup$

Recall that the null hypothesis for an odds ratio (OR) is that $\text{OR} = 1$, which occurs when $\text{log(OR)} = 0$. The confidence interval you produced is around the OR; it contains 1, and therefore is nonsignificant, consistent with the p-value. If you produce a confidence interval around the $\text{log(OR)}$ (i.e., the coefficient itself), you will see that it will contain 0.

$\endgroup$
1
  • $\begingroup$ Ach, that's embarrassing. Not sure how i managed it. all well, at least the problem is solved $\endgroup$ Dec 7, 2021 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.