Differences between conclusions from a p-value and confidence intervals

Question

I have an issue with a result I just found.

I ran a sub-population analysis (for 2 age groups) with a complex survey design.

For this purpose I fitted a generalized linear model (family:normal,link:identity), using the function svyglm from the R package survey.

Here is my code for the model:

model=svyglm(score ~ DLA +relevel(sex,2)+raeth+pir+age+educa+(DLA:re‌​level(sex,2)),
               design,data=imp1)

DLA is my exposure of interest and was included as categorical (tertiles)

For one of my sub-population analyses (for older age group), my results suggest a significant difference of (-7.50 (95% CI:-14.54, -0.46)) between the third tertile and the first tertile of exposure), but the p-value=0.058 suggested a non significant result.

There is not such a big difference, but I wonder how it could be possible that the confidence interval does not include 0, whereas the p-value suggests a non-significant result.

Thank you very much for your answers

Answer 1

Both should be considered as ''borderline''... this is not so surprising.

Are you familiar with likelihood theory, and the three tests? Assuming it is the case, assume that the confidence interval is a Wald confidence interval, and that the $p$-value is obtained from a score test or a likelihood ratio test. Asymptotically, you will have $p < 0.05$ if, and only if, $0$ is not in the confidence interval, but for small samples you can meet an exception from time.

More generally, the confidence interval and the $p$-value can be obtained by procedures which are not exactly the dual of each other. This example (generated after a few trials) exhibits a similar behavior:

> x <- c(7, 3, 6, 7, 8, 4, 4, 3, 10, 5)
> y <- c(9, 19, 5, 12, 9, 13, 7, 19, 11, 7)
>  fit <- glm(y~x, family=poisson)
> summary(fit)

Call:
glm(formula = y ~ x, family = poisson)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-1.92492  -1.15183   0.06332   1.07970   1.31742  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  2.88977    0.25941  11.140   <2e-16 ***
x           -0.08786    0.04557  -1.928   0.0539 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 


> confint(fit)
                 2.5 %       97.5 %
(Intercept)  2.3732479  3.391536328
x           -0.1791838 -0.000207765