Margins in “R” throws confidence intervals that do not contain cero, yet p-value > 0.05. Is this possible?

I am running a logit model on survey data using the svyglm function in the survey package in R. I am using the margins function to find the average marginal effects, but then something weird happens. The confidence intervals for the 's1_022' variable have confidence intervals that do not contain 0, yet the reported p-value is higher than 0.05.

> summary(margins(model1, design=svyvis1))
factor     AME     SE       z      p   lower   upper
esup1 -0.1332 0.0155 -8.5863 0.0000 -0.1690 -0.1082
export_g -0.0022 0.0217 -0.1018 0.9189 -0.0448  0.0402
import_g -0.0051 0.0133 -0.3830 0.7018 -0.0314  0.0208
s1_022 -0.0278 0.0147 -1.8938 0.0583 -0.0578 -0.0003
s1_03a -0.0010 0.0007 -1.5805 0.1140 -0.0024  0.0002
s1_101  0.0209 0.0144  1.4543 0.1459 -0.0064  0.0500
s2_202 -0.1031 0.0230 -4.4755 0.0000 -0.1521 -0.0619
s2_203 -0.0063 0.0290 -0.2189 0.8267 -0.0634  0.0503
s2_204 -0.0230 0.2224 -0.1033 0.9177 -0.4597  0.4122
s2_205 -0.0595 0.0296 -2.0074 0.0447 -0.1197 -0.0035
s2_36a2  0.2657 0.0169 15.7025 0.0000  0.2383  0.3046
tothrs  0.0002 0.0005  0.4866 0.6265 -0.0007  0.0012
trim227 -0.0147 0.0252 -0.5834 0.5596 -0.0648  0.0341
trim228  0.0066 0.0269  0.2465 0.8053 -0.0458  0.0596
trim229 -0.0645 0.0261 -2.4755 0.0133 -0.1185 -0.0163
trim230 -0.0129 0.0244 -0.5309 0.5955 -0.0613  0.0343
trim231 -0.0398 0.0265 -1.5025 0.1330 -0.0934  0.0104
yprilab -0.0008 0.0003 -2.3803 0.0173 -0.0014 -0.0002


As far as I know, this should not happen since the fact that the confidence intervals do not include zero means that the average marginal effect is not significant. Am I getting something wrong? In which circumstances could this happen?

• The confidence intervals are not symmetric about the AME. When this is the case, this result is possible. The CI is not simply $AME \pm c \times SE$. So they are arrived at differently. The p-values are based on the Wald z-statistic, the CI are not Wald CI. Different assumptions, different results. This is assuming the package is not making a programming error. – Heteroskedastic Jim Oct 18 '18 at 22:02
• Are you sure that the reported confidence intervals are 95%? – St. Inkbug Oct 19 '18 at 5:28
• I googled the information about svyglm. Its manual does not say what methods are used. So it just likes black box. I will not use it before I get the statistical theory behind the black box. – user158565 Oct 19 '18 at 14:28
• @a_statistician in fact the confidence intervals come from the margins package but that is not explicit about how it estimates them either. I think the OP has to look at the code for enlightenment. – mdewey Oct 19 '18 at 14:51
• @mdewey I do not use R. But I hope I could find the theory that support svyglm. I am very disappointed by the search results. – user158565 Oct 19 '18 at 14:56

If confidence and p-values are calculated using different methods, then this situation is possible. The $$z$$-test you have there is a Wald's $$z$$-test:

$$z=\frac{AME}{SE}\quad\text{then}\quad p=\min{\Big\{1,2\times\big(1-\Phi(\lvert z\rvert)\big)\Big\}}$$

# in R syntax:
min(1, 2 * (1 - pnorm(abs(AME / SE))))


For the confidence intervals to correspond to the p-value as you want, then one must calculate Wald confidence intervals using:

$$AME\pm\Phi^{-1}(.975)\times SE\approx AME\pm1.96\times SE$$

# in R syntax:
AME + qnorm(.975) * c(-SE, SE)


This will produce confidence intervals that are symmetric about the AME i.e. the CI limits will be equidistant from the AME. In your pasted results, the confidence intervals are clearly not symmetric about the AME.

Hence, the CI and p were calculated using different methods, possibly under different assumptions, there is no reason for them to "agree".

I have no idea how the CIs are calculated here. You should probably email the package creator.

• You are right. The methods for computing confidence intervals and p-values must be different. Stata seems to calculate symmetric confidence intervals instead. I could not find how CIs are calculated in the R margins documentation, so if nothing else comes up here, emailing the package creator might be a good idea. Thanks!!! – J.M. Molina Oct 19 '18 at 15:20
• @jojas412 usually, if you accept the answer, then you can tick it as accepted. Also, if you're able to read code easily, it might be faster to explore the source code yourself here: github.com/leeper/margins – Heteroskedastic Jim Oct 20 '18 at 4:21