0
$\begingroup$

I have a logistic regression model with several variables and one of those variables (called x3 in my example below) is not significant. However, x3 should remain in the model because it is scientifically important.

Now, x3 is continuous and I want to create a plot of the predicted probability vs x3. Even though x3 is not statistically significant, it has an effect on my outcome and therefore it has an effect on the predicted probability. This means that I can see from the graph, that the probability changes with increasing x3. However, how should I interpret the graph and the change in the predicted probability, given that x3 is indeed not statistically significant?

Below is a simulated data in R set to illustrate my question. The graph also contains a 95% confidence interval for the predicted probability (dashed lines):

> set.seed(314)
> n <- 300
> x1 <- rbinom(n,1,0.5)
> x2 <- rbinom(n,1,0.5)
> x3 <- rexp(n)
> logit <- 0.5+0.9*x1-0.5*x2
> prob <- exp(logit)/(1+exp(logit))
> y <- rbinom(n,1,prob)
> 
> model <- glm(y~x1+x2+x3, family="binomial")
> summary(model)

Call:
glm(formula = y ~ x1 + x2 + x3, family = "binomial")

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.0394  -1.1254   0.5604   0.8554   1.4457  

Coefficients:
            Estimate Std. Error z value Pr(    >|z|)    
(Intercept)   1.1402     0.2638   4.323 1.54e-05 ***
x1            0.8256     0.2653   3.112  0.00186 ** 
x2           -1.1338     0.2658  -4.266 1.99e-05 ***
x3           -0.1478     0.1249  -1.183  0.23681    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 373.05  on 299  degrees of freedom
Residual deviance: 341.21  on 296  degrees of freedom
AIC: 349.21

Number of Fisher Scoring iterations: 3

> 
> dat <- data.frame(x1=1, x2=1, x3=seq(0,5,0.1))
> preds <- predict(model, dat,type = "link", se.fit = TRUE )
> critval <- 1.96
> upr <- preds$fit + (critval * preds$se.fit)
> lwr <- preds$fit - (critval * preds$se.fit)
> fit <- preds$fit
> 
> fit2 <- mod$family$linkinv(fit)
> upr2 <- mod$family$linkinv(upr)
> lwr2 <- mod$family$linkinv(lwr)
> 
> plot(dat$x3, fit2, lwd=2, type="l", main="Predicted Probability", ylab="Probability", xlab="x3", ylim=c(0,1.00))
> lines(dat$x3, upr2, lty=2)
> lines(dat$x3, lwr2, lty=2)

enter image description here

Thanks!

Emilia

$\endgroup$
2
$\begingroup$

The "significance" of the effect has no effect at all on the interpretation, given that you pay attention to confidence intervals. Even better might be to compute simultaneous confidence intervals as made easy by the R rms package Predict, plot.Predict, and lrm functions using the R multcomp package.

$\endgroup$
  • $\begingroup$ Thank you for your comment. I will look into the rms package. Could you please further clarify what you mean by your first sentence? $\endgroup$ – Emilia Oct 23 '13 at 13:42
  • 1
    $\begingroup$ I mean that confidence intervals always have an interpretation no matter what the statistical test says. And unlike large $P$-values, confidence intervals can provide evidence in favor of the null. $\endgroup$ – Frank Harrell Oct 23 '13 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.