1
$\begingroup$

I made a logit link, GLM model with 7 explanatory variables. How do I interpret the coefficients and CI?

> summary(w1.glm)

Call:
glm(formula = DV ~ x1 + x2 + x3 + x4 + x5 + x6 + x7, family = binomial("logit"), data = myData)

Deviance Residuals: 
Min       1Q   Median       3Q      Max  
-6.6379  -1.3183  -0.0639   1.3031   9.5950  

Coefficients:  
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) -9.445065   0.326242 -28.951  < 2e-16 ***
fgp         19.579405   0.575186  34.040  < 2e-16 ***
ftp          0.207124   0.301471   0.687    0.492    
trbg         0.080704   0.004006  20.144  < 2e-16 ***
astg        -0.035330   0.005031  -7.023 2.17e-12 ***
stlg         0.119556   0.008310  14.388  < 2e-16 ***
blkg         0.087662   0.008508  10.303  < 2e-16 ***
tovg        -0.240836   0.005490 -43.868  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)

Null deviance: 6773.3  on 833  degrees of freedom
Residual deviance: 3308.5  on 826  degrees of freedom
AIC: 7276.5

Number of Fisher Scoring iterations: 4

> (exp(cbind(OR=coef(winp.glm), confint(winp.glm))))
Waiting for profiling to be done...
                      OR        2.5 %       97.5 %
(Intercept) 7.907884e-05 4.169786e-05 1.498016e-04
x1          3.185866e+08 1.033026e+08 9.847440e+08
x2          1.230135e+00 6.812482e-01 2.220951e+00
x3          1.084050e+00 1.075576e+00 1.092601e+00
x4          9.652867e-01 9.558130e-01 9.748488e-01
x5          1.126997e+00 1.108799e+00 1.145511e+00
x6          1.091619e+00 1.073572e+00 1.109981e+00
x7          7.859708e-01 7.775485e-01 7.944634e-01
$\endgroup$
  • $\begingroup$ A confusing aspect of this question is that it refers to p-values for the coefficients and to residuals as if they were somehow the same thing, even though those are entirely different. Could you clarify? BTW, there's nothing unusual with p-values of coefficients being this small (or even much, much smaller), especially with large datasets. $\endgroup$ – whuber Jan 17 '14 at 0:05
3
$\begingroup$

The direct interpretation of the coefficients in the logit model is somehow difficult. Because the inverse of the link function is not constant and it depends on the value of explanatory variables as mentioned here. But you can use the odd ratio as explained in the link. First we need to define the odd as $$Odds=\dfrac{p}{1-p}=\exp(X\beta)=\exp(\sum_{i=1}^kX_{i}\beta_i),$$ i.e. the ratio of probability of success to the probability of failure. Then the odds ratio is the ratio of two odds and we can obtain it for each explanatory variable. For example the the odds ratio for $X_1$ is $$\dfrac{\exp((X_1+1)\beta_1)+\sum_{i=2}^kX_{i}\beta_i)}{\exp(\sum_{i=1}^kX_{i}\beta_i)}=\exp{(\beta_1)}.$$ Now as an example, the odds ratio for trbg is $e^{0.080704}=1.08405$. This means that a successful outcome has almost the same chance if we increase the trbg by one unit.

If you look at the column Pr(>|z|), you can see that all of them are very small except the one for ftp. This means that inclusion of all the other explanatory variables (except the one for ftp) in your model is statistically significant at $\alpha=0.001$. However, the contribution of the variable ftp is not statistically significant as its $p$-value is large.

The confidence intervals can be intrepreted as how much we are confident that those intervals cover the true parameter. For example: for $x1$, you see $(18.45317347,20.70789229)$. It means that we are 95% confident that the interval from 18.45317347 to 20.70789229 covers the true parameter corresponding to fgp variable in your model. However, the confidence interval for $x2$ (i.e. from -0.38382854 to 0.79793539) contains zero. This confirms our previous conclusion that adding the variable ftp is not statistically significant.

$\endgroup$
  • $\begingroup$ Thanks! This was very helpful. I read somewhere about how the interpretation of logistic model output differed from linear, but couldn't find detail comprehensible to newbies. $\endgroup$ – user2205916 Jan 17 '14 at 2:01
3
$\begingroup$

You seem to be confusing $z$ scores or similar for $z$-statistics. The values in the column labelled z value are Wald statistics and are part of a Wald test that the maximum likelihood estimate of a model coefficient is equal to 0. The test assumes that the difference between the maximum likelihood estimate and 0 is asymptotically normally distributed.

The $z$ statistic is

$$ z = \frac{\hat{\theta} - \theta_0}{\mathrm{se}(\hat{\theta})} $$

is where $\hat{\theta}$ is the MLE of the coefficient of interest, $\theta_0$ is the value to compare with (0 in the case of the summary() output) and $\mathrm{se}(\hat{\theta})$ is the standard error of the MLE. The corresponding Wald statistic is $w = z^2$.

$z$ is approximately normally distributed and the $p$-value is the probability of achieving a value as extreme or more extreme than the observed value. $w$ is approximately $\chi^2_1$ distributed.

Crudely, you could treat these like the $t$ statistics in the summary output from a linear model, but they are not the same and have different properties.

The confidence intervals are profile likelihood intervals but can be interpreted like ordinary confidence intervals. They are computed from the shape of the likelihood function about the MLE, which need not be symmetric. Wald confidence intervals can be formed using an assumption of normality (using confint.default() instead of confint(), but the profile likelihood versions are generally favoured.

$\endgroup$
  • 3
    $\begingroup$ I won't attempt to give a definition of exactly what the confidence interval means as this is formally tricky and I'll probably get it wrong. $\endgroup$ – Reinstate Monica - G. Simpson Jan 17 '14 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.