# How to interpret the significance code?

I'm studying for some time and I´m trying to do a logistic regression (using GLM in R) and now it´s extremely difficult to know what to do.

I have a binary dependant variable and 15 independent variables. As result of running GLM I've got:

glm(formula = y ~ ., family = binomial(link = "logit"), data = crs$dataset[crs$train,
c(crs$input, crs$target)])

Deviance Residuals:
Min       1Q   Median       3Q      Max
-2.9601  -0.5093   0.3074   0.4809   2.9348

Coefficients:
Estimate  Std. Error z value Pr(>|z|)
(Intercept) -0.52793176  0.19178032  -2.753  0.00591 **
x1          -0.03566232  0.01605379  -2.221  0.02632 *
x2           0.00497392  0.00110514   4.501 6.77e-06 ***
x3          -0.00002352  0.00000944  -2.491  0.01272 *
x4           0.01004249  0.01174335   0.855  0.39246
x5           0.10133956  0.01674787   6.051 1.44e-09 ***
x6           0.11445741  0.01819984   6.289 3.20e-10 ***
x7           0.06258882  0.01386824   4.513 6.39e-06 ***
x8           0.02266609  0.00103133  21.978  < 2e-16 ***
x9           1.05134745  0.11131339   9.445  < 2e-16 ***
x10         -0.46848579  0.09126661  -5.133 2.85e-07 ***
x11         -0.63923543  0.09490545  -6.735 1.63e-11 ***
x12         -0.36519602  0.08772172  -4.163 3.14e-05 ***
x13          0.02825176  0.00712733   3.964 7.37e-05 ***
x14          0.05050850  0.01282973   3.937 8.26e-05 ***
x15         -0.05083407  0.03117515  -1.631  0.10298
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 31488  on 26813  degrees of freedom
Residual deviance: 19489  on 26798  degrees of freedom
AIC: 19521

Number of Fisher Scoring iterations: 5

Log likelihood: -9744.441 (16 df)
Null/Residual deviance difference: 11999.293 (15 df)
Chi-square p-value: 0.00000000
Pseudo R-Square (optimistic): 0.66283672


I´d like to ask for help to see at this result to know if its a acceptable model.

I evaluated this model using ROC, and it gave me the result of 0,8843. Does it means that the prediction is 88,43% right?

Can I use as reference these significance codes (***,**,*,., )? Which one should I choose to keep in my model?

You still have a fair amount of studying to do (that can be a good thing).

First the way that you fit your model has 1 dependent variable and 15 independent, unless the definition of those terms have changed. Confusion like this is one of the reasons that I wish the entire field would ban the terms "independent variable" and "dependent variable".

If you compute a single number from an ROC curve it is generally the area under the curve (or some function of that) and does not represent the percentage correct. The percent correct depends on what cutoff value you use. If you look at the percent correct each way for every reasonable cut-off value and plot those then that is the ROC curve. An area under the curve (AUC) of 1 would mean perfect prediction and an AUC of 0.5 means that your model is equivalent to tossing a coin (not good), yours is in between, so it is better than a coin, but not yet perfect. To get the percent correct you need to decide on a cutoff value (but even that will be deceiving since you are testing with the same data used to create the model and a strict cut-off is less informative than the actual predictions).

The significance codes (*, etc.) are just categorizations of the p-values.

Which terms to keep in the model depends on the science behind the data and the question that you want to answer. Remember that the values in the output only measure the effect of that term conditioned on all the other variables being in the model, remove one term and all the p-values on the other terms can change (and change quite significantly).

Keep studying.

• Thanks @gregsnow, helped a lot, even to make sure that I need to study a lot more! – carlosedubarreto Oct 8 '13 at 18:26
• Could you clarify on the reasons for why you're against the use of IV and DV terms (or point to the corresponding materials online)? Thank you in advance! – Aleksandr Blekh Jan 14 '15 at 4:50
• @AleksandrBlekh, first the 2 words are similar enough that it is easy for people (especially students new to the field) to mix them up or otherwise confuse the terms. I also think that those 2 names are less descriptive than the alternatives. The "dependent" variable may be dependent on the others, but it also may not be, that is what we are testing, and the independent variables are rarely independent. And what happens if I believe that A causes B, but I want to use B to predict A (so B is x, A is y), which is the dependent and which is the independent variable? – Greg Snow Jan 14 '15 at 20:49
• Thank you, I understand your points. However, what is the alternative terminology? I'm not sure that it cannot also introduce some ambiguity (maybe less so). I was thinking about "predictor" and "response" (for predictive models) as well as "factor" ["feature" for machine learning models; some prefer "explanatory variable", but IMHO it's too long] and "outcome" (for causal/explanatory models). Any thoughts? – Aleksandr Blekh Jan 15 '15 at 0:31
• @AleksandrBlekh, I usually use predictor and response, though I have used explanatory and outcome from time to time as well. I don't think that any terms will ever be completely proof against misunderstanding, but I feel the risk is much lower with predictor/response than for independent/dependent. – Greg Snow Jan 15 '15 at 18:27