0
$\begingroup$

I am doing statistics for the first time in my life and I am not quite sure what to include and how to interpret the results. I am doing a logistic regression in R. Here is what I have so far:

  1. GLM with family = binomial (dependent ~ indep1 + indep2 + ...+ indep7 +0) If I dont include the 0 I get NA for my last independent variable in the summary output..

  2. Update the model (indep2 has a p-value > 0.05 and is left out)

  3. I am applying anova

    anova(original_model,updated_model, test="Chisq")
    
       Resid.Df  Resid.Dev Df Deviance Pr(>Chi)
    1     34067      18078                     
    2     34066      18075  1   2.4137   0.1203
    

    Here I am not sure how to interpret it. What tells me if the simplification of the model is significant? the p-value is with 0.12 bigger than 0.05, does this mean that the simplification is not significant?

  4. make a cross-table (compare predicted (probability >0.5) - observed)

    fit
          FALSE  TRUE
      No  30572    68
      yes  3407    31
    

    I'd say that 31 values are predicted correctly (yes-true), resp 68 (no-true) but that most values are classified wrong, which means that the model is really bad?

  5. then I make a wald test for each independent variable for the first independent variable it would look like this:

    > wald.test(b = coef(model_updated), Sigma = vcov(model_updated), Terms
    > = 1:1)
    

    here I only look if the p-values are significant and if they are it means that all variables contribute significantly to the predictive ability of the model

  6. I calculate the odds with their confidence intervals (this is basically exp(estimate)

    oddsCI <- exp(cbind(OR = coef(model_updated), confint(model_updated)))
    

    For all odds smaller than 1 i do 1/odd

    Estimate        Odds Ratio      Inverse Odds
    -0.000203       0.999801041     1.000198999
     0.000332       1.000326571     odd bigger than 1
    -0.000133       0.999846418     1.000153605
    -3.48       0.008696665     114.9866056
    -4.85       0.029747223     33.61658319
    -2.37       0.000438382     2281.113996
    -8.16       0.110348634     9.062187402
    -2.93       0.062668509     15.95697759
    -3.65       0.020156889     49.61083057
    -5.45       0.033996464     29.41482359
    -4.02       0.004837987     206.6975334
    

    This O would interpret like that for the "odd bigger than 1" the case is over 1 times more likely to occur. (Is is incorrect to say that, or not?) Or for the last row you could say that t for every subtraction of a unit, the odds for the case to appear decreases by a factor of 206.

  7. Then I look at

    with(model_updated, null.deviance - deviance) #deviance
    with(model_updated, df.null - df.residsual) #degrees of freedom
     # pvalue
    with(Amodel_updated, pchisq(null.deviance - deviance, df.null - df.residual, 
    lower.tail = FALSE))
    logLik(model_updated)
    

    But I don't really know what this tells me.

  8. In a last step I do

    stepAIC(model_updated, direction="both")
    

    but also here I don't know how to interpret the outcome. I see that it looks at all interactions between my independent variables but I don't know what it tells me.

After this, I can make a prediction by using the updated model and by separating it into training data and validation data I suppose?

$\endgroup$

closed as too broad by gung, Nick Cox, Glen_b, Nick Stauner, Peter Flom Apr 27 '14 at 21:52

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ You have very many, & very basic questions here. What you need is to take (possibly several) statistics courses, or, if you are not a student, get several statistics books & work through them. Even if all of these questions were answered here, & the ideas behind these issues were explained, I suspect there would be other topics you don't know or misunderstand that you didn't think to ask about. Providing all of the relevant information here is, in effect, a statistics course / textbook. You should work w/ a statistical consultant. $\endgroup$ – gung Apr 27 '14 at 14:05
4
$\begingroup$

You are making a number of assumptions that seem to come from the machine language community and ignore decades of statistical theory and application.

  1. Something is wrong with pre-specifying a multivariable model
  2. Something is wrong with leaving insignificant variables in a model
  3. Classification is useful and well-defined, with a cutpoint of 0.5 yielding the correct loss function
  4. Proportion classified correctly is a useful measure of predictive accuracy with direct probability models
  5. All predictor effects are linear
  6. Odds ratios per 1-unit change in all $X$s are interesting

In fact none of these assumptions is likely to be true.

It would be valuable to spend a good deal of time studying multivariable modeling, and logistic regression in particular, before running analyses.

$\endgroup$
  • $\begingroup$ I try to understand the whole topic, but it is quite complexe. Beside the fact that alot is wrong, would you recommend including another sort of test or should it be possible to interprete the results with these steps (applyed correctly then of course) $\endgroup$ – Pat Apr 27 '14 at 13:04
  • 4
    $\begingroup$ Please forgive the sensitivity of a biostatistics professor but if I wanted to build a house I would study carpentry first. $\endgroup$ – Frank Harrell Apr 27 '14 at 13:09
  • 2
    $\begingroup$ @user777, you should check out his book, Regression Modeling Strategies, which covers these kinds of topics in great detail. $\endgroup$ – gung Apr 27 '14 at 14:07
  • 2
    $\begingroup$ Some if this is covered in my course notes at biostat.mc.vanderbilt.edu/rms $\endgroup$ – Frank Harrell Apr 27 '14 at 14:10
  • 2
    $\begingroup$ I think there needs to be some credentialing. The American Statistical Association has a credentialing system for professional statisticians. I think that anyone doing data analysis should have some minimum level of credentials. Sometimes by helping each other we delay the inevitable need for studying - I'm torn about whether it's always good to help with isolated questions when there are deeper questions. $\endgroup$ – Frank Harrell Apr 27 '14 at 23:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.