14
$\begingroup$

In R, the drop1command outputs something neat.
These two commands should get you some output:
example(step)#-> swiss
drop1(lm1, test="F")

Mine looks like this:

> drop1(lm1, test="F")
Single term deletions

Model:
Fertility ~ Agriculture + Examination + Education + Catholic + 
    Infant.Mortality
                 Df Sum of Sq    RSS    AIC F value     Pr(F)    
<none>                        2105.0 190.69                      
Agriculture       1    307.72 2412.8 195.10  5.9934  0.018727 *  
Examination       1     53.03 2158.1 189.86  1.0328  0.315462    
Education         1   1162.56 3267.6 209.36 22.6432 2.431e-05 ***
Catholic          1    447.71 2552.8 197.75  8.7200  0.005190 ** 
Infant.Mortality  1    408.75 2513.8 197.03  7.9612  0.007336 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

What does all of this mean? I'm assuming that the "stars" help in deciding which input variables are to be kept. Looking at the output above, I want to throw away the "Examination" variable and focus on the "Education" variable, is interpretation this correct?

Also, the AIC value, lower is better, yes?

Ed. Please note the Community Wiki answer below and add to it if you see fit, to clarify this output.

$\endgroup$
  • 7
    $\begingroup$ the help in R is meant to explain you how to use the function. It's not meant to be a course on statistics. And regarding that, in general I believe the R help pages are among the most complete and handy from all open source packages I know of. And paying packages for that matter. SPSS and SAS give you a lot of mumbo-jumbo with half-truths and complete nonsens as a "guide for interpretation". $\endgroup$ – Joris Meys Nov 17 '10 at 16:03
  • 1
    $\begingroup$ This question has been downvoted. I didn't intend to give my +1, but it seems to me now that voting it down is not very constructive: (1) the OP makes clear this is homework and uses an R built-in data set for illustration, not his data, (2) a related question with step() has been rated +2 at the time of this writing (so why?!), (3) the OP acknowledged the usefulness of @Joris's response. $\endgroup$ – chl Nov 18 '10 at 9:52
  • $\begingroup$ @chl : seems like I'm not the only one with sensitive toes when it comes to the R help pages :-). But I agree wholeheartedly with you. The question is valid, asked in a clear way and hence there is absolutely no reason whatsoever to downvote it. $\endgroup$ – Joris Meys Nov 18 '10 at 10:31
  • $\begingroup$ Heh, I'm sorry if I stepped on your toes with my snide at the help, I'm just not very patient when it comes to anything with a command line really. I'm weird that way, I know. You wouldn't be the first ones to call me out on it :) I like this place, people are honest. $\endgroup$ – gakera Nov 18 '10 at 10:49
  • $\begingroup$ There we go, I edited the question so that it's not as off-putting to advocates of R and the R help :) And reworded the question on AIC to avoid misleading OP only readers. $\endgroup$ – gakera Nov 18 '10 at 10:55
10
$\begingroup$

drop1 gives you a comparison of models based on the AIC criterion, and when using the option test="F" you add a "type II ANOVA" to it, as explained in the help files. As long as you only have continuous variables, this table is exactly equivalent to summary(lm1), as the F-values are just those T-values squared. P-values are exactly the same.

So what to do with it? Interprete it in exactly that way: it expresses in a way if the model without that term is "significantly" different from the model with that term. Mind the "" around significantly, as the significance here cannot be interpreted as most people think. (multi-testing problem and all...)

And regarding the AIC : the lower the better seems more like it. AIC is a value that goes for the model, not for the variable. So the best model from that output would be the one without the variable examination.

Mind you, the calculation of both AIC and the F statistic are different from the R functions AIC(lm1) resp. anova(lm1). For AIC(), that information is given on the help pages of extractAIC(). For the anova() function, it's rather obvious that type I and type II SS are not the same.

I'm trying not to be rude, but if you don't understand what is explained in the help files there, you shouldn't be using the function in the first place. Stepwise regression is incredibly tricky, jeopardizing your p-values in a most profound manner. So again, do not base yourself on the p-values. Your model should reflect your hypothesis and not the other way around.

$\endgroup$
  • 1
    $\begingroup$ I like this sentiment, "if I don't understand what I'm doing already, I shouldn't try to learn it..." This is also the approach taken in the R help - it's not helpful unless you already know what's going on. I was hoping this could be the start of something different. $\endgroup$ – gakera Nov 17 '10 at 16:37
  • $\begingroup$ But I can use this part of your answer: "Interprete it in exactly that way: it expresses if the model without that term is significantly different from the model with that term." To me this means that the Pr(F) values are the significance of each of these terms, and a small value means that this variable is important. So, a good model should include the "***" variables and not the ones that have no stars. $\endgroup$ – gakera Nov 17 '10 at 16:41
  • 4
    $\begingroup$ @gakera: You got me wrong. If you don't understand what you're doing, you should definitely try to learn it before using it. That means, reading up on statistics and following a course. So, a good model should include the variables that are formulated in the hypothesis. If you base yourself on the "***" variables, you need a thorough course on modeling first. You obviously didn't understand my last comment. Sorry for the direct communication, comes with the guy. Nothing personal. $\endgroup$ – Joris Meys Nov 17 '10 at 17:01
  • $\begingroup$ @gakera: I updated my answer to clarify some points that are important. Mainly because you misinterpreted the part you thought you could use. $\endgroup$ – Joris Meys Nov 17 '10 at 17:09
  • $\begingroup$ I'm learning by doing, this is homework after all, nobody is going to die if I don't get this right - the fish are already dead :P Thanks for the help so far, and don't worry, this is not my first time on the internet :) $\endgroup$ – gakera Nov 17 '10 at 18:16
4
$\begingroup$

For reference, these are the values that are included in the table:
Df refers to Degrees of freedom, "the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary."

The Sum of Sq column refers to the sum of squares (or more precisely sum of squared deviations). In short this is a measure of the amount that each individual value deviates from the overall mean of those values.
RSS is the Residual Sum of Squares. These are a measure of how much the predicted value of the dependent (or output) variable varies from the true value for each data point in the set (or more colloquially: each "line" in the data table).

AIC is the Akaike information criterion which is generally regarded as "too complex to explain" but is, in short, a measure of the goodness of fit of an estimated statistical model. If you require further details, you will have to turn to dead trees with words on them (i.e., books). Or Wikipedia and the resources there.

The F value is used to perform what's called an F-test and from it is derived the Pr(F) value, which describes how likely (or Probable = Pr) that F value is. A Pr(F) value close to zero (indicated by ***) is indicative of an input variable that is in some way important to include in a good model, that is, a model that does not include it is "significantly" different than the one that does.

All of these values are, in the context of the drop1 command, calculated to compare the overall model (including all the input variables) with the model resulting from removing that one specific variable per each line in the output table.

Now, if this can be improved upon, please feel free to add to it or clarify any issues. My goal is only to clarify and provide a better "reverse lookup" reference from the output of an R command to the actual meaning of it.

$\endgroup$
  • $\begingroup$ @gakera Practical Regression and Anova using R is a good starting point for understanding linear models, and methods related to variables/model selection. As pointed by @Joris, stepwise regression is rarely the panacea. $\endgroup$ – chl Nov 17 '10 at 18:33
  • $\begingroup$ hah, thanks for adding the links @chl while maintaining my disclaimer as to why I can't post them. You must agree that I suck :D $\endgroup$ – gakera Nov 17 '10 at 18:36
  • 1
    $\begingroup$ @gakera I think you need to have more rep to add more than one link per edit -- I can understand this is not very pleasant when starting on an Q&A website. I was assuming that you would remove your last sentence yourself. On the other hand, I feel you shouldn't expect too much upvotes for providing an answer to your own question, since it is a sort of recap' (useful, though). $\endgroup$ – chl Nov 17 '10 at 19:02
  • $\begingroup$ I'm not doing this for upvotes (that's so Reddit :P ) - useful recap is exactly what I'm going for - mainly for myself but probably useful for others as well. $\endgroup$ – gakera Nov 17 '10 at 20:03
  • $\begingroup$ @gakera I'm sure this was not for getting upvotes. Most of the times, we set our own response as Community Wiki (CW), when they don't add further or contradictory information. This is a neutral way to sum up or aggregate others' responses. $\endgroup$ – chl Nov 18 '10 at 9:30

protected by whuber Aug 18 '16 at 18:08

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.