1
$\begingroup$

I realize that this may be answered somewhere but I cannot find it, either on Cross Validated or with Google. I can find numerous examples of how to use multiple regression with R, and I have done that. Then I used summary to view the output, like so:

fit <- lm(target_output ~ weekday + day + month + year + max_temp + min_temp + precip + tmax_trailing_7 + tmax_trailing_30 + factor(notes) + weekday:day + day:month + factor(notes):day + factor(notes):weekday, data=data)
summary(fit)

The "summary" works fine for a small model, but given the large number of factors I have, I am looking for an example on how to, well, summarize the summary. In particular, I would like a list of all of the factors and interactions which are marked with "Signif codes" of 2 or 3 stars.

Ideally, I would also like the output to be something which I can then manipulate in the script. For example, to dump the factors which are not significant at the "2 asterisk" level, and run the regression again.

If I am just missing the awesome R mulreg tutorial which shows how to do this, I apologize, but if you could point me at it that would be awesome, because I am having trouble finding it. The tutorial examples I can find all have small numbers of input variables, and don't seem to cover pruning out the ones that are less significant. Thanks for your time.

$\endgroup$
3
  • 1
    $\begingroup$ This is not multivariate regression (which, by definition, involves more than one response per observation): it is multiple regression. $\endgroup$
    – whuber
    Commented Jan 1, 2018 at 0:11
  • $\begingroup$ @whuber Agreed, but I have seen multivariate syllabi that consider multiple regression to be multivariate $\endgroup$
    – Mark White
    Commented Jan 1, 2018 at 0:21
  • $\begingroup$ I will keep the distinction between multivariate and multiple regression in mind in the future. I normally have just read/said "mulreg", and now I wonder if that was intentionally vague in order to avoid having to remember which is which. : ) $\endgroup$
    – rossdavidh
    Commented Jan 2, 2018 at 0:46

1 Answer 1

9
$\begingroup$

I will show you how to do what you want to do but also give some reasons for why you may or may not want to do what you're interested in in the way that you describe.

I would like a list of all of the factors and interactions which are marked with "Signif codes" of 2 or 3 stars.

You can do this by turning your summary coefficients table into a matrix and then sorting out any rows where the p-values are less than some value, say .05. Here's an example. First, I'll generate data:

set.seed(1839)
# create four independent variables
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100) 
x4 <- rnorm(100)
# make y predicted by only two of them, plus some error
y <- x1 + x2 + rnorm(100)

Then you can extract the coefficients table as a matrix. Note that the fourth column are the p-values, so you can just ask for rows where p-values are less than .05:

model <- lm(y ~ x1 * x2 * x3 * x4)
coefficients <- round(summary(model)$coef, 4)
coefficients[coefficients[, 4] < .05, ]

Which returns:

   Estimate Std. Error t value Pr(>|t|)
x1   0.8575     0.1145  7.4900   0.0000
x2   0.9755     0.1162  8.3929   0.0000
x4   0.3679     0.1199  3.0671   0.0029

The three significant coefficients out of all the main effects, two-way, three-way, and four-way interactions I specified.

The resulting coefficients table brings out an issue with doing this: Type I error. Even though I coded it so that x4 was not related to y, the results still tell me that it does. When you are doing tons of tests, you're going to get some that are significant just by chance, so you might consider lowering your alpha level. But that type of indexing by row is how you could get that table.

For example, to dump the factors which are not significant at the "2 asterisk" level, and run the regression again.

One could code this by taking the rownames of the output above, then pasting all of them together using paste(rownames(output_above), "+"), then pasting that after "y ~ ", saving as.formula, then putting it back into an lm object...

HOWEVER, what you are doing is variable-selection, and this choosing based off of an arbitrary threshold is generally not suggested. Instead, you could look at the literature on variable selection or regularization. In particular, there is a thing called a LASSO, which adds a penalty term to your regression that will have the effect of eliminating variables that are not very informative. There are many good tutorials on this using R. A classic book you can use is Introduction to Statistical Learning with Applications in R, which can be accessed on the authors' website here. In particular, you should look at Section 6.2, especially 6.2.2, which will give you the effect that you are looking for. There are also exercises showing you how to implement the LASSO regression in R in Section 6.8.

$\endgroup$
1
  • $\begingroup$ Excellent, both the "here is how you could do that" part and the "here's where you read on what you should do instead" part. I will read up on chapter 6 in the reference you cite. $\endgroup$
    – rossdavidh
    Commented Jan 2, 2018 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.