0
$\begingroup$

I would like to compute R-squared change for the interaction/moderation term in a multiple regression model, along with the corresponding F- and p-values. Previously, I have worked with the modprobe macro by A. Hayes, which can produce this for SPSS. As I am transitioning to R now, I am trying to find a function/package or a custom-made script in R that does this. In case it helps, my current interaction model looks like this:

m1 <- lm(all_ART~Neuroticism*Agreeableness+Attentional.Control, 
         data=stp2_sub2, na.action=na.omit)

Any pointers on how to compute these values (i.e., $R^2$(interaction), F-value(interaction) and p-value(interaction)) for the interaction term in R would be much appreciated!

$\endgroup$
  • $\begingroup$ Asking for code / packages is off topic here. $\endgroup$ – gung - Reinstate Monica Jun 30 '15 at 13:10
  • 1
    $\begingroup$ @gung Would an answer that provides formulas for $R^2$, $F$, and a $p-value$ be off-topic? (I suspect such an answer exists in some other thread--perhaps we could locate it and redirect this question there.) $\endgroup$ – whuber Jun 30 '15 at 14:36
  • $\begingroup$ @whuber, if the OP were asking for that, it would be on topic. Note that, in my answer below, I link to an answer that provides one version of the formula. $\endgroup$ – gung - Reinstate Monica Jun 30 '15 at 15:02
  • $\begingroup$ @ging It's not clear to me that asking for code itself is automatically off-topic. The paragraph at the on-topic help center includes "if it needs statistical expertise to understand or answer, ask it here". I know you've expressed a dislike of that phrase, but as I read it, that's the situation we presently work under. I'd prefer to avoid suggesting to people that the actual situation is quite so black and white as that first short sentence. (That's not to state a position on this particular post, only on the general principles involved.) $\endgroup$ – Glen_b -Reinstate Monica Jun 30 '15 at 23:53
3
$\begingroup$

$R^2$ change and $F$-change (etc.) are just some of the many names for a nested model test. You can find some information on it and one version of the formula in my answer here: Testing for moderation with continuous vs. categorical moderators. In R (or any other software), you fit a full model with all variables, and a reduced model without the variables you want to test. (If you drop more than one variable, they are tested as a set, not individually, i.e., you get a single p-value for all of them.) Then you perform a nested model test.

m1.full = lm(all_ART~Neuroticism*Agreeableness+Attentional.Control, 
         data=stp2_sub2, na.action=na.omit)
summary(m1.full)  # this will give you the F value for the full model, if you want it
m1.red  = lm(all_ART~Neuroticism+Agreeableness+Attentional.Control, 
         data=stp2_sub2, na.action=na.omit)  # I switched * to +
summary(m1.red)  # this will give you the F value for the reduced model, if you want it
anova(m1.red, m1.full) # this performs the nested model test
$\endgroup$
  • $\begingroup$ Thank you so much for your comments and apologies for potentially misposting! Strangely I knew of the use of anova() to compare regression models, but hadn't, in my mind, made the connection to doing the same for an interaction model (I somehow assumed a more complex solution was needed for the "change measures"). So, it turns out that my question was indeed more about statistics than R itself. $\endgroup$ – Tiberius Jun 30 '15 at 22:23
  • $\begingroup$ Btw, I just noticed that the p-value for the interaction (model m5, see below) is identical to the p-value resulting from anova(m5, m6). The former is 0.0138 and the latter, as can be seen in my answer below, 0.01383. The same pattern appears for another similar regression model I am running (both p-values are the same). Is this to be expected? $\endgroup$ – Tiberius Jul 1 '15 at 16:41
  • $\begingroup$ @Marcel, yes. If the variable has 1 df, then the F-change test will be the same as the t-test. If you have a categorical moderator with k>2 levels, then you would have 2 t-tests with standard output (that you should ignore) & the single F-test will differ. It may help to read my answer here. $\endgroup$ – gung - Reinstate Monica Jul 1 '15 at 16:55
  • $\begingroup$ Thank you very much. That clarifies it. Last question, do you agree that the change values in my answer below are as stated in the conclusion (F-change = 6.5014, p-change = 0.01383 and R-sqd-change = 0.1095803). In particular, I still feel a bit uncertain about whether I correctly computed the R-sqd-change value. Any guidance would be much appreciated. $\endgroup$ – Tiberius Jul 1 '15 at 17:32
  • $\begingroup$ @Marcel, you may want to calculate the semi-partial r-squared for the interaction term, see here. $\endgroup$ – gung - Reinstate Monica Jul 1 '15 at 18:12
0
$\begingroup$

I am posting below how I have now resolved this:

Linear models

m5<-lm(data=stp2_sub2, all_ART~Neuroticism*Agreeableness+Attentional.Control, na.action=na.omit)

m6<-lm(data=stp2_sub2, all_ART~Neuroticism+Agreeableness+Attentional.Control, na.action=na.omit)

Computing F-change and p-change

anova(m5, m6)

Analysis of Variance Table

Model 1: all_ART ~ Neuroticism * Agreeableness + Attentional.Control Model 2: all_ART ~ Neuroticism + Agreeableness + Attentional.Control

Res.Df RSS Df Sum of Sq F Pr(>F)
1 51 69011
2 52 77809 -1 -8797.5 6.5014 0.01383 *


Signif. codes: 0 ‘’ 0.001 ‘’ 0.01 ‘’ 0.05 ‘.’ 0.1 ‘ ’ 1

Computing R-sqd-change

rs1<-summary(m5)$r.squared

rs1

[1] 0.140404

rs2<-summary(m6)$r.squared

rs2

[1] 0.03082376

rs.change<-rs1-rs2 # provides R-sqd-change

rs.change

[1] 0.1095803

Conclusion

So, in this particular case: F-change = 6.5014, p-change = 0.01383 and R-sqd-change = 0.1095803. Would be great if you could confirm, if I have understood this correctly.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.