1
$\begingroup$

I have the following regression outputs from a model that includes both quadratic and cubic interaction terms. I calculated the simple slopes using the simle_slope from reghelper.

x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
x4 <- rnorm(100)

y   <- x1 + x2 + x2**2 + x1*x2 + x3 + x4 + rnorm(100)
fit <- lm(y ~ x1 + x2 + I(x2^2) + I(x2^3) + x1:x2 + x1:I(x2^2) + 
    x1:I(x2^3) + x3 + x4)

summary(fit)

library(reghelper)
test2 <- simple_slopes(fit, level = list(x2 = c(0,1,2,3), confint = T))
test2$L95 <- test2$`Test Estimate`-1.96*test2$`Std. Error`
test2$U95 <- test2$`Test Estimate`+1.96*test2$`Std. Error`
test2

test2
      x1 x2 Test Estimate Std. Error t value df  Pr(>|t|)        L95      U95 Sig.
1 sstest  0        1.1733     0.1642  7.1437 92 2.085e-10  0.8514158 1.495274  ***
2 sstest  1        2.0379     0.1895 10.7562 92 < 2.2e-16  1.6665505 2.409244  ***
3 sstest  2        2.7164     0.8014  3.3896 92  0.001033  1.1456684 4.287227   **
4 sstest  3        3.2395     2.6804  1.2086 92  0.229911 -2.0140209 8.493070     

I understand simple slope can tell us whether or not a conditional effect based on a specific value of moderator is statistically different from zero.

My question is, the confidence interval (L95, U95) of the simple slope of x1 when x2 = 0,1 is not overlapped. Can I say there is a statistically significant difference in the simple slopes of x1 when x2 = (0,1)? In other words, can I tell the simple slopes of x1 when x2 = 0,1 are significantly different from each other?

$\endgroup$

1 Answer 1

1
$\begingroup$

Non-overlap of confidence intervals is too stringent for determining "statistically significant" differences between estimates. This page has a superb illustration.

If your interest is in differences between "simple-slope"* estimates, then evaluate those differences directly. For example, the emmeans package provides ways to examine all pairwise differences among a set of "simple slopes" evaluated as emtrends:

library(emmeans)
emtrends(fit,pairwise~x2,at=list(x2=c(0,1,2)),var="x1")
# $emtrends
#  x2 x1.trend    SE df lower.CL upper.CL
#   0     1.02 0.134 90    0.755     1.29
#   1     2.39 0.176 90    2.040     2.74
#   2     2.80 0.365 90    2.069     3.52
# 
# Confidence level used: 0.95 
# 
# $contrasts
#  contrast  estimate    SE df t.ratio p.value
#  x20 - x21   -1.371 0.160 90  -8.542  <.0001
#  x20 - x22   -1.776 0.417 90  -4.255  0.0002
#  x21 - x22   -0.405 0.381 90  -1.063  0.5393

The "simple slopes" at each value of x2 are indicated by x1.trend in the $emtrends output, along with standard errors and confidence levels. They differ somewhat from yours as you didn't specify a random seed, and I limited the range of x2 as there weren't any values as high as 3 in the data. (I used set.seed(101) before running your code.) Your example shows a much higher standard error for the slope when x2=2, which I also assume is because of the difference in random seeds.

The differences in slopes among the pairs of x2 values are in the $contrasts output. x20 means the situation in which x2=0, etc., so the first line for the contrast x20-x21 is the particular comparison you note in the question.


*I personally find "simple slopes" to be not so simple, so I put quotes around the phrase.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.