1
$\begingroup$

We've conducted a quasi-experiment where an intervention was available for students to use in four semesters and was not available in other four semesters in a large college-level course. Students started each semester by taking a pre-test and they ended with a post-test.

We need to analyze the effect of the intervention on the correlation between the pre-test and post-test. Obviously, the pre-test and post-test are correlated. If the intervention had no effect on students' learning, the correlation between pre-test and post-test in semesters with the intervention should not be significantly different from the correlation in semesters without the intervention. That's the main reason we need to use emtrends from the emmeans package in R.

In our regression model, we need to consider the fixed effects of each semester, but at the end of the day, we should estimate the effect of the intervention, not each semester. So, we use add_grouping to group semesters with the intervention under "Treatment" and those without the intervention under "Control" groups.

The main issue is that add_grouping returns an emmGrid object, but emtrends does not accept an emmGrid object. Is there any other way we can group the effects and use emtrends? Or is there any better method of analyzing the effect of intervention on the correlation between pre-test and post-test?

Here is our R code:

m1 <- lm(post_test ~ semester * pre_test + GPA + gender + ethnicity + 
    academicLevel + termCreditsGPA + parentsMaxEducation + program_Cat + 
    PubMathScore + honorsPro + international, data = d1)
m1_Grid <- ref_grid(m1)
conditions_Fact <- factor(c("Control", "Control", "Treatment", "Treatment", "Control", "Control", "Treatment", "Treatment"), levels = c("Control", "Treatment"))
m1_Grid <- add_grouping(m1_Grid, "Condition", "semester", conditions_Fact)
m1_emtrends <- emtrends(m1_Grid, revpairwise ~ Condition, var = "pre_test")
summary(m1_emtrends, adjust = "tukey", infer = TRUE, type = "response")
$\endgroup$

1 Answer 1

1
$\begingroup$

Do the trends first (keeping the nested factor), and do the grouping later:

tmp <- emtrends(m1, "semester", var = "pre_test")
conditions_Fact <- factor(c("Control", "Control", "Treatment", "Treatment", "Control", "Control", "Treatment", "Treatment"), levels = c("Control", "Treatment"))
m1_emtrends <- add_grouping(tmp, "Condition", "semester", conditions_Fact)
emm <- emmeans(m1_emtrends, "Condition")
contrast(emm, "revpairwise")

The reason that emtrends() cannot be used with an emmGrid object is because, in order to estimate trends, we need to use the model with slightly different values of var in order to construct a difference quotient.

$\endgroup$
1
  • 1
    $\begingroup$ PS - in this example, there is no need for type = "response" since there is no response transformation or link function. And adjust = "tukey" is the default for pairwise comparisons. $\endgroup$
    – Russ Lenth
    Commented Nov 12, 2020 at 16:33

Not the answer you're looking for? Browse other questions tagged or ask your own question.