3
$\begingroup$

Let's say I run the following regression specification:

lm(outcome ~ treatment) 

Where treatment is a factor variable that can take four values A, B, C, or D . I suspect that there is this predictor, X that moderates the relationship between the treatment and the outcome. However, I don't really care about comparisons across treatment groups -- what I want to be able to say is that, within a given treatment (e.g. A), people with high values of X have better outcomes then individuals with low values of X.

What comes to mind is some kind of interaction effect like:

lm(outcome ~ treatment*X)

But that will only tell me how the treatment varies at different levels of X, in comparison to some left out reference group (rather than in comparison to the same treatment but low values of X).

What is the right way of going about doing this?


Edit: Ran the nested specification suggested by @cdtip and it resulted in the following output:

             Estimate Std. Error  t value  Pr(>|t|) CI Lower CI Upper  DF
(Intercept)   -1.33113   4.96200 -0.26826 0.7889642 -11.1572  8.49498 118
treatmentC    -5.90437   6.94422 -0.85026 0.3969036 -19.6558  7.84706 118
treatmentB    24.92750   9.27155  2.68860 0.0082136   6.5673 43.28769 118
treatmentA    7.26947    5.73785  1.26693 0.2076739  -4.0930 18.63197 118
treatmentD:X  0.65245    1.27021  0.51366 0.6084530  -1.8629  3.16782 118
treatmentC:X  5.15284    1.64776  3.12717 0.0022231   1.8898  8.41586 118
treatmentB:X -2.65812    1.08513 -2.44959 0.0157709  -4.8070 -0.50927 118
treatmentA:X -0.69822    0.67031 -1.04163 0.2997141  -2.0256  0.62919 118

I'm somewhat unclear how to interpret this. The coefficient on treatmentC:X is 5.15 and significant. But it seems like it's significant relative to the left out condition which is treatmentD, rather than making a within treatment comparison. How am I supposed to interpret these results?

It seems like the / operator is just shorthand for:

lm(outcome ~ treatment + treatment:x)

(so removing the main effect of X).

I have found references to "subgroup analysis" and "heterogeneous treatment effect", but these seem to refer to a singular treatment condition, in which the size of the effect varies by some moderator.


$\endgroup$

This question has an open bounty worth +50 reputation from Parseltongue ending in 4 days.

This question has not received enough attention.

I'm not sure if ANCOVA is the right approach here and, if it is, I'm not sure how to interpret the results from the above linear regression

2
$\begingroup$

It seems like what you want is to avoid reference cell coding. You want a slope and intercept for the effect of X on outcome at each level of treatment. You need to modify @cdtip's code to exclude an intercept, and you should get what you need:

lm(outcome ~ treatment/X - 1)

Now, each coefficient treatmentA:X, etc., is the effect of X on the outcome for those in treatment A. By removing the intercept, you get a slope for each treatment level. To test whether there is an overall moderating effect of X, you can use anova() to compare the above model with a model that excludes X (but is otherwise the same); a significant p-value indicates the presence of moderation.


In response to question edits:

This parameterization essentially creates 4 separate regression models of outcome on X, one for each treatment group, but estimated together in a single model. The "main effects" are the intercepts of each treatment group-specific regression. They are not treatment effects. Treatment effects would be comparisons between these intercepts, which represent the difference in the treatment group means when X is 0. The interaction terms are interpreted simply as the slopes of each treatment group-specific regression. So from this coefficient table, you can describe the slope and intercept of the regression of outcome on X in each treatment group. This parameterization does not allow you to make any comparisons between treatment groups, but it is possible to reparameterize to get the contrasts you want, if any.

$\endgroup$
  • $\begingroup$ Just to be clear. With this specification, if I see a significant effect on an interaction term that implies that, within that treatment, the moderator was significantly predictive of the outcome? $\endgroup$ – Parseltongue 2 days ago
  • $\begingroup$ Also do you have a sense of how to test the overall effect of X with cluster-robust standard errors? I'm using lm_robust for clustering, and it doesn't look like anova is compatible $\endgroup$ – Parseltongue 2 days ago
  • 1
    $\begingroup$ Your clarification statement is correct. If you are willing to migrate to the survey package, the functions there are compatible with anova(). $\endgroup$ – Noah 2 days ago
  • $\begingroup$ Thank you so much, Noah! Do you mind taking one last look at my edits? I'm having trouble making sense of the main effects. Are the main effects just uninterpretable in light of the interactions? Can I interpret the interactions without reference to the main effects? If not, is there a way to make the interactions interpretable on their own? $\endgroup$ – Parseltongue 2 days ago
  • 1
    $\begingroup$ I edited my answer. $\endgroup$ – Noah 2 days ago
4
$\begingroup$

If I'm understanding right, you may be wanting to do an analysis of covariance (ANCOVA). There is a helpful tutorial here (http://r-eco-evo.blogspot.com/2011/08/comparing-two-regression-slopes-by.html).

Specifically, read down to the bottom where comparing slopes and intercepts is discussed. This was achieved through a slightly different version of your code.

lm(outcome ~ treatment/x) # assuming x is a continuous predictor

Because of the nested design, you are essentially only comparing the values of x within their respective treatments.

Hope that helps!

$\endgroup$
  • $\begingroup$ Thanks so much! This certainly seems like the right direction. Could you take a look at my edit? I'm not sure how to make sense of the resulting values. $\endgroup$ – Parseltongue Aug 13 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.