Skip to main content
deleted 33 characters in body
Source Link
Sointu
  • 2.8k
  • 5
  • 13

I have no great insight, but since nobody else has answered I figured I'd say something. So it sounds like you have two issues here: you want to find out the treatment effect but you think baseline may be very strongly related to post-treatment outcome, possibly strong enough to "outshadow" the treatment effect?

But if it is indeed so that the outcome you are studying is strongly dependent on person characteristicsvery stable, so much so that intervention effects - or at least the one you have used - are negligible, there's really not much to do statistically. You'd need to plan a different kind of intervention, or maybe conclude that this outcome is not amendable to interventions?

In practice, in my understanding, if your randomization was successful (and your samples are large enough), and it sounds like it was (you say the control and intervention groups were similar), the above issue will be taken into account by any suitable modeling approach, such as ANCOVA, RM-ANOVA, multilevel regression or GEE. If there is very little treatment-related change, then your time*intervention effect will have a high standard errorbe weak and it's likely to be non-significant. And this will reflect the way things are in the world (or at least in your data).

So, in my opinion, you can model your data using any of the modelling approaches you list that are suitable for pre-post testing (I like multilevel regression with random intercept, but GEE or a suitable ANOVA will do) and see the treatment effect you get. Then, you could visualize the change for each participant from baseline to post-treatment by treatment group. This visualization is likely to tell you a lot about the possible issue of the within-person stability of your outcome.

A detail: As you noticed, lmer won't estimate random slopes when you have only 2 observations per cluster. nlme will.

I have no great insight, but since nobody else has answered I figured I'd say something. So it sounds like you have two issues here: you want to find out the treatment effect but you think baseline may be very strongly related to post-treatment outcome, possibly strong enough to "outshadow" the treatment effect?

But if it is indeed so that the outcome you are studying is strongly dependent on person characteristics, so much so that intervention effects - or at least the one you have used - are negligible, there's really not much to do statistically. You'd need to plan a different kind of intervention, or maybe conclude that this outcome is not amendable to interventions?

In practice, in my understanding, if your randomization was successful (and your samples are large enough), and it sounds like it was (you say the control and intervention groups were similar), the above issue will be taken into account by any suitable modeling approach, such as ANCOVA, RM-ANOVA, multilevel regression or GEE. If there is very little treatment-related change, then your time*intervention effect will have a high standard error and it's likely to be non-significant. And this will reflect the way things are in the world (or at least in your data).

So, in my opinion, you can model your data using any of the modelling approaches you list that are suitable for pre-post testing (I like multilevel regression with random intercept, but GEE or a suitable ANOVA will do) and see the treatment effect you get. Then, you could visualize the change for each participant from baseline to post-treatment by treatment group. This visualization is likely to tell you a lot about the possible issue of the within-person stability of your outcome.

A detail: As you noticed, lmer won't estimate random slopes when you have only 2 observations per cluster. nlme will.

I have no great insight, but since nobody else has answered I figured I'd say something. So it sounds like you have two issues here: you want to find out the treatment effect but you think baseline may be very strongly related to post-treatment outcome, possibly strong enough to "outshadow" the treatment effect?

But if it is indeed so that the outcome you are studying is very stable, so much so that intervention effects - or at least the one you have used - are negligible, there's really not much to do statistically. You'd need to plan a different kind of intervention, or maybe conclude that this outcome is not amendable to interventions?

In practice, in my understanding, if your randomization was successful (and your samples are large enough), and it sounds like it was (you say the control and intervention groups were similar), the above issue will be taken into account by any suitable modeling approach, such as ANCOVA, RM-ANOVA, multilevel regression or GEE. If there is very little treatment-related change, then your time*intervention effect will be weak and likely non-significant. And this will reflect the way things are in the world (or at least in your data).

So, in my opinion, you can model your data using any of the modelling approaches you list that are suitable for pre-post testing (I like multilevel regression with random intercept, but GEE or a suitable ANOVA will do) and see the treatment effect you get. Then, you could visualize the change for each participant from baseline to post-treatment by treatment group. This visualization is likely to tell you a lot about the possible issue of the within-person stability of your outcome.

A detail: As you noticed, lmer won't estimate random slopes when you have only 2 observations per cluster. nlme will.

Source Link
Sointu
  • 2.8k
  • 5
  • 13

I have no great insight, but since nobody else has answered I figured I'd say something. So it sounds like you have two issues here: you want to find out the treatment effect but you think baseline may be very strongly related to post-treatment outcome, possibly strong enough to "outshadow" the treatment effect?

But if it is indeed so that the outcome you are studying is strongly dependent on person characteristics, so much so that intervention effects - or at least the one you have used - are negligible, there's really not much to do statistically. You'd need to plan a different kind of intervention, or maybe conclude that this outcome is not amendable to interventions?

In practice, in my understanding, if your randomization was successful (and your samples are large enough), and it sounds like it was (you say the control and intervention groups were similar), the above issue will be taken into account by any suitable modeling approach, such as ANCOVA, RM-ANOVA, multilevel regression or GEE. If there is very little treatment-related change, then your time*intervention effect will have a high standard error and it's likely to be non-significant. And this will reflect the way things are in the world (or at least in your data).

So, in my opinion, you can model your data using any of the modelling approaches you list that are suitable for pre-post testing (I like multilevel regression with random intercept, but GEE or a suitable ANOVA will do) and see the treatment effect you get. Then, you could visualize the change for each participant from baseline to post-treatment by treatment group. This visualization is likely to tell you a lot about the possible issue of the within-person stability of your outcome.

A detail: As you noticed, lmer won't estimate random slopes when you have only 2 observations per cluster. nlme will.