5
$\begingroup$

I have the results of a 2*5 mixed ANOVA drug trial, where there’s a non significant interaction but main effects of both time and group. Scores were taken before and after trial (time). 4 drugs were tested and 1 control.

How do I interpret this non-interaction and main effects? Does the interaction mean that there was no difference between groups on the time measure?

Also, is the control group throwing off the whole study? If there's no interaction does a higher control mean no drugs have any effect? graph

$\endgroup$
15
  • $\begingroup$ Welcome to CV! Could you plot your score as a function of time and group (either from raw data or from model results), and edit the plot and your ANOVA output into your question? $\endgroup$
    – Sointu
    Feb 17 at 7:45
  • 1
    $\begingroup$ Yes, because when you choose to use mixed ANOVA to investigate a drug effect in a pre-post design, the interaction between time and group is your critical effect, not the main effects. $\endgroup$
    – Sointu
    Feb 17 at 8:35
  • 1
    $\begingroup$ But if you have a mixed ANOVA with time included, the main effects of group from that model are overall effects (averaged over both time points) so they don't tell you whether changes over time differ between groups, which is what would tell you whether the drug worked. $\endgroup$
    – Sointu
    Feb 17 at 8:40
  • 1
    $\begingroup$ @Sointu "In mixed ANOVA you are using each participant as their own control, so if you randomized your participants..." I agree with this but I think mixed ANOVA makes sense for cross over trials only. That is, when the same patients receives more than one treatment during the experiment. Then that raises the question of how the participants are randomized. The OP doesn't explicitly mention that the trial is a cross over, so I'm curious why they fitted a mixed ANOVA to begin with. $\endgroup$
    – dipetkov
    Feb 17 at 16:40
  • 2
    $\begingroup$ Please do not vandalize your question. When you posted on SE, you gave up ownership of the content under CC BY-SA 4.0. If there are no answers, you may delete your own question (see here ): just click the faint gray 'delete' at lower left (your account needs to be registered for this). Otherwise, the thread will remain according to SE's rules. $\endgroup$ Feb 18 at 16:38

1 Answer 1

4
$\begingroup$

Your study objective is to assess effects of four drugs on a health outcome with reference to the effect of a control substance on a health outcome. It is not about influence of time on drug or drug on time. Both drug and time are predictors not the response. An effect is of a predictor on the response.

The main effect of time is negative, shown by the negative slopes of lines from Time 1 to Time 2. It means the health outcome decreased over time no matter which of the five medications to use. The main effects of four treatment-group indicators measure the difference between a treatment group and the control group at Time 1 before using any medication. They are all negative if the control group had the highest score to begin with.

If group assignment is completely random, we expect that the main effects of treatment group to be zero. Significant main effects of treatment group means systematic differences at the starting line, nonrandom group assignment, and a sample-selection bias that undermines causal inference. For sample-selection remedy in causal inference, see my answer at Seeking Assistance in Evaluating My Research Plan for Regression Analysis. You can also use measurements at Time 1 as a predictor instead of a response, to adjust for baseline differences. See Senn, S. (2006). Change from baseline and analysis of covariance revisited. Statistics in Medicine, 25(24), 4334–4344. https://doi.org/10.1002/sim.2682.

You are using the difference-in-difference design, except that you do it four times. For the interpretation of interaction effects in this design, see my answer at Frank Harrell's interpretation of interaction in regression results. In your case, nonsignificant interaction between time and treatment means that the effects of 4 tested drugs on the health outcome were no different from that of the control substance, shown by the parallelism among the five downward lines. It does not mean that the tested drugs have no effects but that the tested drugs have no additional effects that the control substance cannot reach. This is a collective F test. To compare each drug to the control, we need to use t tests on individual coefficients of the four interaction terms in a regression table.

If you are conducting noninferiority study to demonstrate that new drugs are no worse than the current standard, nonsignificant interaction is a desirable result. Note that nonsignificant difference does not necessarily mean noninferiority or equivalence. For the definition and determination, see U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, & Center for Biologics Evaluation and Research. (2016). Non-inferiority clinical trials to establish effectiveness: Guidance for industry. https://www.fda.gov/media/78504/download

$\endgroup$
1
  • $\begingroup$ With five groups, you will have four group indicators. The treatment effect in causal inference has a specific definition, distinct from efficacy of a drug. See my answer. If all four new drugs have the same efficacy as the control, the treatment effect is zero. $\endgroup$
    – DrJerryTAO
    Feb 17 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.