Skip to main content
Cross Validated

Return to Question

Bumped by Community user
deleted 1 character in body
Source Link
Yamper
Yamper
  • 31
  • 4

I'm currently struggling with the following problem and would like to ask you what in your opinion is the best way to deal with this problem (and maybe can recommend literature about the same topic):

I have conducted an experiment where I have exposed 25 individuals in randomized order to 8 treatments each (1 control + 7 treatments). Post-hoc tests (as well as a previous experiment) suggest that only one of the treatment induces a behavioral change that differs significantly from the control treatment (p = 0.01), all other p > 0.2.

However, when I run an anova over all 128 treatments so as to test for the effect of "treatment" , the result is nonsignificant (p = 0.11). This seems to be a false negative although I need a significant value here to justify conducting those post-hoc analyses.

The statistical method I currently apply are LMEs (R package "nlme") (data is normally distributed) with individual ID as random effect.

Do you know how to solve this problem of a false negative aside from increasing sample size?

Thanks for your help!

I'm currently struggling with the following problem and would like to ask you what in your opinion is the best way to deal with this problem (and maybe can recommend literature about the same topic):

I have conducted an experiment where I have exposed 25 individuals in randomized order to 8 treatments each (1 control + 7 treatments). Post-hoc tests (as well as a previous experiment) suggest that only one of the treatment induces a behavioral change that differs significantly from the control treatment (p = 0.01), all other p > 0.2.

However, when I run an anova over all 12 treatments so as to test for the effect of "treatment" , the result is nonsignificant (p = 0.11). This seems to be a false negative although I need a significant value here to justify conducting those post-hoc analyses.

The statistical method I currently apply are LMEs (R package "nlme") (data is normally distributed) with individual ID as random effect.

Do you know how to solve this problem of a false negative aside from increasing sample size?

Thanks for your help!

I'm currently struggling with the following problem and would like to ask you what in your opinion is the best way to deal with this problem (and maybe can recommend literature about the same topic):

I have conducted an experiment where I have exposed 25 individuals in randomized order to 8 treatments each (1 control + 7 treatments). Post-hoc tests (as well as a previous experiment) suggest that only one of the treatment induces a behavioral change that differs significantly from the control treatment (p = 0.01), all other p > 0.2.

However, when I run an anova over all 8 treatments so as to test for the effect of "treatment" , the result is nonsignificant (p = 0.11). This seems to be a false negative although I need a significant value here to justify conducting those post-hoc analyses.

The statistical method I currently apply are LMEs (R package "nlme") (data is normally distributed) with individual ID as random effect.

Do you know how to solve this problem of a false negative aside from increasing sample size?

Thanks for your help!

Source Link
Yamper
Yamper
  • 31
  • 4

ANOVA - False negatives due to many nonsignificant treatments where one treatment is significant

I'm currently struggling with the following problem and would like to ask you what in your opinion is the best way to deal with this problem (and maybe can recommend literature about the same topic):

I have conducted an experiment where I have exposed 25 individuals in randomized order to 8 treatments each (1 control + 7 treatments). Post-hoc tests (as well as a previous experiment) suggest that only one of the treatment induces a behavioral change that differs significantly from the control treatment (p = 0.01), all other p > 0.2.

However, when I run an anova over all 12 treatments so as to test for the effect of "treatment" , the result is nonsignificant (p = 0.11). This seems to be a false negative although I need a significant value here to justify conducting those post-hoc analyses.

The statistical method I currently apply are LMEs (R package "nlme") (data is normally distributed) with individual ID as random effect.

Do you know how to solve this problem of a false negative aside from increasing sample size?

Thanks for your help!