I was conducting a 'recall of negative words' memory experiment with a two-way between-group ANOVA and got no significant main effects and no significant interaction! How can I go about explaining this in the Discussion section? Do I just state that it seems that my 2 between subjects factors have no influence over memory? Or do I try to fault the sample size [N=67].

All help will be greatly appreciated!!!

PS: Do I write: the lack of significant main effects allowed for rejection of all experimental hypotheses? it that correct?

  • $\begingroup$ Beside possible lack of power (but power/sample size should have been determined beforehand), what else would you like to say? If you cannot highlight an effect of your factors on this sample, that's all what you can say, IMO (keeping in mind that you can't accept the null). $\endgroup$ – chl Nov 1 '12 at 12:12
  • $\begingroup$ I am just confused about how to explain that in Discussion. Do I just say that affective memory is a complex topic and there is no consensus anyway? Or do I come up with some complex possible explanations like that my sample was too small. Is 67 too small for a 2 way between subjects anova? If yes, do you know an article I could read which explains this so I can have a citation for this? $\endgroup$ – Anna Nov 1 '12 at 12:15
  • $\begingroup$ Describe your effects (means, SDs) and provide measure of effect sizes, discuss possible power issues--there's nothing wrong with negative findings if they are correctly discussed with reference to the literature. G Power 3 provides way to assess statistical power, but as you may have guessed this has to be done before you started your study. $\endgroup$ – chl Nov 1 '12 at 12:29
  • $\begingroup$ Can I ask you one more thing? I have no significant interaction between two variables but the lines on the plot DO cross. Is that even possible? $\endgroup$ – Anna Nov 1 '12 at 13:00
  • $\begingroup$ Yes, it is possible: it is a good visual check but statistical significance depends on many other factors. $\endgroup$ – chl Nov 1 '12 at 23:27

There are at least two schools of thought here.

First school: Since nothing was significant, there's nothing statistical you can say except that you failed to reject the null hypothesis. You could, however, then give some explanation of why you couldn't reject it. This explanation, though, would have to be substantive rather than statistical. E.g., there could be omitted variables, the sample could be unusual, etc.

Second school: Despite the lack of significance, you can still comment on the effect sizes of the model. You have to preface this by some phrase such as "Although the results were not statistically significant, they indicate that ...." and then go into what you found.

I am more sympathetic to the second school, myself, but in fields like psychology (which it seems is your field) that may be a difficult thing to 'sell'.

One thing you should not say is that there is "no effect" - that comes close to "accepting the null" which is not right. You should say "no statistically significant effect".

  • $\begingroup$ And would I actually write: the lack of significant effect allowed researchers to reject the experimental hypothesis nr 1? etc Is that correct? $\endgroup$ – Anna Nov 1 '12 at 12:32
  • $\begingroup$ Also, I was thinking that because we tested at different times of day that could have confounded the results. would it be appropriate to suggest this? My other question: do you think a N=67 sample was too small for a 2*2 between subjects anova? $\endgroup$ – Anna Nov 1 '12 at 12:34
  • $\begingroup$ Times of day could have an effect, I suppose. Whether 67 is enough people depends on the effect size; you should have done a power analysis prior to collecting the data. But it's not a ridiculously small number. $\endgroup$ – Peter Flom - Reinstate Monica Nov 1 '12 at 12:36
  • $\begingroup$ One more question: when I do the write up - do I write in the introduction about the variables other researchers mention as important even if I don't use them? There is this issue that apparently use cannot mention anything is discussion if you haven't mentioned it in introduction. But does that hold for unexpected results? $\endgroup$ – Anna Nov 1 '12 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.