2
$\begingroup$

I have the following data:

  • The same 25 subjects have participated in 3 tasks, once in city A and once in city B. So for each city, I have 75 datapoints - one datapoint for each task and each subject.

If I use a one-way repeated measures anova for city B to compare task performance of the subjects, there is no significant difference (p = 0.48).

Now when I use a two-way repeated measures Anova, taking data from both cities, the main effect of city proves significant (p < 0.0001). How is this possible? Can the main effect be so low even if task is only relevant in one of the two levels of the factor city (ie task performances are only significantly different in one of the cities)?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ I don't understand the question. So the tasks are not signif different in city B. Are they signif different in city A? If yes, then cities A and B seem to be different between each other, which is what two-way anova is telling you. So what's the problem? $\endgroup$
    – amoeba
    Dec 13, 2017 at 11:12
  • $\begingroup$ It is not very clear what you are asking. A main effect of city indicates that, regardless of the task, subjects performed better in one of the two cities. I don't see how this is in contrast with the finding that task are not significantly different in city B. Are they significantly different when the rmANOVA is done only for city A? Is the interaction task:city significant in the two-way anova? $\endgroup$
    – matteo
    Dec 15, 2017 at 11:45
  • $\begingroup$ You could drastically improve the chances for a good answer by presenting a graphic depiction of your data and/or outputs of the software you're using. It's difficult (for me at least) of getting an overview with this limitted amount of information. $\endgroup$
    – COOLSerdash
    Dec 16, 2017 at 14:56

1 Answer 1

Reset to default
1
$\begingroup$

It sounds like you expected city to have little-to-no effect. In this case, you might want to use the Bayesian approach with a prior distribution on city effect size, where the prior is based on previous studies in the literature on similar environmental variables and how they effect performance on similar tasks. It would also make sense to compare your effect size with similar environmental effect sizes in the literature.

On the other hand, I am wondering if all the subjects performed the tasks in City A before City B, or vice-versa? Or was the order switched up? This would imply a difference in time and experience, rather than just location. Are these tasks on which performance can be improved with one experience, and could there be a differential in performance on the different tasks? In this case, the main effect may be experience rather than location.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.