0
$\begingroup$

I'm doing a research project for university and am trying to figure out how I should be interpreting my between-group effect in my mixed ANOVA.

My within-subjects variable is time and my between-group variable is game group. I’m testing the impact between two mobile games on a person's general well-being over time. My hypothesis is “Pokémon Go will have a greater effect on improving the player’s life satisfaction, social functioning, and depressive/anxiety symptoms when compared to players of Candy Crush”

Am I right in thinking that any significant difference in the between-group effect is irrelevant to this research question if there’s no significant effect of time or significant interaction?

Please let me know if you need any more info.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

From your question, it seems that you are running a model which looks roughly like: wellbeing~game*time+(1|person)

  • If the interaction between game and time is non significant, it means that the effect of time spent playing on well-being is similar between the 2 games. That is to say, that the dynamics of well-being along time is the same whether the people play one game or an other. However it does not mean that the effect of the 2 games on well being is similar. The dynamics of well being along time could be similar between the 2 games, but much higher for one game compared to another.

  • Still in the absence of significant interaction, the absence of a significant single effect of time (in the additive model well being~time+game+(1|person)) means that the effect of playing the game does not vary over time, i.e. the well being of the person would be similar whether they play it 5 minutes or 1 hour.

  • Despite both these features of your model, if the variable game has an effect (in well being~game+(1|person)), it means the well being is higher for people playing one game compared to the other. In the case the interaction is non significant, and time as a simple effect is not significant, it means that the well being of the gamers is constant over time for both games, and higher for one game than for an other.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you so much, really helpful stuff! So what if there's a significant difference at the baseline assessment for the game effect? Would the model still be accurately predicting that participants in one game group has higher well-being than the other due to the game they're playing? $\endgroup$
    – Liam
    Sep 23 at 18:36
  • $\begingroup$ If only "game" as a variable has a significant effect, then you should look at the results of the model well being~game+(1|person). It tells you participants in one group have higher well being than in the other. $\endgroup$
    – CaroZ
    Sep 23 at 23:45
1
$\begingroup$

You are basically correct, except that you have over-emphasized statistical significance. Your hypothesis is about the interaction, so it is the estimate of the interaction that you should concentrate on. But interactions often have low reliability (especially for variables that are are measured with considerable error, like the ones you are interested in). So, look at effect size.

Also, look at graphs. I would make line graphs with time on the x axis, your DV on the y axis, and a line for each group.

Also, if the people choose the games themselves, then that makes things trickier. You will want to avoid causal language.

And be careful of any possible ceiling or floor effects.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you, Peter. $\endgroup$
    – Liam
    Sep 23 at 21:07
  • $\begingroup$ Yeah, I had line graphs set up as you described, but there were no statistically significant interactions anywhere in my analysis. There was only a statistically significant difference between groups for 2 of my analyses, however, I ran a one-way ANOVA for the baseline scores for these variables and it was significant. This means I can't interpret the significant result from the mixed ANOVA as being due to the game right? This was for people who weren't playing the game until the study by the way - should have cleared that up beforehand. $\endgroup$
    – Liam
    Sep 23 at 21:29
  • $\begingroup$ It seems like you have a difference between players of the two games, but not in the effect of the games over time. $\endgroup$
    – Peter Flom
    Sep 23 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.