0
$\begingroup$

I am using a mixed effects model to analyse data, but am unsure whether I am committing any violations due to the nature of the data I have.

My data comes from a game whereby people have to identify an object presented on a screen as fast as possible. I won't go into much detail, but there are two methods by which they can identify the object (e.g., method A or method B), and they are free to choose which method to use every time they replay the game. There are also about a hundred possible objects which can be presented, some of which occur much more frequently than others. The downside to the data is that players can play the game for as long as they want, using either method, and the objects they encounter varies from player to player. I am looking to see whether method A is more efficient (aka faster) for object identification than method B, and whether this difference is stronger when rarer objects are presented.

Currently, using R, I am using the following models, and then using an anova between .model and .null to determine differences (rt stands for time it takes to identify the object, par stands for player ID, freq is a percent indicating how common it is for a certain object to appear on any game replay):

search.model = lmer(rt ~ method + freq + (1|par) + (1+method|objectID),
                    data=search, REML=FALSE)

search.null = lmer(rt ~ freq + (1|par) + (1+method|objectID),
                   data=search, REML=FALSE)

For interaction testing, above .model becomes .null and I use the following as the new .model: 

search.model = lmer(rt ~ method * freq + (1|par) + (1+method|objectID),
                    data=search, REML=FALSE)

Is this the correct method to use in analysing this data? One major concern is whether it is okay to include data whereby some players have alternated using different methods, while others have stuck to a single method. In addition, whether it is okay to include data that comes from objects only seen by a handful of people, or have only been used in the game via one of the two methods. For instance, there are some objects which have been seen by maybe 20 people out of all the players (full data contains over 100000 people), and of those 20 people, all only used method B to identify the object. Some objects may only have data from a single person because it was so rare to encounter. Should I be removing certain objects or data before using this analysis?

Improve this question
$\endgroup$
3
  • $\begingroup$ Is it correct that your first seach.model and your second search.model are the same? $\endgroup$
    – Pieter
    Commented Jan 8, 2017 at 21:56
  • $\begingroup$ It looks like this problem as a lot of different uncertainties to assess on different levels. E.g. uncertainty caused by the player (not many games), uncertainty about objects (not recognized/seen often) and uncertainty about the method (less data about one of the methods). This is difficult to model, but if you really want to get to the bottom of this you can use Bayesian modelling. Popular tools are Stan and PyMC. $\endgroup$
    – Pieter
    Commented Jan 8, 2017 at 22:03
  • $\begingroup$ @Pieter The first and second 'search.model' are slightly different, one uses a * for interacting factors. I am assuming based on what you said that using a mixed effects model would not adequately account for all the uncertainty for all of the levels you pointed out, and I would have to resort to Bayesian modelling to test the hypothesis? $\endgroup$
    – Paul Scotti
    Commented Jan 10, 2017 at 14:40

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.