So, I have some data from video game playtests, where players were allowed to play a game at home for a week, and were asked to fill out a daily survey. In particular, they were asked to rate a number of things on a 5-point Likert scale, such as how much fun they had that day, and how difficult they found the game that day. Here's an example of a data table for the fun rating (PID = Player ID):

PID 1   2   3   4   5   6   7   8   9
p02 4       5       3   3   4   3   
p03 5   4   5   5   4       4   5   
p05 5   4           3       1   1   
p06 5   5               5   5       5
p07 4   5   3   2           3       
p09 5   5   4   5   3   2   1   1   2
p11 4   3   3   4   3   3       3   
p12 4   4   4   4   4   4   3   4   
p13 3       3       2       4       
p14 5   5   5   5   5   5   5   5   

As you can see, players did not fill out the survey every day, leading to a somewhat sparse data set.

What we're looking to do is find a statistically sound method for finding the point where the mean fun rating drops off, so that we can use it as a metric to compare each game to others (i.e. "The fun rating for this game dropped off after day 4, whereas the Fun rating for other games dropped off after 5.4 days on average"). I was told by someone who knows a fair bit about stats to perhaps run a change point analysis on each player, and then run a survival analysis using the resultant change points. However, he admitted that he has very little experience with this type of analysis, and suggested that I post this question here.

So, my main questions are: (a) How should I handle the missing data points, and (b) how should I conduct the analysis?

Any help would be greatly appreciated.


My recommendation would be to avoid imputing values. Imputing regressors is one thing, but imputing the outcome can impart a large amount of bias. You seem to have more than enough data to study how the outcome changes over time.

As for the method, I wonder if it would be better to study how the outcome changes over time rather than posit that the outcome is constant or some period and then abruptly changes. A mixed effects ordinal logistic regression may be able to handle this data.

Can you post the data as a tidy dataset? Columns should be PID, Day, Rating.

| cite | improve this answer | |

You can use standard changepoint analysis techniques on data with missing points but you just have to be careful about what you infer when the changepoints are between missing values.

As an example consider p02. If an algorithm returned a changepoint at day 3 then this doesn't mean that the change is at day 3, instead it means that the change is somewhere after the data point on day 3 was collected and before the data point on day 5.

You will want to be careful as you only have integer values so you shouldn't really use a Normal distribution for your data - especially as you have so few data points. You might be best using a multinomial distribution as you have 5 outcomes.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.