Guidelines for when you can collapse ordered categorical variables into proportion (ratio) variables?

Question

Please provide suggestions or resources that address the question of "how much is enough" data to transform a set of ordinal outcome variables to a proportion score

A single multiple choice question can have fully correct, partially correct and incorrect answers

• this yields an ordered categorical variable of possible values (0 - 1 - 2)

• when analyzed individually, this item type would need a logistic (in the case of correct/incorrect) or ordinal (polynomial) regression (e.g. clm from the ordinal package in R)

Given that individual multiple choice questions can only provide binomial or polynomial data, are there general guidelines about how many such items you need in an assessment before you can scale(?) each item and average them to create a proportion correct score?

so the idea is to do this --

max_val <- max(target_var, na.rm = TRUE)
scaled_var <- target_var/max_val 

-- for every item and average the result to create a general "proportion correct" score

Another way to ask the question -- I know that an outcome variable with a limited number of possible values is bad for linear analyses (anova, linear regression), so how "rich" does a variable need to be to be appropriate for this type of analysis?

Answer 1

This is a general answer. See http://psych.colorado.edu/~willcutt/pdfs/Rhemtulla_2012.pdf Research indicates that if you have 6-7 categories, then the results of specialized methods and continuous methods are very similar. In the paper more references can be found to studies with similar results.