My general question is when should one use OLS and when binomial regression when the outcome is count data with a fixed upper limit. When the upper limit is large (like 1000) and most values are around 500, residuals can still be almost exactly normally distributed. However, when the number of trials is very low (like 3) normality assumption obviously cannot hold.

Is there a rule of thumb for the minimum number of trials such that OLS can be used (of course normality of residuals still should be checked)? Furthermore, if normality of residuals is approximately normal with OLS, what is the downside of using binomial regression?

In our case, we have a few different data sets where the outcomes are always (integer) scores on different tests. The score indicates how many exercises could a subject solve from the given test. The maximum scores can vary between different tests although they measure the same thing. There are tests with a maximum of 3 points, and tests with a maximum of 30. I guess with 30 points OLS might work, but not with 3. Should we just analyze everything with binomial (of course again, we need to check the assumptions)

Bonus question: If we fit OLS and binomial regression to the dataset with max 30 points, and one of the coefficients is significant for the OLS estimate but not significant for the binomial one, how can we interpret this?

  • $\begingroup$ You need to give us more details! What is the integer scores, that does not sound like counts? Assuming they are binomial counts, why not use logistic regression, which has NO normality assumption? There is little to gain by using OLS, when assumptions are not met ... $\endgroup$ Jun 4 at 22:00
  • $\begingroup$ That would also be my guess. However, we are reanalyzing articles and they all used OLS for tests where sometimes the maximum score was 5 or 10. The score indicates how many exercises could a subject solve from the given test. $\endgroup$
    – zerz
    Jun 5 at 8:22
  • $\begingroup$ Please add that new nfo as an edit to the post! Post should preferably be self-contained, and not everybody reads comments. $\endgroup$ Jun 5 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.