# The probability of a student getting a given question correct and the associated uncertainty (Item response theory)

I'm doing some statistics on multiple choice test scores (with item response theory, (http://en.wikipedia.org/wiki/Item_response_theory). I want to know the probability that a student with a certain grade (say between 70-80% on the rest of the test) got a specific question right, and the associated uncertainty to this probability.

Say I have 160 students that got between 70-80% on the test, and that 113 of these got the question I'm looking at right. The chance of another student in the same bin getting the question right is 113/160 -- but what is the uncertainty of this measurement?

Uncertainty is also related to $\hat{\theta}$ in the form $SE(\hat{\theta})$ should help to answer the last part of your query. Of course this also relates to item-parameter uncertainty as well since less precise parameters will contribute to the overall uncertainty in the expected probabilities, but that's a different story.
• Using the test scores is actual one kind of approximation for $\hat{\theta}$, so you are generally getting at what you want there. If you collect all the sum scores and standardized them (i.e., convert to z-scores) this is one crude approach. Much like other latent variable models though it is subject to measurement error, which is where the $SE(\theta)$ term comes in (IRT has a specific meaning for this). Determining model implied standard errors will help you form intervals for the probabilities since you can evaluate the upper/lower CIs of $P(\hat{\theta})$ given the item parameters. Commented Oct 16, 2014 at 22:50