2
$\begingroup$

In classical test theory (CTT), we would say something like: "Your score is higher than 80% of people in the normgroup."

In item response theory (IRT), how do we interpret/feedback a theta score of 0.85? I know that it is supposed to be 'item-referenced' as opposed to be 'norm-referenced', but I am unsure how to put this to practice.

You are able to solve tasks that are so difficult that only 20% of others can?

$\endgroup$
4
  • 1
    $\begingroup$ Could you expand "CTT". IRT is a family of models, so i is important to say which models you consider (binary, 2-parameter Rasch model?). $\endgroup$ Commented Feb 2, 2016 at 12:04
  • $\begingroup$ I am using a 2 parameter model now. Is the interpretation of theta that different depending upon the number of parameters in the model? $\endgroup$
    – SunWuKung
    Commented Feb 2, 2016 at 14:48
  • $\begingroup$ And what is the most important - what is the distribution of thetas? Typically it's Gaussian (mean 0 std 1, or something different?) but may be something different (fixed or fitted to the data). $\endgroup$ Commented Feb 4, 2016 at 9:29
  • $\begingroup$ @PiotrMigdal CTT is classical test theory from psychometrics. $\endgroup$ Commented Feb 23, 2021 at 2:34

2 Answers 2

3
$\begingroup$

As @user136371 says, item response theory (IRT) theta scores estimates $\hat\theta$ can be interpreted as standardized z-scores when the latent variable mean $\mu_\theta$ and standard deviation $\sigma_\theta$ are fixed to 0 and 1 respectively. Remember that this is only the case when an estimation method such as the marginal maximum likelihood (MML) is used.

Additionally, a strength of IRT methods is that reliability is not fixed across $\theta$, so $SE(\hat\theta)$ should be used in conjunction with $\hat\theta$ when interpreting $\hat\theta$ as standardized z-scores.

Finally, see a previous answer of mine here that goes into the z-score interpretation of IRT theta scores in more detail.

$\endgroup$
2
  • 1
    $\begingroup$ I understand the statistical interpretation. What I would like to know is how to link back the Theta score to the observable performance. Theta 1 would mean that 68% of users are unable to solve tasks that are as difficult as this user can? $\endgroup$
    – SunWuKung
    Commented Oct 17, 2022 at 13:20
  • $\begingroup$ Yes, @SunWuKung, that is a correct interpretation. $\endgroup$ Commented Nov 21, 2022 at 19:33
0
$\begingroup$

theta is a standardized z-score. theta of 0 = mean of your latent trait, SD = 1. theta of 0.85 means nearly 1 standard deviation above the mean.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.