I have been teaching myself IRT. I think I get it, however, I've not yet reached a point where I would say that I'm totally confident.
In assuming that the latent trait is normally distributed with mean θ = 0 and SD = 1, am I correct in believing that θ estimates can be thought of as z-scores on a standard scale and as a corollary, converted to percentiles?
The reason why I ask this is because very rarely is this even mentioned in any references. Actually, I can't recall ever coming across a single reference discussing.
The following article claims that although IRT models assume normality this is not necessarily the case:
It’s common in Item response theory (IRT) item parameter estimation to assume that the latent variable (the trait/concept/construct) being measured follows this standard normal distribution (the assumption of normality). The reason for this assumption is that it makes the calculations for getting items parameters a lot simpler, compared to if you didn’t make the assumption… While research has shown that assuming the standard normal distribution for a population that isn’t standard normally distributed isn’t necessarily the end of the world (De Ayala, 1995; Stone, 1992; Woods & Lin, 2009), if a population distribution is decidedly non-normal and decidedly non-symmetric, parameter estimates can become quite biased/wrong (e.g., Woods, 2015).
Furthermore, in Item Response Theory by DeMars (2010) she states:
The metric of the parameters is somewhat arbitrary. The metric is indeterminate until we fix the center (zero-point) of the scale and the unit size. Most frequently, in a reference population (perhaps represented by the norming sample) the estimated mean of θ is set to 0 and the estimated standard deviation of θ is set to 1. θ can then be interpreted similarly to a z-score. Because θ is not a linear function of the number-correct (raw or observed) score, θ will not be exactly the same as the z-score on the number-correct metric, but the interpretation is similar. When an examinee is 1 standard deviation above the mean in the reference population, the examinee’s θ = 1. There is no assumption that the scores are normally distributed, so the θ does not necessarily translate to a percentile score in the normal distribution. Values for θ theoretically range from –∞ to ∞; most examinees will have values between –3 and 3.
One additional note on data requirements: IRT models in no way assume a normal distribution. The estimation procedures do not require any assumption of normally distributed examinee ability or normally distributed item parameters. The standard errors of the estimated and item parameters are asymptotically normal, but the examinees and item parameters do not need to follow a normal distribution. Severely non-normal distributions may present some practical problems to estimation, as discussed earlier, but this is not due to model assumptions.
So I guess my question could be re-worded as follows:
I would like to report theta estimates in addition to converting them to the equivalent z-score percentiles, primarily because percentiles make sense to people. If I were to do this, would my doing so be considered misleading or otherwise inappropriate?