# How to calculate test information in computerized adaptive tests

Under Item Response Theory, test information $I(\hat\theta)$ is a function of the examinee's estimated ability $\hat\theta$, discovered a the end of the test, and the items that were answered during said test. The standard error of estimation $SE$, in turn, is a function of the test information. I would like to calculate $SE$ in a CAT, but first I need to calculate the test information.

Question: In a computerized adaptive test (CAT), how do you calculate test information if, at each step of the test, a new value of $\hat\theta$ is estimated? Do we use the final estimation, like this: $$I(\hat\theta) = \sum_{0<t<N} I_t(\hat\theta_N)$$ or do we use the partial estimations of $\theta$ during the test to calculate item information at that particular point, like $$I(\hat\theta) = \sum_{0<t<N} I_t(\hat\theta_t)$$

• Not sure I understand your question. The SEE definition here is the expected standard error, while the standard error of $\hat{\theta}$ is based on the observed responses. You can use SSE as the SE estimate of $\theta$, but for estimators other than ML it won't be correct (e.g., using a Bayesian estimator the SEE term will not be appropriate). Feb 13, 2016 at 19:54
• I think I used the wrong $\theta$ in my $SEE$. If we calculate the standard error using $I(\hat{\theta})$ instead of $I(\theta)$, can it then be considered IRT ordinary standard error? Also, thanks for the edit. Once this pre-question is out of the way, I'll elaborate more in my question. Feb 13, 2016 at 22:34
• You have to use $\hat{\theta}$, you never actually have access to the population parameter in order to calculate the true information. The information function can be used as a suitable standard error when the observed information is identical to the expected information (which for 2PL and GRM's is true) and only when the estimator is found via maximum-likelihood. Feb 14, 2016 at 1:33
• I guess my question is better now. If I find a way to calculate a CAT information, I can then calculate its standard error. Feb 14, 2016 at 3:17
• Information (test or item) are just functions, so it doesn't matter if $\hat{\theta}$ changes during your estimation. Just change the respective inputs to obtain the desired values. Perhaps where you are getting hung up is that information functions assume that $\theta$ is fixed and known, where the trick in CATs is to both select informative items and improve the potential accuracy of $\hat{\theta}$ at the same time. Feb 14, 2016 at 4:08