1
$\begingroup$

I'm relatively new to Item Response Theory. After having read some materials on 1PL and 2PL, I have a few thoughts and questions.

Say you have a questionnaire that a social psychologist will complete when evaluating a child age 12-24 months. The psychologist must record {0,1} for {yes, no} on the following, did the child use word, X? Where X consists of {"mom", "trash", and "yesterday"}. So, the child vector [0,0,0] signifies that the child used none of these words, whereas [1,1,1] signifies that the child used all three words.

Given the above hypothetical setup, it's my understanding that IRT aims to measure two latent factors, child language proficiency and item difficulty. The ideas of entropy and information come to mind. If all questions receive the same response, 0 or 1, then the question is either too easy or too difficult; regardless, we learn very little about the child when asking said question (I would be very surprised if a 1-2 year old used the word "nuclei", so it probably doesn't belong in said language assessment.)

1PL essentially performs logistic regression with a single predictor variable x and response variable, y. Where this approach differs from logistic regression is that: (A) x is not known, it's latent and (B) more specifically, x = ability - difficulty. This feature is then sent to a sigmoid function, followed by a Bernoulli likelihood. Using MCMC methods, various values for the vectors, ability and difficulty, could be proposed/explored where individual vector elements correspond to specific children and specific questions, respectively.

In the IRT resources I've viewed, a characteristic (sigmoid) curve is usually presented where the x-axis is relative easiness of the question (ability minus difficulty) and the y-axis represents correct answer to the question (or knowing the word in our case.) High x values indicate a very advanced child answering a very easy question, (y is virtually guaranteed to be 1 with very little variance) whereas low x values indicate a far less advanced child answering a very difficult question (y is virtually guaranteed to be 0 with very little variance.) Interestingly, moderate values of x indicate that the child and question are equally matched, and this where variance around outcome y is maximized.

Q1. Is my understanding correct thus far?

2PL models build upon this paradigm by introducing a second parameter, in addition to the derived variable x. Namely, x= v * ability - difficulty. (I might be mistaken here.) To the best of my knowledge, the parameter, v, controls the slope of the function. And this might be desirable as it basically says, "how sensitive is y, the probability of correctly answering question q, to small changes in relative easiness of the question?" As discussed previously, when relative easiness is at moderate values, specifically 0.5, we maximize variance in the outcome, y. And this extra parameter, v, allows us to think in terms of "how rapidly does variance shrink as x departs from 0.5?"

Q2. Is my formula for 2PL model accurate? I've seen multiple variations across resources, such as z = v * (ability - difficulty).

Q3. What does v capture? Child ability variance, question difficulty variance? Something else?

Thank you in advance!

$\endgroup$
1
$\begingroup$

The 1PL (one-parameter logistic model) is given by

$$p(\theta|b_i) = \dfrac{\exp(\theta - b_i)}{1 +\exp(\theta - b_i)}.$$

This is trying to model the probability of solving a given item with difficulty $b_i$ when the person has the ability $\theta$. It is clear that larger $\theta$ will increase the probability of solving the task if the difficulty $b_i$ does not change. Increasing $b_i$ will decrease the probability of solving a task if the ability $\theta$ is fixed.

This function has a shape like the letter s if $\theta$ is treated as the independent variable and the probability is your dependent variable. This is the reason why this function is of a sigmoid-type (sigma: greek letter for s).

enter image description here

The 2PL (two-parameter logistic model) is given by

$$p(\theta|a_i, b_i) = \dfrac{\exp[a_i(\theta - b_i)]}{1 +\exp[a_i(\theta - b_i)]}.$$

The effect $a_i$ (discrimination factor) is that the transition from the lower probabilities to larger probabilities will be more rapid. The following picture contains a black and a blue function. Both functions have the same difficulty $b_i$ but the discrimination $a_i$ is different. The discrimination of the blue function is larger than the discrimination of the black function.

enter image description here

Items with large discrimination are very good for distinguishing between people with abilities larger than the item difficulty and people with abilities smaller than the item difficulty. But these items are very by for discriminating abilities farther away from the difficulty of the item. See the next paragraph for an item with ideal discrimination ($a_i \to \infty$).

For the limit $a_i \to \infty$ we will obtain a step function. It will be zero on the left-hand side and 1 on the right-hand side of the ability $\theta$ which corresponds to the difficulty of the item. This model is called the deterministic Guttman-Model. It is clear that you will not be able to distinguish abilities larger than the difficulty (curve is flat).

Credit for pictures: Both pictures were taken from this source.

$\endgroup$
5
  • $\begingroup$ This is very helpful! Can you help me understand the role of a in the context of the language questionnaire? My thinking is that there's a tradeoff: A low a value (or very gentle slope) means that the question can be applied to virtually every child, however, we learn very little about him/her by doing so. Conversely, a high value for a (and a steep slope) means that we can only ask this question to a very small subset of children but we learn a good deal about him/her when near the theta value corresponding to y=0.5. Is this a good way to think about this? $\endgroup$ – jbuddy_13 Sep 24 '20 at 20:38
  • 1
    $\begingroup$ Correct! That is exactly the idea behind adaptive Item Response Theory. You start with a medium difficulty item (good discrimination). Then you estimate the ability of the person. Based on the ability of the person you select the next item with a difficulty as close as possible to the ability of the person. Then you repeat this procedure until you have obtained enough information (confidence interval is small enough) to stop the questionnaire. $\endgroup$ – MachineLearner Sep 24 '20 at 20:48
  • $\begingroup$ Awesome, thank you! $\endgroup$ – jbuddy_13 Sep 24 '20 at 21:15
  • $\begingroup$ quick follow up. In the 2PL model, is a(i) is a vector, with one value corresponding to each b(i) value right? I wasn't sure if it was a single scalar for all theta:b combinations or a matrix for all possible theta:b combinations. @MachineLearner $\endgroup$ – jbuddy_13 Sep 25 '20 at 17:04
  • $\begingroup$ You have a pair of difficulty and discrimination (a_i, b_i). The i can be understood as item. It is not common to think of a_i as a vector. $\endgroup$ – MachineLearner Sep 25 '20 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.