I am an applied linguist and I am modelling responses to a vocabulary test taken by second language learners of English; the aim is to test theoretical hypotheses regarding the relationship between the nature of the word and the likelihood of the learners’ knowing that word. The models I am using to explore the role of explanatory variables are the random item Item Response Theory model LLTM+e (see de Boeck et al. 2011; de Boeck 2008). These are created using the lmer function in the LME4 package in R and treat item and person responses as random. The estimates shown below are effectively like those from a binary logistic regression model, indicating the log-odds of a correct response on a word, given certain properties. Covariates relate to both item and person characteristics.

The nature of my concern is actually a basic regression issue. I have found a significant interaction between ability grouping of the test taker (GRP; with two levels High and Low) and the length of the word in letters (LEN_L). As far as I can see the estimate of the fixed effects for the first model shows that (a) the lower level learners have an overall lower probability of giving a correct answer (b) that LEN_L does not provide a significant explanation across the pattern of responses for the whole test-taker population, and (c) a significant interaction between GRP and LEN_L indicates that the lower ability learners are less likely to give a correct answer for a longer word. This is in keeping with theory.

However, when I model the data without including the main effect for LEN_L, I am not seeing a significant effect for either high or low groups as shown in Model 2. LEN_L does not show as significant if modelled without interaction with grp low (not shown). I feel that I am missing something obvious, but I cannot quite grasp what is happening. And my references have run dry on this particular issue and I am thinking myself around in circles about it.

(NB: in my full model I have many other significant covariates, but this pattern holds true.) My query basically regards whether I should use Model 1, including the non-significant main effect, or whether my findings from Model 2 indicate that the finding is a little unstable. Any advice would be much appreciated! (I can post more details if necessary.) Karen

Model 1 Fixed effects:

              Estimate Std. Error z value Pr(>|z|)
(Intercept)    0.25244    0.83194   0.303  0.76156    
grp low       -2.09126    0.26406  -7.920 2.38e-15 ***
LEN_L          0.02093    0.13062   0.160  0.87272    
grp low:LEN_L -0.09185    0.03146  -2.919  0.00351 **

Model 2 Fixed effects:

               Estimate Std. Error z value Pr(>|z|)
(Intercept)     0.25243    0.83194   0.303    0.762    
grp low        -2.09126    0.26406  -7.920 2.38e-15 ***
grp high:LEN_L  0.02093    0.13062   0.160    0.873    
grp low:LEN_L  -0.07092    0.13202  -0.537    0.591  

De Boeck, P. et al. (2011) ‘The Estimation of Item Response Models with the 'lmer' Function from the lme4 Package in R’. Journal of Statistical Software (39:12) pp 1-28

P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., Tuerlinckx, F. and Partchev, I.