Analyzing changes in a repeated categorical measure

Question

I am trying to analyze why some individuals change their response to a categorical survey item Y between time T1 and T2, given some explanatory variables X1 and X2

Specifically, the data looks like this

    ID Y  T  X1 X2   
    1  1  1  a  b
    1  0  2  a  b
    2  1  1  b  b 
    2  1  2  b  b
    3  0  1  a  a
    3  0  2  a  a
         ...

So far I transformed the response variable Y into the four possible combinations of Y at T1 and T2. Specifically,

Y_new  Y_T1 Y_T2
 A       0   0
 B       0   1
 C       1   0
 D       1   1

and then I ran a multinomial logit model on the new variable, using the covariates X1 and X2.

Is this a feasible approach ? I worry that the transformation violates the IIA assumption. Also, are there any other methods that are better suited for this kind of data?

Answer 1

If you used a simple multinomial model without explicitly including the time dependency, I don't think your approach works well.

Luckily, researchers in psychometrics have developed a number of sophisticated tools for measurement of change based on items (like survey questions). The key idea is to decompose the change over time into an effect for trend and one for group or covariate. You can check out the work done by Gerhard Fischer (which is probably a bit outdated), for example:

Fischer, G. Some Probabilistic Models for Measuring Change. In De Gruijter, D. N. M. & L. J. Th. van der Kamp (eds), Advances in Psychological and Educational Measurement, London: John Wiley, 97-110, 1976.

One class of models might be particularly useful in your case, dichotomous or polytomous linear logistic model with relaxed assumptions. They adapt commonly used IRT models to allow for measurement of change for continuous and categorical covariates for any arbitrary number of items. You might check out this presentation Measuring Change: Linear Logistic Models With Relaxed Assumptions (LLRA) and this paper Hatzinger & Rusch (2009) to see whether this model class is of interest for you and some of the statistical and computational ideas behind it.

Although the LLRA are a bit outdated (newer models exist), I mention them because software for fitting them is easily and freely available in the R package eRm. Hence, if you find this useful, the eRm package contains the function eRm::LLRA (or simply LLRA) that allows you to fit such models for the measurement of change in a very straightforward manner. The function will take care of all the steps from design matrix to model fitting for you (edit: made the example to reflect your data, i.e., one item and 4 groups):

#Example 6 from Hatzinger & Rusch (2009) adapted to one dichotomous item 
#four groups (a to d), two time points   
require(eRm) 
data(llradat3)
data<-llradat3[,c(2,5)] #select item 2 at time point 1 and time point 2 
groups <- c(rep("a",15),rep("b",15),rep("c",15),rep("d",15))
llra1 <- LLRA(data,mpoints=2,groups=groups)
summary(llra1)


Results of LLRA via LPCM estimation: 

Call:  LLRA(X = data, mpoints = 2, groups = groups) 

Conditional log-likelihood: -11.27 
Number of iterations: 14 
Number of parameters: 4 

Estimated parameters with 0.95 CI:
            Estimate Std.Error lower.CI upper.CI
d.I1.t2        0.981     1.443   -1.848    3.810
c.I1.t2        1.204     1.426   -1.591    3.999
b.I1.t2        1.204     1.426   -1.591    3.999
trend.I1.t2    0.405     0.913   -1.384    2.195

Reference Group:  a 

You can plot the relative change due to the groups effect by

 plotGR(LLRA1)

References:

• Fischer, G. Some Probabilistic Models for Measuring Change. In De Gruijter, D. N. M. & L. J. Th. van der Kamp (eds), Advances in Psychological and Educational Measurement, London: John Wiley, 97-110,
• Hatzinger & Rusch (2009) IRT models with relaxed assumptions in eRm: A manual-like instruction. Psychology Science Quarterly, 51, pp. 87 - 120
• Fischer, G. H. (1995). Linear logistic models for change. In G. H. Fischer & I. W. Molenaar (Eds.),Rasch models. Foundations, Recent developments and Applications (pp. 157-181). New York: Springer.