Compared one-factor and two-factor models

I am trying to test in AMOS the discriminant validity of a 3-item measure (items "1, 2 and 3" against a 1-item measure "item Z" (both are Likert scale questionnaire instruments), by comparing whether the fit of a two-factor solution is better than a one-factor solution (where all four items load on the same factor). The two measures are theoretically positively related to one another, but should be clearly different constructs. The models are shown in the picture (above). However, when I run the models separately I get identical chi-square values and other indices.

I have a few questions related to this (sorry if something is not adequately explained, since I'm rather new to AMOS):

  1. Is this a good way of testing discriminant validity when one of the measures is a one-item measure? I have used Average Variance Extracted estimates before, but based on what I've read they don't seem to work with one-item measures. If not, I would be extremely happy to hear thoughts on other possible analyses to use for this. Using simple correlations is difficult, since by theory they are in the low-moderate range (and I'm not aware of good cutoff value criteria for what correlation is "too high" for two variables to no longer be considered discriminant from each other).

  2. Is the model specified correctly? Since there is a high number of factors compared to the number of items, I had to fix some estimates (namely, factor-level means and the error variance for the single-item factor) to get enough degrees of freedom, and I'm not sure whether I chose the correct estimates to fix and/or correct values for them.

  3. What could be causing the exactly similar chi-squared values and indices of these two models? And if there's nothing wrong with the models (I assume there is), how to interpret this?

  • $\begingroup$ Update: I was able to consult a senior colleague on this problem, and he instructed me to remove the error variance e4 from the 2-factor model. However, I still get exactly same indices (chi-squared, RMSEA and AIC values) for the two models.. $\endgroup$ – MiguelC Jan 23 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.