Reporting the use of different likert-scales in survey for structural equation modelling?

Question

Suppose you are doing a survey and from the data you want to build a structural equation model. The research model consists of 5 constructs and each of them has between 2 and 5 items. The survey uses Likert scales but the scales differ between the constructs: 2 constructs use 7-point likert scales, one uses a 6-point likert scale and two use 4-point likert scales.

1. Is it a problem from a methodological point of view to mix these different scales when doing structural equation modelling? (perhaps especially when odd and even number scales are mixed?)

2. Should I report the use of different likert-scales somewhere in the body of the paper, e.g. in the limitations section?

Answer 1

Is it a problem from a methodological point of view? Sort of, since strictly speaking, SEM assumes that the observed variables are normally distributed, which a fortiori, your likert items are not.

So what to do? You could hold your nose and pretend that everything is normal, trusting in the Central Limit Theorem. I would probably do that, at least as a preliminary, to see if there's anything going on.

A cleaner solution is to use a SEM method adjusted for likert items. Instead of working with the correlation matrix, these methods treat the likert responses as cut points for an underlying continuous variable, whose correlations one then seeks to estimate. Any time I've done this, all variables have had the same number of likert responses, so I don't know if there's an off-the-shelf package for estimating these correlations with discordant likert items. However it should be possible in principle, and it has probably been done in practice somewhere, by someone. If you are using R, you could check out the user group for package lavaan.

In answer to your final question, of course you report all this. In the Methods section of your paper, you will have described the data you are using, including it's Likertship and other issues. You can then explain how you addressed the difficulty.

EDIT. I did some googling and came up with this. There is software that does polychoric correlations with mixed levels. That's what I would advise. Be aware, however, that you need more subjects for polychoric correlations than you would if you could observe the continuous latent variables directly.

Answer 2

It's a problem in as much as it's something you ought to address appropriately, but there are suitable means of doing so. It certainly makes sense to point out that different Likert scales were used if you're handling them properly, because it is the motivation for using those methods. If you're not...I guess I wouldn't want to offer any advice for that scenario.

As @Placidia has written, polychoric correlations are a good idea. They seem to produce less biased results with maximum likelihood estimation than fitting SEMs to ordinary covariance matrices ^{_{(Wang & Cunningham, 2005)}}. Weighted least squares estimation may be even better. lavaan supports both estimators, and psych package has a function for polychoric correlations. See Factor analysis of questionnaires composed of Likert items for more on this train of thought.

mirt may also be worth checking out, though I haven't used it much yet – it uses multidimensional item response theory models (hence the name), and may be able to handle several different rating scale models. AFAIK, that's the appropriate IRT model for several items that indicate the same latent construct and share the same Likert scale. I wouldn't necessarily use the same rating scale model thresholds for two different questionnaires with the same number of response options, especially if the options are defined differently – I would prefer to let thresholds differ across questionnaires, but not across items within them. Letting thresholds differ for each item would be a partial credit model BTW. I wrote more about this in an answer to, "Can a dichotomous variable (yes/no) be merged with a Likert measure (1,2,3,4) using z scores?"

---

^{References
· Wang, W. C., & Cunningham, E. G. (2005). Comparison of alternative estimation methods in confirmatory factor analyses of the General Health Questionnaire. Psychological Reports, 97, 3–10.}

Answer 3

Ditto some of the above posts...

It's not a problem to mix different likert scales, but you do need to account for the fact that you have likert data as the indicators for your SEM model. Classic estimation methods in SEM, (i.e., maximum likelihood (ML)), are based on the assumption that the observed variables are measured on a continuous scale. I don't know about your field, but in my field you'd be fooling yourself if you considered the likert data interval or continuous. Therefore, you probably need to take special care to account for the likert data.

I have used polychromic correlations as the input data for likert SEM models to deal with the fact that likert scales are ordinal. However, depending on how your data is distributed, you also could solve this problem by using different estimation procedures, such as the WLSMV estimation procedure, which helps to crack the issue of pilling up within each question. I think lavaan will allow for WLSMV estimation, though when I last used WLSMV I was using MPlus. In other instances where the likert items are distributed normally, and you have more than 3 indicators for a construct, then parceling is used to make sure that the construct only has 3 indicators. Parceling means adding together two or more indicators in order to form a single indicator. Here's a nice article on parceling.

The other thing you should consider is how your constructs are identified, anything less than 3 indicators or more than 3 indicators, you are either under or over-identifying your construct. This last point has been written about extensively in the psychometric literature.

Edit: The other question I have is are you considering just using a single question as an indicator for your construct? It's not clear to me in how you asked your original question. If this is correct, that's a dubious practice, and I can followup with citations, if not then disregard.