Can a dichotomous variable (yes/no) be merged with a Likert measure (1,2,3,4) using z scores?

Question

I would like to create a composite scale using items that measure parent communication extracted from a UK panel survey (n=941). However although all 6 variables in question measure similar constructs - 2 of the items are dichotomous (yes/no) the other 4 are likert (1,2,3,4). My intention is to transform the 6 variables to z scores and then sum them to form the composite score which will be used in a path analysis. I need to know if it is correct to do this, i.e., merge dichotomous variable (yes/No) and Likert scale (1,2,3,4) after they have been standardized to z scores.

Pending this being ok, would it be necessary to do an exploratory factor analysis to ensure the the composite scale measures one underlying construct or it is enough to do a reliability analysis?

This analysis is forming part of my PhD so I need to defend what I am doing and I would appreciate any guidance anyone can give me.

Answer 1

I'm not sure about the first part of your question. But regarding the second bit: a reliability analysis of itself does not tell you if you have one underlying construct or several. You can have a high cronbach-alpha (for reliability) in the presence of two or more factors. Definitely, do the factor analysis as well as reliability. You might also want to check out the latent variable and item response literature. Some of these models are set up to handle dichotomous and polytomous outcomes - which might deal with the z-score problem as well.

Answer 2

I just checked the textbook Introduction to Meta-analysis (Chapter 7: Converting Among Effect Sizes) and it is possible to standardize the outcome measures to combine them using the z scores (as you already know). Whether or not you do this will depend on the origin of the data and how similar the questions are to each other. If you believe that the 6 questions are really asking the same thing but presented using two different effect estimates then it should be fine to combine them.

Hope this helps.

AMAS

Answer 3

This answer is based on another answer of mine (which I've also adapted here), but this version is adapted to your question slightly.

If you're not sure your items are all measuring the same latent construct, you probably should consider exploratory item factor analysis. See these questions for some tips:

• Factor analysis of questionnaires composed of Likert items
• Validating questionnaires
• How to carry out a Likert scale analysis?
• and possibly Can one validly reduce the numbers of items in a published Likert-scale?

If (or once) you're sure your six items are all measuring the same latent construct, you could use a partial credit model to account for differences in response scaling across all six items. If the four items with 4-point Likert scale (polytomous) measurements are all on the exact same scale though, you might be better off dropping the binary items and using a rating scale model of the four polytomous items, depending partly on how much unique and valid information you get out of those binary items. John Michael Linacre and Benjamin D. Wright posted some discussions of the differences between partial credit and rating scale models over at rasch.org that might give you a better sense of what you'd be dealing with if you go the item response theory route here.

Some latent variable analysis programs will let you set certain thresholds to be equal across certain items and leave another item's threshold freely estimated. You might be able to blend the partial credit and rating scale models this way by setting your six polytomous items' thresholds (each item will have three) to be equal across items, and estimating the binary items' single threshold independently (each could also have its own threshold, depending on how different the binary items are)...but I'm not exactly sure this is all you'd need to do to have the best of both worlds.

The simple, "classical test theory" approach that weighs every item equally would probably have you just standardize all the items and average the $z$-scores, but I don't think that's a good idea, because four-point Likert scales may not approximate a continuous dimension well enough (and a binary item definitely won't; it might not even make sense), nor can you be sure that the average of four polytomous items will be approximately continuous enough. I've seen it suggested that each item's Likert scale should have at least five options to approximate a normal distribution, and at least five Likert scale items should measure the same construct if their simple sum / average is to approximate a continuous dimension. (I've seen Berry, 1993 cited as a source for an argument that five or fewer options is unacceptable, but I haven't read it myself.)

Reference

Berry, W. D. (1993). Understanding regression assumptions. Newbury Park, CA: Sage.