8
$\begingroup$

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories.

For example:

1. Can you do painting (Yes- 1 point, No- 0 Point)
2. Do you have qualification in painting 
   (No - 0 point; Bachelor - 1 point; Postgraduate - 2 points)
3. Is there any painter(s) in your family (Yes- 1 point, No- 0 Point)     
Etc.

The questionnaire also contains conditional items. E.g., if someone chooses "No" for question 1, he or she will skip question 2 and jump to question 3.

Question

Can I use Cronbach's alpha on a survey that contains categorical factual items?

I assume questions like these cannot be measured with Cronbach's Alpha. I can't seem to find anyone using cronbach's alpha for such questionnaire.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
5
$\begingroup$

Some quick rules

  • If you have unordered categorical data (i.e., three or more unordered categories; which you do), then you don't use Cronbach's alpha.
  • If you have binary data (e.g., incorrect/correct data), then many people do use Cronbach's alpha, but see the Sjitsma reference given by @Momo.
  • If you have conditional data, then that would at the very least complicate the application of Cronbach's alpha. Skip patterns often imply the existence of an implicit additional category (e.g., "Do you play soccer?" if yes, "what day of the week do you play most often?", you could say that for the second question, there is an implicit category of "not applicable") . However, in your example, skipping item 2 means that the person does not have a degree in painting. So you could fill in that information. In all these examples there are more than 2 unordered categories so you would not apply cronbach's alpha.

Other thoughts

  • Cronbach's alpha relies on internal consistency to evaluate reliability. However, if your scale is formative, then internal consistency measures don't make much sense. In your case, I think your scale could be conceptualised as formative rather than reflective. I.e., the items in their totality represent something like "painting experience".
  • You might want to look at something like test-retest correlation or categorical PCA if you need to calculate some form of reliability.
Improve this answer
$\endgroup$
2
$\begingroup$

Generally Cronbach's coefficient $\alpha$ should not be used if you want a measure of reliability or internal consistency (which is what you need it for, I presume). See the OA Psychometrika article by Sjitsma.

An easily available alternative is the GLB statistic (e.g. in R psych::glb).

Edit

Based on the comments by @chl I think the following caveat is in order: The "conditional questionnaire" structure will likely introduce blocks of missing values. I suppose the skip pattern and the missing value patterns induced will affect the (co-)variance estimation usually used in reliability coefficients if the missing value mechanism is not missing completely at random. Unfortunately, I don't know how this effect will look like though.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Any thought about the impact of responses imbalancement (due to skip rules) on reliability analysis? $\endgroup$
    – chl
    Commented Jun 18, 2012 at 17:39
  • $\begingroup$ I think you are referring to the "conditional questionnaire" remark. No, I never encountered this situation before (which, frankly, strikes me as a little bit of an odd construction). But you are right, maybe this could lead to a lot of missing data for particular items and an imbalance problem which in turn might influence reliability. Like I said, I don't know more about this, so maybe some else can provide a better explanation. Since reliability is usually correlation I would hope it might play out nicely. But then again, I would use IRT from the start (which doesn't help the OP here). $\endgroup$
    – Momo
    Commented Jun 18, 2012 at 17:57
  • $\begingroup$ Thanks. Indeed I was referring to skip patterns (or branching in computer-assisted survey) which may introduce missing responses or simple errors at the person level. $\endgroup$
    – chl
    Commented Jun 18, 2012 at 18:24
  • $\begingroup$ That's actually quite an interesting question. Maybe someone can provide more background. Just out of the top of my head, reliability is usually defined as something like $\sigma^2_T/\sigma^2_X$ and hence the estimation uses some function of (co-)variances. Therefore it should be affected by the skip patterns the same way (co-)variances are affected by missing values. I added a caveat to the answer and will wait for a brighter person to take over. $\endgroup$
    – Momo
    Commented Jun 18, 2012 at 19:43
0
$\begingroup$

If you are looking at Yes/No items or items coded of 0's and 1's, I have used Guttman's split Lambda 4 coefficient, which can be done in SPSS easily.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Hi @DrStevenMason, welcome to CV! Could you expand on your answer a little bit? Perhaps by indicating what the "Guttman's split Lambda 4 coefficient" is? Also, nothing in the question indicates that the OP uses SPSS. $\endgroup$
    – Patrick Coulombe
    Commented Apr 11, 2014 at 3:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.