How to calculate Cronbach's Alpha for a total scale that consists of four subscales?

Question

I have a scale that has 4 subscales. Each subscale has 5 items. For calculating Cronbach's alpha for the total scale, should I use subscale values (in this case it would be 4) or should I use individual item values (it would be 20). Thanks and would appreciate if you attach a link to any publication on this topic.

Answer 1

I don't have a citation handy (I might be able to dig one up, I'm just in a bit of a rush), but I would certainly use all 20 items. The 4 scales aren't items.

However, a fuller study of the validity and reliability of a scale would involve more work, probably including either a confirmatory factor analysis to see if the items do in fact match the scales, and maybe a reliability analysis of each scale, separately.

And you would also want to look at statistics for each item.

Answer 2

I'm assuming that because you have a number of sub-scales involved in your composite, they are probably also multi-dimensional, in that as a whole they cover an over-arching theme, but separately each sub-scale is another latent structure related to the whole construct. A good example is a factor "psychological cost" related to four sub-constructs, which can be devised as such (from Flora, 2020):

I think what would be better than the typical Cronbach's alpha is a coefficient that captures this sort of structure. This can be achieved with McDonald's hierarchical omega, which takes a factor analysis style approach to the problem. Here is an example using R:

#### Load Libraries ####
library(lavaan)
library(semTools)
library(MBESS)

#### Get Data ####
pcs <- read.csv("https://osf.io/xd2tu/download")

#### Fit Bifactor Model ####
modBf <- '
gen =~ TE1+TE2+TE3+TE4+TE5+
OE1+OE2+OE3+OE4+LVA1+LVA2+LVA3+ 
LVA4 +EM1+EM2+EM3+EM4+EM5+EM6 
s1 =~ TE1 + TE2 + TE3 + TE4 + TE5 
s2 =~ OE1 + OE2 + OE3 + OE4 
s3 =~ LVA1 + LVA2 + LVA3 + LVA4 
s4 =~ EM1 + EM2 + EM3 + EM4 + EM5 + EM6
'

#### Fit Factors ####
fitBf <- cfa(modBf,
             data=pcs, 
             std.lv=T,
             estimator='MLR', 
             orthogonal=T)

#### Pull Reliability Estimate ####
reliability(fitBf)

Fitting a general factor "gen" here and four sub-factors called "s", we get the following estimates per construct:

             gen         s1        s2        s3
alpha  0.9638781 0.92504205 0.8992820 0.9052459
omega  0.9741033 0.56377307 0.7884791 0.6766430
omega2 0.9094893 0.09237594 0.3666293 0.1880759
omega3 0.9077636 0.09240479 0.3666634 0.1878380
avevar        NA         NA        NA        NA
              s4
alpha  0.9405882
omega  0.7816839
omega2 0.2054075
omega3 0.2053012
avevar        NA

You can see that the general factor has an acceptable reliability, but each sub-scale varies a lot. A very practical and insightful article on this coefficient can be found in Flora's article below. For a more in-depth treatment, McDonald's original book on this goes into more detail, but the other articles I have below are more brief and offer a lot of detail.

Citations

• Dunn, T. J., Baguley, T., & Brunsden, V. (2013). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
• Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484–501. https://doi.org/10.1177/2515245920951747
• McDonald, R. P. (1999). Test theory: A unified treatment. Routledge.
• Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395–1411. https://doi.org/10.1037/pas0000754
• Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

Answer 3

Cronbach's alpha is a measure of composite reliability that is based on the classical test theory models of (essential or strict) tau equivalence. Tau equivalence means that the components (items or subtests) of the composite (sum score, aggregate) are unidimensional (measure a single latent true score variable) with equal weights (factor loadings) and uncorrelated error variables. It does not make sense to compute Cronbach's alpha for a set of items or scales that are multidimensional (i.e., that measure multiple distinct true score/latent variables). There would be no clear interpretation in terms of composite reliability.

It could make sense to apply alpha to each of the 4 item sets separately if the given item set meets the assumptions of (essential or strict) tau equivalence (the tau-equivalence assumptions are testable via confirmatory factor analysis). But it would not make sense to apply alpha to all 20 items combined if those items measure multiple dimensions (factors, latent variables).

A relevant reference here is:

Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge/Taylor & Francis Group.