Likert scale based composite variables

Question

I am conducting a small research into the adoption of smart home technologies for elderly people. In this research I have conducted both questionnaires and interviews, with the questionnaire being conducted among people who were interviewed. In total 11 people were interviewed and so 11 responses were obtained for the questionnaire.

Currently I am trying to analyze the questionnaire, I am, however, a little bit stuck so maybe you can provide us with some advice.

In my questionnaire I am aiming to measure both loneliness and the health status of elderly people. These variables are measured through 3 and 6 five point Likert scales (strongly disagree - disagree - neutral - agree - strongly agree) respectively. Now that I have got the questionnaire results, I'd like aggregate the different Likert scales, measuring the same concept, into one score that represents the concept, in order to be able to compare participants on their scores.

I, however, have no clue how to do this. What I do know is that averaging the scores composing one scale is not reliable as the data is ordinal. Furthermore, median and modes do not make sense either as the number of statements is too low for them to be valuable. Simply adding the scores might be a solution, though I have the feeling that this is not entirely correct due to the possible implied intervals that are suggested by this method.

Do you have any advice on how to best calculate a composite score for each of the two scales consisting of 3 and 6 items respectively?

Answer 1

There are two ways you can take: (1) just use the sums of scores, (2) use an Item Response Theory (IRT) based method. Using sums of raw scores is very common in social sciences but many psychometricians do not consider it being a sound approach. If you sum up the different questions from the questionnaire you assume that every answer provides you with the same amount of information - and in the real life that is not true. However, your data provides you in information on both the "abilities" of your responders and on precision of your questions, so that you can use both sources of information to gain deeper understanding of both your questionnaire and your responders. This is a pretty wide topic so you can check different resources on this topic, e.g. here, here or in this book. IRT will let you to use your data to obtain information on latent features measured by the questionnaires on continuous $Normal(0, 1)$ scale, so it also makes life easier with further analysis. It is mostly used in the area of educational research, so don't get discouraged that most examples in the books and articles are on measuring student abilities, because the method could be used for analyzing any kind of test or questionnaire data to obtain the latent profiles of the responders.

There are many statistical packages for IRT, for example, in R you can use mirt or ltm.

Answer 2

I'm a little late, but Aleksandr's point about using structural equation modeling for this is spot on. Basically, when you say your Likert scales are measuring an underlying concept, you're saying they are a reflection of a latent variable that can only be indirectly measured. SEM is built for this and can produce estimates of those underlying latent variables as well.

Here's a simulation of your situation in R, using the lavaan package:

set.seed(1)
library(lavaan)
library(infotheo)

# sample size
N=100

# simulate "loneliness", a latent variable with an arbitrary scale
loneliness = runif(N,0,1)

# simulate "health", a latent variable dependent on loneliness and other unknown factors
B0 = 15
B1 = -4
E = rnorm(N,0,1)
health = B0 + B1*loneliness + E

# display
plot(loneliness, health)

# simulate latent responses to 3 questions about loneliness
b1 = 1.5
e1 = rnorm(N,0,1)
l1 = b1*loneliness+e1

b2 = 2.5
e2 = rnorm(N,0,1)
l2 = b2*loneliness+e2

b3 = .75
e3 = rnorm(N,0,1)
l3 = b3*loneliness+e3

# simulate latent responses to 6 questions about health
b4 = 1
e4 = rnorm(N,0,1)
h1 = b4*health+e4

b5 = 2
e5 = rnorm(N,0,1)
h2 = b5*health+e5

b6 = 5
e6 = rnorm(N,0,1)
h3 = b6*health+e6

b7 = 7
e7 = rnorm(N,0,1)
h4 = b7*health+e7

b8 = .25
e8 = rnorm(N,0,1)
h5 = b8*health+e8

b9 = 3
e9 = rnorm(N,0,1)
h6 = b9*health+e9

# discretize latent responses to get the "observed" responses to likert questions
l1 = discretize(l1, nbins=5)$X
l2 = discretize(l2, nbins=5)$X
l3 = discretize(l3, nbins=5)$X

h1 = discretize(h1, nbins=5)$X
h2 = discretize(h2, nbins=5)$X
h3 = discretize(h3, nbins=5)$X
h4 = discretize(h4, nbins=5)$X
h5 = discretize(h5, nbins=5)$X
h6 = discretize(h6, nbins=5)$X

# put all "observed" data together
data = data.frame(l1, l2, l3, h1, h2, h3, h4, h5, h6)

# define structural equation model
model = '
    # define latent model
    health ~ loneliness

    # define measurement models
    health =~ h1 + h2 + h3 + h4 + h5 + h6
    loneliness =~ l1 + l2 + l3
'

# fit structural equation model
semFit = sem(model, data)

# display results. note that the relative magnitude of the coefficient estimates corresponds to b1-b9
summary(semFit)

# predict composite estimates health and loneliness
predictions = as.data.frame(predict(semFit))

# display fit
plot(predictions$health, health)
plot(predictions$loneliness, loneliness)

There are a number of assumptions in this model that I won't go into, but basically I'm generating fake Likert variables assuming that they have real correlation with their respective latent variables. Then, I define the structural equation model and fit it.

The important part for your question is at the end where I predict the latent variables. This produces estimates of what the latent variables' values are for each individual. The values are on a completely arbitrary scale, because "loneliness" is an abstract concept. Nonetheless, the relative values matter. You can see in the final two plots that the original simulated latent variables do correlate with SEM's predictions of them.

Note, however, that I used a sample size of 100. Your sample of 11 might be too small for this to work with real data that may be biased or have poor construct validity.

Here's a good reference to learn more about SEM: http://davidakenny.net/cm/causalm.htm