2
$\begingroup$

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points.

$1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ always.
Yes $=1$, No $=0$.

If I add up all these with highest possible score and divide it by the total score received, I can then retrieve a total score for the survey.

For example, let's say I have 5 responses for the Likert and 5 for the yes/no. $5\times4$ $=20$ (with 4 being the highest possible score you can get on the Likert scale) and $5\times1$ $=5$ $(1$ being the highest possible score for yes/no question). I then add up both to find the highest possible score $=25$. If I get a combined score of $20$, do I divide it by the possible score, $20\div25=80$%? Can I build my questionnaire and calculations like this?

My main objective is to get one percentage / score for the entire questionnaire. I thought of accomplishing this in 2 ways:

  1. Get the sum of earned score from both scales divided by total possible scores for both scales.
  2. Get a percentage by using the above calculation for individual scales, which will give me two sets of percentages, and then take an average for those percentages.

I am not sure which way I should go with, or if this even makes sense doing it this way.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Are all of the items (likert or yes/no) intended to measure the same underlying construct? Have you checked anything to determine if they do? $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 7, 2012 at 19:21
  • $\begingroup$ All likert and yes/no questions are targeting the same dimension/composite. The address a specific area within the organization. For example, communication to clients have both yes/no and 4 point scale. $\endgroup$
    – Newb
    Commented Dec 7, 2012 at 23:57

2 Answers 2

Reset to default
2
$\begingroup$

This answer is based on another answer of mine, but adapted to your question slightly.

If you're sure your items are all measuring the same latent construct, you could use a partial credit model to account for differences in response scaling across all items. If the items with four-point Likert scale (polytomous) measurements are all on the exact same scale though, you might be better off using a rating scale model of the polytomous items and a separate, probably more basic item response theory model for the binary items. John Michael Linacre and Benjamin D. Wright posted some discussions of the differences between partial credit and rating scale models over at rasch.org that might give you a better sense of what you'd be dealing with if you go the item response theory route here.

Some latent variable analysis programs will let you set certain thresholds to be equal across certain items and leave another item's threshold freely estimated. You might be able to blend the partial credit and rating scale models this way by setting your polytomous items' thresholds (each item will have three) to be equal across items, and estimating the binary items' single threshold independently of the polytomous items. Depending on your theory about the binary items, they could all have the same threshold as each other, or they could each have their own, or maybe somewhere between those two extremes...but I'm not exactly sure this is all you'd need to do to have the best of both worlds.

The simple, "classical test theory" approach that weighs every item equally would probably have you just standardize all the items and average the $z$-scores, but I don't think that's a good idea, because four-point Likert scales may not approximate a continuous dimension well enough (and a binary item definitely won't; it might not even make sense), though the average of 12 polytomous items might be approximately continuous enough. I've seen it suggested that each item's Likert scale should have at least five items to approximate a continuous distribution, and at least five Likert scale items should measure the same scale if their simple sum / average is to approximate a continuous dimension. (Can't remember where, but I can look it up and edit it in if you want a source but can't find one yourself; just comment!)

If you're not sure your items are all measuring the same latent construct, I'm afraid you have other things to worry about; see these questions:

Improve this answer
$\endgroup$
1
$\begingroup$

As long as you consider the weight of the response, e.g is each response to the question worth the same.

Example: Question 1 - No (0/1) Question 2 - 3 (3/5) Question 3 - 3 (3/5)

Total score=6/11 or ~ 54%
If they were weighted equally it would be (0+3/5+3/5)/3 = 0.4 or 40%

Improve this answer
$\endgroup$
2
  • $\begingroup$ The weight of the response is not equal because yes/no has a total weight of 1 and the likert scale is weighted with a highest possible score of 3. According to your last equation, I am dividing the total from the amount of responses? (0/1+2/3+2/3+1/3)/4 = 41.67% $\endgroup$
    – user17616
    Commented Dec 8, 2012 at 23:31
  • $\begingroup$ @Newb You appear to have two unregistered accounts. Could you confirm that the one you used to leave this comment is the same as the OP? A moderator can merge your accounts, but you'll need to register once and for all. $\endgroup$
    – chl
    Commented Dec 9, 2012 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.