0
$\begingroup$

I have 3 matched pretest/postests sets. Participants sizes in the groups are 4, 8 and 13. My Likert scale has 5 places, and I have 14 valid questions. We are applying for a grant and want to demonstrate to the grantors that our treatment has been helpful to our clients. Grantors would like us to provide a percentage of "improvement" for each item and overall. My statistics are rusty, but I am good at application. Any suggestions on how I can provide the grantors with a percentage measurement?

Edit: The scale responses are as follows: "Not at all," "Rarely," "Sometimes," "Often," and "Very Often." Participants can start anywhere, and can move anywhere from 0 to 4 steps up or down the scale. I considered whether I could have each step as a 20% change, then again it seems as if there are actually 9 places of measurement since participants can move up or down the scale. The question is not if the person reaches a particular level on the scale, but rather if they perceive any changes in their way of viewing of or reacting to situations. The simpler the better, as long as it is an acceptable from of measurement. The intervals are indeed quite large thus far, however there is only a space on the grant application for %, and there is not a place for descriptors of significance.

$\endgroup$
2
  • $\begingroup$ Lots of ways to cut this cake, some context would help. One way is to look at proportion of individuals within each group who moved from X to Y on a given question. Picking the X and Y should be based on the initial hypothesis and context it provides. You could also abstract away the magnitude of the changes and just look at the direction if that's more relevant. You could also compare between groups. Once you have the numbers, you can test for whether the changes within each question are statistically significant, though expect very large intervals with such small samples. $\endgroup$ Jan 3 '19 at 17:17
  • $\begingroup$ That grant sounds cockamamie, but, having filled out grant forms, I can believe it. $\endgroup$
    – Peter Flom
    Jan 4 '19 at 10:05
0
$\begingroup$

If you need a single number for % improvement (which is a pretty bad way of summarizing results, but hey, you didn't write the grant form) then I would just take the % of people who got better from time 1 to 2 for each item. Then, for overall, you could just sum the likert responses (using 0, 1, 2, 3, 4 for the levels). Strictly speaking, this isn't really legitimate but a) People do it all the time and b) They made the grant form over-simplified. A more sophisticated approach would be to do factor analysis of all the items and then give % improved on that factor. In my experience, this usually is very close to the sum of items.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.