Suppose I have a survey where respondents rate different questions as positive (1), negative (-1), or neutral (0), and their responses are given a different weight based on which job they have and which city they live in. Let's say we have questions A, B, C and D, where A and B are grouped and C and D are also group. Suppose we have respondents r1 through r6 rate them (but also suppose we could have different respondents rate each of the questions)

    A   B   C   D   Weight
r1  1   1   1   0   72
r2  0   1   0   0   72
r3  1   -1  0   0   23
r4  0   1   1   1   45
r5  1   0   -1  0   23
r6  -1  1   -1  1   45

Responses for each question are weighted based on the respondent and then added up for a difference between positive and negative ratings (Pos * Weight - Neg * Weight). This result is then divided by the total weights for all respondents (Proportion of pos weights - Prop of neg weights). For example, for question A we would have:

     A  Weight  WeightedA
r1   1      72     72
r2   0      72      0
r3   1      23     23
r4   0      45      0
r5   1      23     23
r6  -1      45    -45

The final score for question A would be $\sum(WeightedA)/Sum(Weight) = 26%$.

In this case, this would mean that question A is more positively viewed by respondents when taking their weights into account. The same is done for other questions so we end up with:

A Score:     26%
B Score:     44%
C Score:    -20%
D Score:      1%

A final score for question group AB is the average 35% and for CD it is -10%.

How can I test the difference between 35% and -10% to understand whether the difference in respondent satisfaction is statistically significant? I was thinking some sort of t-test but not sure what is appropriate in this instance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.