0
$\begingroup$

Suppose I have a survey where respondents rate different questions as positive (1), negative (-1), or neutral (0), and their responses are given a different weight based on which job they have and which city they live in. Let's say we have questions A, B, C and D, where A and B are grouped and C and D are also group. Suppose we have respondents r1 through r6 rate them (but also suppose we could have different respondents rate each of the questions)

    A   B   C   D   Weight
r1  1   1   1   0   72
r2  0   1   0   0   72
r3  1   -1  0   0   23
r4  0   1   1   1   45
r5  1   0   -1  0   23
r6  -1  1   -1  1   45

Responses for each question are weighted based on the respondent and then added up for a difference between positive and negative ratings (Pos * Weight - Neg * Weight). This result is then divided by the total weights for all respondents (Proportion of pos weights - Prop of neg weights). For example, for question A we would have:

     A  Weight  WeightedA
r1   1      72     72
r2   0      72      0
r3   1      23     23
r4   0      45      0
r5   1      23     23
r6  -1      45    -45

The final score for question A would be $\sum(WeightedA)/Sum(Weight) = 26%$.

In this case, this would mean that question A is more positively viewed by respondents when taking their weights into account. The same is done for other questions so we end up with:

A Score:     26%
B Score:     44%
C Score:    -20%
D Score:      1%

A final score for question group AB is the average 35% and for CD it is -10%.

How can I test the difference between 35% and -10% to understand whether the difference in respondent satisfaction is statistically significant? I was thinking some sort of t-test but not sure what is appropriate in this instance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.