Let's assume I have a control and a treatment group with 1000 participants each and I ask each of them three questions regarding their risk perception. Each of the three questions is concerned with another domain: a, b, c

Furthermore I am also interested in the overall risk perception and I assume that this can be inferred from the answers to a, b, and c.

Unfortunately I missed to ask a fourth question concerning the overall risk perception ( That's why I want to know if this alternative way might work or not)

In a first step I check for significant differences between treatment and control group for each variable using a chi squared test.

That is:

Variable A:
Treatment:  900 yes / 100 no
Control:       847 yes / 153 no

Variable B:
Treatment: 948  yes / 152 no
Control:      888 yes / 112 no

Variable C:
Treatment: 808 yes / 193 no
Control:      760 yes / 240 no

I find that the differences are significant for A and B but not for C.

Now I am wondering if it is possible to construct one general variable based on A,B C-

  1. Is it statistically meaningful to construct a new variable 'overall' that takes on the average value from a, b and c?

That is:

Variable Overall (average from A , B and C)
Treatment:  885 yes / 115 no
Control:       832 yes /  168 no
  1. If 1 makes sense, can I test for significant differences for this constructed variable using a chi squared test?

1 Answer 1


Tricky! Coming from pyschometrics, I would say that it should be possible to build a so called "scale", which is basically the sum of the scores on several questions; in your case 0 or 1.

A person then could have a score of 0, 1, 2, 3. On the total scores of a person then you could perform a t-test for two groups.

For this to make sense you would have to show that the scale you created is consistent. (In your case look for Kuder–Richardson Formula 20) This means that the items in the scale are coherent.

Very loosely this means that if a person scores 1 on say item A, then generally there is a bigger chance of scoring 1 on B or C. This is the same as saying that all the items represent parts of risk perception, and it cannot be that a high score on A generally leads to a low score on B and C; this would be then as adding random noise. Beware of negations in the questions!

  • $\begingroup$ ok, do you know how to handle missings in this setting? E.g. variable B is only answered by 800 participants and variable C by 600. What is an appropriate way to deal with this? $\endgroup$
    – Tom
    Dec 2, 2015 at 15:08
  • $\begingroup$ That is a different question. Key would be to find out if there is reason for these answers to be missing. If not, you could only count complete answers. Be sure to report this with the final results. $\endgroup$
    – spdrnl
    Dec 2, 2015 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.