I have a survey in which I asked, "Would you be ready for a t-shirt shop in St Jean Avenue? [yes/no/don't know]". Then I asked many other questions, ultimately including, "Would you be ready for a t-shirt shop in St Jean Avenue knowing that the staff working there is disabled (mentally or physically)? [yes/no/don't know]"

I would like to know:

  1. What's the best way to visualize the two questions?
  2. What's the best statistical test to perform? I would like to know if the people saying yes to the question 1 say yes to the question 2 and the people saying no to question 1 says no to question 2.

Your data should be an ID vector indexing people, and their responses to the questions:

ID Q1 ... Q17
 1  1       1
 2  1       0
 3  0       0

These can be organized into a contingency table:

       1   0
Q1 1  32  15
   0   7  26

There are two ways you could analyze these, depending on what question you want to answer. If you were thinking of these as different ways of asking the same question, and wanted to check for consistency (i.e., are people just marking the sheet randomly to get through it as quickly as possible), you could run Cohen's $\kappa$. However, I doubt that is your question. More likely, you wonder if the addition of the information about the potential presence of disabled workers changes people's opinion. The way to answer this question is using McNemar's test. I have a rather thorough explanation of McNemar's test here: What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each?

Visualizing this won't show too much, because you only have a 2x2 table. Just presenting the table is probably enough. If you wanted, you could make a mosaic plot. Here is an example of such a mosaic plot (copied from my question, Alternative to sieve / mosaic plots for contingency tables, which includes some other possibilities as well):

enter image description here

If you use R, the above could be performed thusly:

tab = with(my.data, table(Q1, Q17))

See: ?table, ?kappa, ?mcnemar.test, and ?mosaicplot.

  • $\begingroup$ Hello thanks for the answer I have a question. McNemar's test seems to be according to Wikipedia for 2x2 matrix. But in my case I have 3x3 matrix cuz i have yes , no and "don't know...". So should I use another test $\endgroup$ – S12000 Mar 1 '14 at 22:03
  • $\begingroup$ I notice that the Wikipedia entry for McNemar's test states at the top that it may be hard to follow. Sorry about that; I just link to Wikipedia by default as a way to give people an initial step towards getting more information on their own. You might try reading this page. Note there is a description of a generalized version of McNemar's test for >2 levels, & evidently there is some free software on the site that you can use to run the analysis, although I haven't tried it. $\endgroup$ – gung Mar 3 '14 at 4:08

I would introduce a latent variables L1 and L2.

  • L1 - desire for t-shirt shop
  • L2 - desire to help disabled

Then you have Q1,Q2 <- L1, and Q2 <- L2

Your questions comprise measurement model. Q1 and Q2 measure L1, and Q2 measures also L2. You can then test your model in SEM framework, e.g. like here in Stata Two-factor measurement model


It would be useful to visualize the two questions in the context of gender or age, or any other pertinent variable.

Could you provide more information about the survey items and purpose? Do you have other similarly paired questions in the survey? It would be useful to measure agreement in those pairs as well.

  • $\begingroup$ This isn't an answer to the OP's question, @Emme. It is a comment / a request for more information. Please only use the "Your Answer" field to provide answers. I recognize it's frustrating, but you will be able to comment anywhere when your reputation >50. Alternatively, you could try to expand it to make it more of an answer. Since you are new here, you may want to take our tour, which contains information for new users. $\endgroup$ – gung Mar 1 '14 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.