2
$\begingroup$

I want to compare two types of designs. After showing them the images of two designs, I have asked the user (7 questions), which design they like most (A or B)? The user can chose only 1 out of 2 options (A or B).

Example of the data collected is as follows:

Sub A   B
1   5   2
2   5   2
3   6   1
4   7   0
5   4   3
6   7   0
7   7   0
8   6   1
9   6   1
10  6   1
11  7   0
12  5   2
13  6   1
14  0   7
15  6   1
16  7   0
17  7   0
18  7   0
19  7   0
20  7   0
21  6   1
22  7   0
23  2   5
24  7   0
25  7   0
26  1   6
27  5   2
28  6   1
29  6   1
30  7   0

My options are as follows:

  • Can I use a paired t-test to find if there is a significant difference between the two choices?
  • Should I use one-sample t-test (e.g. for A, Hypothesised mean = 3.5)?
  • Should I go for chi-square test?

Which of the three above options is the best considering the data that I have provided?

$\endgroup$

1 Answer 1

1
$\begingroup$

Of those three, I would say a paired t-test is best. But, if it were my analysis, I would use a nonlinear multilevel model with count responses.

$\endgroup$
4
  • $\begingroup$ Thanks Peter for your quick response. Is it possible for you to send me some links where I can see the examples of how 'nonlinear multilevel model with count responses' can be applied? $\endgroup$
    – Tony
    Commented Sep 4, 2017 at 18:47
  • $\begingroup$ In R, there is the nlme package. In SAS there is GLIMMIX. I don't know other software. Both of those packages have examples. These models have been discussed here on CrossValidated. $\endgroup$
    – Peter Flom
    Commented Sep 4, 2017 at 20:04
  • 1
    $\begingroup$ A t-test (paired or non-paired) is not at all applicable here, and neither is a Chi-Square test. This is a Binomial (n,p) distribution with n=7 and you are testing for p - whether p=0.5 or not. An approximation would be a (one-sample, looking only at the A counts) Z-test (rather than a t-test) in which you are testing if μ=3.5 or not, and the standard deviation is 1.323 (the square root of 7 x 0.5 x 0.5). A sample size of 30 is considered sufficient for the Normal approximation. $\endgroup$
    – Zahava Kor
    Commented Sep 4, 2017 at 20:09
  • $\begingroup$ The total under B s simply 7 minus the total under A. What you have then is that the information in each subjects response is entirely contained in the count under A. The result under B is perfectly negatively correlated with it and trying to do a paired test in that situation would seem to be counterproductive. I agree that some kind of model incorporating a random-effect would make sense, though. $\endgroup$
    – Glen_b
    Commented Sep 5, 2017 at 2:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.