Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two groups (G1, n=10; G2, n = 10) each representing a separate condition. Participants in each group answered 20 questions and each question is a dichotomous variable coded 0 and 1 (VDD).

I want to compare the group 1 with group 2. I am having some trouble understanding if I have it right, for every participants of both group, to mean their answer (since the variable is dichotomous). It would give me a probability to get an answer more than the other one I guess, but I don't know if I have the right to do that.

share|improve this question
There are only stupid answers (-; – mbq Jun 19 '11 at 9:25
Are the 20 answers replicates for the same item, or are there 20 different items with one response for each? – chl Jun 19 '11 at 21:40
No actually it's 20 different items for a given group (but the same for G1 and G2) with one response for each items. – clowny Jun 20 '11 at 17:11
@clowny I think I understand what you are saying; I've tried to tidy up your question to make it a little clearer. What is your dependent variable? (Is it a test with correct and incorrect answers?) – Jeromy Anglim Aug 20 '11 at 4:57

Items measure same thing

  • If your items measure the same thing (e.g., they are all exam questions, or all measuring the presence or absence of a particular characteristic), then you would typically create an overall score for each participant (e.g., you could get the mean score for each participant). With a 20-item test you have 21 different possible scale values, and that's probably enough to use an independent groups t-test as a reasonable option for comparing group means.

Comparing individual items

  • If you just want to compare the two groups on each item, you could do a chi-square test for each item.
  • However with a sample size of 10 in each group, and 20 questions, you are probably going to run into issues related to multiple significance testing (e.g., lots of significance tests, and a high probability of finding an effect by chance, assuming there is no true effect).
share|improve this answer
As you said, here the crucial point is whether the 20 items define an unidimensional scale (which is doubtful, but let's go for it!). – chl Aug 20 '11 at 21:15

You could sum the responses for each individual. Then you could do a simple chi-square analysis with a 2x2 table: Group by VDD. But that's only if you have no other variables to consider. You could also do a nonlinear mixed model, with person being a random effect and group a fixed effect; this would let you add other variables to the model.

share|improve this answer

You have a couple of different approaches that depend upon how you think about the responses to your twenty questions.

If the responses to the question reveal different types of information about the respondents, you may want to think about each particular set of responses as a multivariate random variable.

In this case, you should first create a frequency table of groups by questions. Given the small sample sizes, you should not likely use Pearson's Chi-Square Test of Independence. You can use Fisher's exact test.

If the responses to the questions are all revealing the same type of information, then you can think of the 20 questions as repeated observations. Again, because of your sample size, while you could do a one-way ANOVA with repeated measures, you are probably safer using the Cochran test.

share|improve this answer
Why Fisher's test? – chl Aug 20 '11 at 21:11
With the relatively small sample size, I would worry about the chi-square approximation. I would also suggest testing doing the the 2 by 20 contingency table at once, instead of for each test item. – fgregg Aug 22 '11 at 20:01

*Based on the information provided, its obvious the participants were asked same question, but have different backgrouds. If I may say you are trying to find if answers given by participants from different groups have anything to do with their backgrouds. I also assume you hope to find the probability that an answer given by a participant is most likely to come from a particular group in a given situation. In cases like this, one of the groups is usually used as a control group

In order to compare the two groups of the participants, we need to establish that there is a significant association between two groups with regards to their answers. Chi-square is normally used for this.

Then, once we are convinced that association exists between the two groups; we need to find out how their answers influence their backgrounds .

From your example, say the G1 represent children with formal education and while G2 represents children without formal education. A picture was presented to each child and asked to identify the event in the picture. Those who identified the event in the picture were coded 1 and those who got theirs' wrong were coded 0.

A good model used for this analysis is logistic regression model, given by log(p/(1-p))=β_0+β_1 X,where p is a binomail proportion and x is the explanantory variable.

The parameters of logistic model are β_0 and β_1. In this case, n= 10 samples each group. The explanatory variable is children groups, coded ‘1’ if the children have formal education, ‘0’ if no formal education. The response variable is also an indicator variable which is "occupation identfication" – coded ‘1’ if they were identified correctly, ‘0’ if not. The model says that the probability ( p) that an occupation will be identifed by a child depends upon if the child has formal education(x=1) or no formal education( x = 0). So there are two possible values for p, say, p_(formal education) and p_(no formal education) .

The logistic regression model specifies the relationship between p and x. Since there are only two values for x, we write both equations. For children groups with formal education, log(P_(formaleducation)/(1-P_(formaleducation ))=β_0+β_1 For children groups with no formal education log(P_(noformaleducation)/(1-P_(no formal education) ))=β_0 Note that there is a β_1term in the equation for children group with formal education because x = 1, but it is missing in the equation for children group with no formal education because x = 0.*

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.