# Which test to use to check for better ratio for matched participants?

I would appreciate if you can tell me which statistical test to use for the following data analysis:

I have data from 10 people and 10 of their siblings. For each person and her sibling, I have two data points: (1) number of attempts and (2) number of actual hits. So the data look like this:

   name           attempt  hit
person1             300     15
person1_sibling      35      5
person2             125     10
person2_sibling      40      8
etc.


I would like to test if the siblings perform statistically better, that is, hit / attempt for sibling of a person is statistically different and greater than the respective person.

How can I test this?

-
Can you tell us what aspect(s) of your data you are interested in. Do you want to make a statement about attempts alone, hits alone, or only percent of attempts that are hits? From the little data you present, it seems the siblings are making fewer attempts by a factor of 10. That might be of interest to you. If so, you could do a paired t-test on attempts. – Joel W. Sep 11 '12 at 19:20

It does not look like you could pool the data across individuals because each individual may have a different hit probability. But the binomial model may be applicable to each individual and their sibling match. So you could do a test theat the difference in proportions is 0 for each individual compared to his own sibling using an approriate formula for the standard deviation of the difference between sample proportions. This would give you as many tests as you have subjects and some multiplicity adjustment is needed. Also you need to consider how many times rejcting equality for the sibling performing better constitutes an indication that your theory that the siblings tend to perfrom better is valid.

-
 Thanks for the reply, Michael. Can you direct me to some source or example to read up on testing difference in proportions? – Vera Sep 11 '12 at 15:05 @Vera Sure almost any introductory statistics text should cover it. Inf you want a book totally on rates and proportions. This book by Fleiss is a good source. – Michael Chernick Sep 11 '12 at 15:18 +1, didn't Fisher have a simple method for combining / averaging p-values? Could that be used here? – gung Sep 11 '12 at 19:47 @gung Fisher's method involves summing -2ln(pi) for i=1 to k when there are k pvalues. Uses fact that under null hypothesis p-values are uniform and -2ln(pi) become independent chi square with 2 df (I think). So the sum is compared to chi square 2k to get combined p-value. – Michael Chernick Sep 11 '12 at 20:24 Makes sense; could that be used here to get a single p-value for the combination of all $k$ binomial tests? – gung Sep 11 '12 at 20:40

You can use a generalized linear mixed model (binomial family), with person as a random effect and sibling as a fixed effect.

In R this would be conducted as

data <- data.frame(person=as.factor(c(1,1,2,2)),
sibling=as.factor(c(0,1,0,1)),
attempts=c(300,35,125,40),
hits=c(15,5,10,8))
data$failures <- data$attempts - data$hits require(lme4) fit <- lmer(cbind(hits, failures) ~ sibling + (1 | person), family=binomial(), data=data) summary(fit)  The output gives a (two-sided) p-value of$0.00101$in this example: Fixed effects: Estimate Std. Error z value Pr(>|z|) (Intercept) -2.7726 0.2062 -13.449 < 2e-16 *** sibling1 1.2104 0.3682 3.288 0.00101 **  The positive estimate indicates the siblings do better. This differs from the result of combining a bunch of chi-squared tests, one for each person: chisq <- by(data, data$person, function(x) chisq.test(cbind(x$failures, x$hits)),
simplify=FALSE) #$stat <- -2 * sum(unlist(lapply(chisq, function(x) log(x$p.value)))) #$pchisq(stat, df=2, lower.tail=FALSE)  The separate p-values are$6.9$% and$6.8$%, respectively, which when combined as shown yield a p-value of$0.47$% rather than the GLMM p-value of$0.10\$%.

One danger of using the chi-squared tests occurs when the effects for different people are in opposite directions: combining their (two-sided) p-values would be erroneous. There is no such problem with the GLMM.

There is more power and flexibility in using the GLMM compared to conducting the chi-squared tests. Moreover, the GLMM will handle multiple siblings per person and multiple experiments per person without any change; it is difficult to see how to adapt chi-squared contingency table analysis to those generalizations.

-
 +1, I appreciate the comparison of the two possible approaches. – gung Sep 12 '12 at 20:17