Which test to use to check for better ratio for matched participants?

Question

I would appreciate if you can tell me which statistical test to use for the following data analysis:

I have data from 10 people and 10 of their siblings. For each person and her sibling, I have two data points: (1) number of attempts and (2) number of actual hits. So the data look like this:

   name           attempt  hit
person1             300     15
person1_sibling      35      5
person2             125     10
person2_sibling      40      8
etc. 

I would like to test if the siblings perform statistically better, that is, hit / attempt for sibling of a person is statistically different and greater than the respective person.

How can I test this?

Answer 1

It does not look like you could pool the data across individuals because each individual may have a different hit probability. But the binomial model may be applicable to each individual and their sibling match. So you could do a test theat the difference in proportions is 0 for each individual compared to his own sibling using an approriate formula for the standard deviation of the difference between sample proportions. This would give you as many tests as you have subjects and some multiplicity adjustment is needed. Also you need to consider how many times rejcting equality for the sibling performing better constitutes an indication that your theory that the siblings tend to perfrom better is valid.

Answer 2

You can use a generalized linear mixed model (binomial family), with person as a random effect and sibling as a fixed effect.

In R this would be conducted as

data <- data.frame(person=as.factor(c(1,1,2,2)), 
                   sibling=as.factor(c(0,1,0,1)), 
                   attempts=c(300,35,125,40), 
                   hits=c(15,5,10,8))
data$failures <- data$attempts - data$hits
require(lme4)
fit <- lmer(cbind(hits, failures) ~ sibling + (1 | person), family=binomial(), data=data)
summary(fit)

The output gives a (two-sided) p-value of $0.00101$ in this example:

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  -2.7726     0.2062 -13.449  < 2e-16 ***
sibling1      1.2104     0.3682   3.288  0.00101 ** 

The positive estimate indicates the siblings do better. This differs from the result of combining a bunch of chi-squared tests, one for each person:

chisq <- by(data, data$person, function(x) chisq.test(cbind(x$failures, x$hits)), 
                simplify=FALSE) #$
stat <- -2 * sum(unlist(lapply(chisq, function(x) log(x$p.value)))) #$
pchisq(stat, df=2, lower.tail=FALSE)

The separate p-values are $6.9$% and $6.8$%, respectively, which when combined as shown yield a p-value of $0.47$% rather than the GLMM p-value of $0.10$%.

One danger of using the chi-squared tests occurs when the effects for different people are in opposite directions: combining their (two-sided) p-values would be erroneous. There is no such problem with the GLMM.

There is more power and flexibility in using the GLMM compared to conducting the chi-squared tests. Moreover, the GLMM will handle multiple siblings per person and multiple experiments per person without any change; it is difficult to see how to adapt chi-squared contingency table analysis to those generalizations.