Based on my data, is Jack likely to be clumsy? I posted this on mathoverflow, but they sent me here. This question relates to a problem I had at work a while ago, doing a little data mining at a car rental company. Names changed, of course. I'm using Oracle DBMS if it matters.
There was a flight of steps out the front of our building. It had a dodgy step on it, on which people often stub their toes.
I had records for everyone who works in the building, detailing how many times they climbed these steps and how many of these times they stubbed their toes on the dodgy step. There's a total of 3000 stair-climbing incidents and 1000 toe-stubbing incidents.
Jack climbed the steps 15 times and stubbed his toes 7 times, which is 2 more than you'd expect. What's the probability that this is just random, vs the probability that Joe is actually clumsy?
I'm pretty sure from half-remembered statistics 1 that its something to do with chi-squared, but beats me where to go from there.
...
Of course, we actually had several flights of steps, each with different rates of toe stubbing and instep bashing. How would I combine the stats from those to get a more accurate better likelihood of Joe being clumsy? We can assume that there's no systematic bias in respect of more clumsy people being inclined to use certain flights of steps.
 A: chisq.test(c(15,7),p=c(3000,1000),rescale.p=TRUE)

    Chi-squared test for given probabilities

data:  c(15, 7)
X-squared = 0.5455, df = 1, p-value = 0.4602

There is not enough evidence against the Null hypothesis (that is just a random incident).
A difference from the expected value as big as or bigger than the one observed will arise by chance alone in more than 46% of cases and is clearly not statistically significant.
The chi-squared value is
sum((c(15,7) - 22*c(3000,1000)/4000)^2 / (22*c(3000,1000)/4000))
[1] 0.5454545

and the p-value comes from the right-hand tail of the cumulative probability function of
the chi-squared distribution 1-pchisq with 1 degrees of freedom (2 comparisons -1 for
contingency; the total count must be 22)
1-pchisq(0.5454545,1)
[1] 0.460181

exactly as we obtained using the built-in chisq.test function, above.
EDIT
Alternatively, you could carry out a binomial test:
binom.test(c(15,7),p=3/4)

        Exact binomial test

data:  c(15, 7)
number of successes = 15, number of trials = 22, p-value = 0.463
alternative hypothesis: true probability of success is not equal to 0.75
95 percent confidence interval:
 0.4512756 0.8613535
sample estimates:
probability of success
             0.6818182

You can see that the 95% confidence interval for the proportion of successes (0.45, 0.86)
contains 0.75, so there is no evidence against a 3:1 success ratio in these data. The p value is slightly different than it was in the chi-squared test, but the interpretation is exactly the same.
A: Perhaps I'm misunderstanding the OP, but shouldn't the test be for 8 successes (of not stubbing the fellow's toe) and 7 failures, for a total of 15 trials? And shouldn't the comparison be to a probability of 2/3 (2000 incidents of not stubbing a toe relative to the 1000 observed ascents)? This, of course, is taking the true population probability of stubbing a toe to be 1/3. 
One way to permit both quantities to be random is to bootstrap. If stubbing is 1 and not stubbing is 0, draw (3000-15) observations with replacement from the total pool. Note that we exclude Joe from the total pool in order to compare him to others. Sum the observations (i.e., count the number of toe stubs) and divide by the total number of observations, 3000-15. Draw 15 observations from Joe's sample, sum, and divide by 15. Subtract Joe's proportion of stubs from the population proportion. Simulate many (1000, 10000 maybe) times, look at the 2.5 and 97.5 percentiles (for a 95% test against the null hypothesis that Joe's stub rate is the same as that of the population; a different level or one-sided test could also be used as desired) and see whether 0 is in that interval. If it is, then you cannot reject the null hypothesis that Joe is as clumsy as everyone else.
