I was reviewing some game statistics of wins and losses and was wondering what is the best approach to differentiate between how likely a person or team who had come in first is statistically that different from an average team in that league. This led me to wonder an answer to the following conundrum:

Say that there is a test for a trivia contest that must be completed to gain entry and the top choices are picked for an appearance for a live taping. The exact degree of hardness of the questions is not known, but each question is expected to be roughly equivalent in difficulty. If there are 50 questions and Person A answers 30 correct while Person B answers 20 correct, what would be a relatively simple yet good test to see if this is statistically significant difference? (For instance, is just one test of 50 questions good enough, or must more questions be added?)

My first thought was to go to A/B testing and compare it to a somewhat similar example like click-through rate, which led me to things like Fisher's exact test and Barnard's test, but are these the best way to answer the problem above? Since many of you hear have much more statistical training then myself, I thought this would be a good place to ask what method would be the best to recommend.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.