# Comparing Group and Expert Result

Say I am predicting swimming times. I get a group of 20 to make predictions on the fastest to slowest swimmers from 1 to 6. I then ask a single expert in swimming to make the same predictions. Is there a way to test statistical significance between the group's predictions versus the expert's?
"significant" would mean if I were to select a random person from the group there's a very low percent he would predict in the same order as the expert.

ex.
We rank 1 to 6 the swimmers who received the highest to lowest average predictions (10 people predict swimmer 1 to get first, 10 predict swimmer 1 to get second, no one predicts swimmer to get other; his average is then 1.5, average of the predictions. We do this for every swimmer and rank the first 6 according to average). Now we have our top 6 from the group, how would I determine significance (as defined above) of an expert's prediction on the same swimmers.

• Different in what sense? Can you give an example of a "significant" and a "nonsignificant" difference? That might help clarify your intention here Aug 12, 2016 at 4:25
• Different in the sense that if I were to choose 1 person from my group randomly what are the chances his prediction would match the expert's prediction given the distribution of the entire group. I guess "significant" would mean if I were to select a random person from the group there's a very low percent he would predict in the same order as the expert. Aug 12, 2016 at 8:03
• Thanks for clarifying. You should edit that into the body of the question itself; comments are not searchable, linkable, or archived in the same way that question text is. Aug 12, 2016 at 14:20
• "Significance" does not seem to be an applicable concept. You seem to be asking for a way of measuring or assessing the discrepancy between the actual outcome and any particular prediction of it. Since there are an abundance of solutions, what additional information can you supply to narrow them down? Could you explain how you plan to use this measure of predictive accuracy, for instance?
– whuber
Aug 12, 2016 at 18:39
• The goal of the problem is to find if there is a difference between the amateur group and the expert. There is no actual outcome yet, we are just checking if there is a difference. We want a score that can tell us how different an expert's result is compared with the amateur group's predictions. In a sense it is a classification problem, but I'm not sure how to go about it, since rankings are dependent of one another. Aug 12, 2016 at 19:43