How scientifically handle outliers fairly?

I conducted a survey, in which participants ranked themselves depending on how often they did something and how good they were at doing that thing. The input were integer values on the survey and deliberately did not have a maximum (or minimum) range.

Now I'm running some analysis and calculations so that each participant is given a calculated score. This score takes into account a variety of factors and I'm quite happy with that calculation.

The graph below plots the participant scores on the left hand side/Y axis, with each data point representing a participant. The results shown in this graph have been unscientifically butchered to give this nice distribution... (more on this below)

My problem is, that a couple of participants scores were ludicrously high, where most people were between 0-160, some scores were coming out at 2500, 3000. The calculation that produced these scores are just fine, it's just based off survey data input, and some individuals entered unrealistic values, hence a unrealistic score that is an outlier (ie, outside the normal looking distribution of between 0-160).

I need to eventually normalize all scores down to a scale between 0 - 10 for a report, but if I do that now the participant that entered 2500 gets a score of 10, and everybody else will get a score of 0 or 1! I don't know how to scientifically deal with these odd results of 2500, 3000, etc. I can manually cut 2500 down to 160, which would still be 10 in the final analysis, but would mean that everybody else gets fair final scores too. However, if I manually cut these scores down on a case by case basis it seems really unscientific and unfair. I could leave those results out of the report, but again, that seems rather unscientific.

So, how do I handle these outliers, so that I can re-plot all scores from 0-160 (a reasonable distribution)?

Note: I expect that there are really quite well defined solutions to this problem, but I'm afraid my Math experience (and education) is negligible and I'm worried about Googling a solution and implementing something that is quite wrong. Therefore, I've taken the time to state my problem/question in full at risk of asking a duplicate question, just so I've got a higher degree of confidence that you're advising me on the most appropriate way to normalize these results.

Additionally, because my Math is quite bad, can I just say that I'm very keen to learn, but would really appreciate it if you can build up your answer in steps as if you were explaining it to a toddler!

migrated from math.stackexchange.comApr 28 '14 at 16:58

This question came from our site for people studying math at any level and professionals in related fields.