1
$\begingroup$

I conducted a survey, in which participants ranked themselves depending on how often they did something and how good they were at doing that thing. The input were integer values on the survey and deliberately did not have a maximum (or minimum) range.

Now I'm running some analysis and calculations so that each participant is given a calculated score. This score takes into account a variety of factors and I'm quite happy with that calculation.

The graph below plots the participant scores on the left hand side/Y axis, with each data point representing a participant. The results shown in this graph have been unscientifically butchered to give this nice distribution... (more on this below)

Distribution of scores

My problem is, that a couple of participants scores were ludicrously high, where most people were between 0-160, some scores were coming out at 2500, 3000. The calculation that produced these scores are just fine, it's just based off survey data input, and some individuals entered unrealistic values, hence a unrealistic score that is an outlier (ie, outside the normal looking distribution of between 0-160).

I need to eventually normalize all scores down to a scale between 0 - 10 for a report, but if I do that now the participant that entered 2500 gets a score of 10, and everybody else will get a score of 0 or 1! I don't know how to scientifically deal with these odd results of 2500, 3000, etc. I can manually cut 2500 down to 160, which would still be 10 in the final analysis, but would mean that everybody else gets fair final scores too. However, if I manually cut these scores down on a case by case basis it seems really unscientific and unfair. I could leave those results out of the report, but again, that seems rather unscientific.

So, how do I handle these outliers, so that I can re-plot all scores from 0-160 (a reasonable distribution)?


Note: I expect that there are really quite well defined solutions to this problem, but I'm afraid my Math experience (and education) is negligible and I'm worried about Googling a solution and implementing something that is quite wrong. Therefore, I've taken the time to state my problem/question in full at risk of asking a duplicate question, just so I've got a higher degree of confidence that you're advising me on the most appropriate way to normalize these results.

Additionally, because my Math is quite bad, can I just say that I'm very keen to learn, but would really appreciate it if you can build up your answer in steps as if you were explaining it to a toddler!

$\endgroup$

migrated from math.stackexchange.com Apr 28 '14 at 16:58

This question came from our site for people studying math at any level and professionals in related fields.

2
$\begingroup$

If you have a solid reason to believe that the outliers are due to errors (as you seem to; also, double check to see if it is a fixable error such as a data entry issue), you have justification for removing them from analysis. You do, however, need to provide details (values, zscores, etc.) about the removed outliers in your final report and explain how you determined that they were caused by errors (i.e., make it easy for a reviewer to assess your methodology).

If the data points are not outliers because of some error, then it is more complicated. You will often have outliers in your data because that is just the nature of a normal distribution. In this case, it is kind of wrong to take them out but, at the same time, leaving them in can skew your results if you analyze your data with standard statistical approaches. An option often taken by researchers is to winsorize (instead of trimming) the data to 3 st. deviations. You should decided ahead of time what approach you want to take (i.e., don't try all the possible methods and then use the one that gives you the best result) and then report the details.

Robust statistical methodology can also be used to analyze data without removing the outliers.

$\endgroup$
  • $\begingroup$ This is an elegant answer to an inelegant question, that pointed me at Winsorising which I've not heard of before. Serendipitously, I also came to the same solution while discussing this with a colleague, thank you so much for taking the time to write this answer, it reassuringly adds credibility. $\endgroup$ – xconspirisist Apr 28 '14 at 18:30
  • $\begingroup$ Winsorization based on SDs would be a statistically suspect (and I hope unusual) procedure because the outliers themselves can radically affect the SD. The standard meaning of "Winsorize" does not involve SDs: instead, data in the extreme percentiles (often the top and bottom 5%) are replaced by the values at the threshold percentiles. The Wikipedia article makes a point of distinguishing Winsorizing from trimming and mentions that Winsorizing is chosen for its robust qualities--which rules out the use of SDs altogether. $\endgroup$ – whuber Apr 28 '14 at 21:18
  • $\begingroup$ Aghh.. yep; you are right on, whuber. Thanks for correcting my error! $\endgroup$ – captain_ahab Apr 29 '14 at 4:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.