# Problem with identifying outliers

I'm running a biological experiment with rodents, have two groups (each consists of 26 animals), where one is treated with a chemical, and one is control (saline).

In one variable, there doesn't seem to be much of a difference between groups. However, when I identify outliers, the data becomes very significant ($p < .01$). I am wondering are my criteria to exclude those data points valid.

I've used two methods to identify outliers. First of all, I was identifying outliers separately for control and treatment, as the data seems very different (check histogram attached for the treatment group)—is this wrong or not? To me, it sounds logical, as the groups are different (in the sense they are chemically treated differently). First method I mentioned—I checked for data that is 2.5 STDEV from the mean, and excluded those points. The second one is similar: http://www.wikihow.com/Calculate-Outliers—this one gives the same result, and identifies the outliers in the treatment group.

However, as I would like to prepare a publication, I need a real "peer-accepted" test for outliers. I have found some tests in StatSoft Statistica, but the box whisker plot does not show outliers there. However, in the Grubbs' test, I get $p < .05$, so it is analytically detected as an outlier, right?

So, my questions are:

1) Is it ok to calculate outliers differently for each group, or should I collapse the sample into one group ($N = 52$) and calculate it then, for both groups together?

2) Which method should I use to identify (and exclude) outliers in order for it to be accepted by a critical review?

I attach the data points here: https://www.mediafire.com/?0qdsifib0hugd9u

To me, it seems that any animal with the score higher than 54 is an outlier.

• Mann-Whitney U test does not assume a symmetrical distribution. If you want to interpret it as a test of medians, then you have to assume that the two distributions have the same shape. Otherwise it is a test of $P(X < Y)=0.5$, where $X$ and $Y$ are random observations from the two distributions. It seems a reasonable test in this case, but I know nothing about your hypothesis. – Aniko Jan 17 '14 at 23:30