Removing outliers from data - maximum number of outliers that you can remove? I have a couple of outliers in my data and I was wanting to exclude them to see if this changes the results. In you opinion, what is the maximum number of outliers one should restrict themselves to?
Thanks! 
 A: There is no maximum or minimum.  Outliers should be removed if they are bad data or if there are other substantive reasons for removing them. If there are no substantive reasons, then I suggest using methods that are robust to outliers. I would not remove outliers just because they are a bit far from other points. 
A: I would emphasize on something that was said in an other answer and comments (I think that @Peter Flom's answers is accurate and that EdM is right on touch about measurements, among all ).
Analyzing data is something that must be done carefully. You must be very well aware of the meaning of outliers in your contact. For example, assuming that your measurement procedure was done "correctly" (I mean, you haven't introduced biases, you equipment was calibrated, the person reading the instrument did it correctly, etc. etc.), some outliers may tell something interesting and sometime very important.
Here is a made up example, please be indulgent (point them in comments) if it is not 100% right on all aspects. ;)
Say that someone is testing the effect of applying a certain amount of a substance to some cultures (populations) of bacteria. Now, "in general", the effect is to stabilize the number of bacteria in the population, but there are some outliers among the different cultures. 
Imagine all your outliers indicate situations where all the bacteria are dead. Or that all outliers represent cultures where the bacteria populations have grown out of control.
What I want to point out is that the nature of your perceived outliers might be meaningful and the consequences of each are different. You might be in a situation where it is intolerable that the number of bacteria increase, or decrease.
Of course, if you noticed that some populations where wiped out by the substance, you would probably investigate on the matter since it is an easily recognizable situation. But not all phenomenon are easily detectable.
To wrap up, the notion of outliers is somewhat arbitrary, but their meanings are multiple and of different importance. Hope it will make you think on the matter... :)
