I need some advice on what is a reasonable number of cases to be deleted as outliers.
I have applied outlier detection methods to identify univariate and multivariate outliers from my dataset. Alltogether 30% of the data was classified as outliers.
If I delete all of these outliers, my results appear to improve. Also, after deleting the outliers my sample size is still good (i.e., n=300).
I would be more than suspicious, if someone told me that 30% of my sample are outliers ...
Rather than blindly trusting a canned routine I would carefully analyze the data and try to find out why an outlier is an outlier. Is it a "bug" or a "feature"? Is it measurement error? Does your sample cover different sub-populations (mixture)?
Moreover, the detection of outliers involves the more or less arbitrary definition of a threshold, which separates "good" and "bad". You should assess if these thresholds are sensible. It could thus be a good idea to move the goalposts and to see what happens.
Also note that rather than dropping observations, you could use robust statistical techniques if you are concerned about outliers.
Absolutely not: Outliers are points that are distant from the bulk of other points in a distribution, and diagnosis of an "outlier" is generally done by comparison to some assumed distributional form. Although outliers can occasionally be caused by measurement error, diagnosis of outliers can also occur when the data follows a distribution with high kurtosis (i.e., fat tails), but the analyst compares the data points to an assumed distributional form with low kurtosis (e.g., the normal distribution).
The entire concept of an "outlier" really does far more harm than good. All that is really needed is to recognise that it's okay to remove data points that have been measured incorrectly, but it's not okay to remove data points that are legitimate observations (except for the limited purposes of sensitivity analysis). Unless the statistical analyst has evidence to conclude that an "outlier" has occurred due to measurement error, it is almost always the case that it is identified because the data follows a distribution with high kurtosis (i.e., fatter tails) than the assumed distributional form. To conclude that this reflects some problem with the data is tantamount to claiming that reality must conform to your statistical assumptions, and when it does not it has made an unfortunate error, that you will rectify in your analysis by removing parts of reality that are non-compliant with your assumptions.
In any case where an analyst identifies a large amount like 30% of the data as "outliers", it is likely either that the outlier test has been incorrectly applied, or the outlier test is based on a distributional assumption that assumes much thinner tails than the data, and is therefore falsified by the data. In either case, it is a sure sign that something has gone wrong. Personally, I would never trust any analysis that has removed a large proportion of the data as "outliers".
In view of this, I would suggest that you first consider whether there are any data points that have incorrect values due to measurement error. If you have good reason to think this has occurred, it is legitimate to remove these and note their removal in your analysis. (Bear in mind that unless the people making the observations are extremely incompetent, then realistically you should not have measurement errors for more than a small number of your points.) If you still find you have high numbers of "outliers" then this almost certainly means you are using a statistical model with a distribution that has thinner tails than is warranted by the data (e.g., you are assuming a normal distribution, but there is substantial excess kurtosis). Find the sample kurtosis of the residuals in your data and compare this to your assumed distributional form to check. If your assumed form does not match the data, consider replacing this with a distribution with higher kurtosis (e.g., you might replace the normal distribution with a t-distribution or generalised error distribution).
