1
$\begingroup$

My question might sound strange but this is the situation I'm dealing with! I have a dataset, consisting of 4 data series, each a measurement of a parameter of a biological sample. We have 31 samples. So we have a matrix of 4x31. I need to compare each series with the other one statistically and find the correlations and dependency of the series. So I ran Shapiro-Wilk and d'Agostino-Pearson tests to see if the data are normally distributed. The result of both tests on each of 4 series was positive and my data are normal. I also checked for outliers using Box and Whisker plots and the tests described here It turned out that there is an outlier in the 1st series (out of 4) in my dataset. Now my question is that if I am allowed to run t-test (knowing it's sensitivity to outliers) with my current data considering that the normal distribution of the data is confirmed through those 2 test? Or I have to reconsider usage (in this case, remove) the outlier and run my ANOVA and T-Test with series without outliers?

Thank you in advance

$\endgroup$
2
$\begingroup$

First, if the data is normal it doesn't have outliers. The results you describe are due to deficiencies in the tests of normality and tests for outliers, especially when you evaluate the results as yes/no based on significance.

Second, never remove outliers just because they are outliers. Remove them if they are data entry errors.

Third, I am a bit worried about your use of the term "series". If these are longitudinal series, I'd be very careful about doing anything and would ask a different question emphasizing that aspect and using the time-series tag.

Fourth, rather than change the data, change the method. Use methods that don't rely on assumptions that are violated. You could, for example, use Mann Whitney test and Spearman rank correlations - however, whether you should use those or something else depends on exactly what you want to find out.

$\endgroup$
  • $\begingroup$ Dear Peter, Thanks for the explanations. I am not a statistical professional, so I might have made a mistake in my calculations. I would be very glad if you could take a look into my dataset available here: 1drv.ms/x/s!ArHvYi6xwZfdgRPU2P_gpLlu7urT I have highlighted the cells which confirm that the data are normal and also the cells with red highlights or tick black borders are recognized as outliers. I would love to know your opinion about this. I'm working on non-parametric tests and hope I can have a better approach to deal with my "outliers". $\endgroup$ – Hadi Jun 15 '18 at 10:09
  • 1
    $\begingroup$ Well, you might have made a calculation error, but you could certainly get the results you got without one. It's a flaw in the whole idea of answering questions about normality and outliers with a simple Yes/No. $\endgroup$ – Peter Flom - Reinstate Monica Jun 15 '18 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.