1
$\begingroup$

I have a small-sample dataset representing observations from a longitudinal study. My principal interest is in 'change scores' across three parameters (A, B, C). This requires simple paired t-tests. However, applying the median absolute deviation rule, I've found that the change scores for each parameter contains a large number of outliers (30-45%).

This represents a substantial amount of data relative to the full sample, and thus my concern. I have several questions I'd appreciate any comment on:

  • Is there a rule for when removal of data from outliers is too much (i.e., where outliers represent too great a proportion of the full dataset)?
  • How should I proceed with analysis? A regular test with all data? A test with data removed? Or a robust t-test using trimmed means and winsorized variance?

Example figures:

Histogram of Parameter A Histogram of Parameter A

Parameter A vs Parameter B Parameter A vs Parameter B

Improve this question
$\endgroup$
12
  • $\begingroup$ this 30%-45% number, do you obtain it by flagging an observation as outlier whenever it is flagged as outlier on either one of the the three parameters? $\endgroup$
    – user603
    Aug 23, 2018 at 8:50
  • $\begingroup$ Yes. For example, the change score for parameter A has 30% outliers, parameter B has 40%, and parameter C 35%. $\endgroup$
    – pomodoro
    Aug 23, 2018 at 10:47
  • $\begingroup$ so if you had only parameter A to apply the mad rule on, you would have 30% outliers? Is this what you mean? Also, can you post the result of doing length(unique(x))/length(x)) when, again, x is just the first parameter ('parameter A')? [length(unique()) counts the number of different values, so length(unique({1,1,1,2})) is 2] $\endgroup$
    – user603
    Aug 23, 2018 at 12:17
  • 1
    $\begingroup$ Yes. If only looking at the change scores for parameter A, I would obtain 30% outliers following the MAD rule. The result of your function is 1: all change score observations are different. $\endgroup$
    – pomodoro
    Aug 24, 2018 at 2:41
  • $\begingroup$ ok. These 30% outliers, when you plot them (histogram of parameters individually or --even better-- considering plots of A vs B and A vs C and B vs C) do they form a cohesive group (are all or a large proportion of the outliers bundled together?) $\endgroup$
    – user603
    Aug 24, 2018 at 5:50

1 Answer 1

Reset to default
3
$\begingroup$

Removal of outliers based on simplistic rules without any assessment is generally inappropriate. Any inappropriate removal of outliers is too much and may raise suspicions of scientific misconduct - i.e. that this was done to get a desired result. Of course, it may be very appropriate to remove outliers that are due to failure of measurement equipment.

If you have too many apparent outliers compared to what you would expect under the distribution you assume, then perhaps you are assuming a distribution that does not suit your problem. What was used for other publications in your field may give you an idea of whether other people have identified some well- suited analysis approach already.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.