3
$\begingroup$

I am trying to conduct a 3 x 2 mixed ANOVA (my measure is reaction time data) and my data is currently not normally distributed. I have read that ANOVA is robust to the violations of normality so I wouldn't need to transform my data.

However, wouldn't I need to identify outliers since it is reaction time data? How would I identify outliers if my data is not normally distributed?

  1. How do I identify outliers for data that isn't normally distributed? Once identified, should I transform them or delete it? (since reaction time data is susceptible to outliers due to people falling asleep in experiments all the time)

  2. Would I still need to transform my data for the ANOVA?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Normal distributions do have outliers. By removing them, you may make your data even less normal. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Jan 1, 2017 at 9:40
  • $\begingroup$ Do people not normally take logs of reaction times? $\endgroup$
    – mdewey
    Commented Jan 1, 2017 at 13:35
  • 1
    $\begingroup$ If you're willing to delete outliers--which can be a radical alteration of your dataset--then why not take the next logical step and just make your data into whatever values you like so that they look normal? ;-) $\endgroup$
    – whuber ♦
    Commented Jan 1, 2017 at 18:08
  • $\begingroup$ @justanotherlayman I read Bill Huber's comment as sarcasm. I hope my answer has convinced you that deleting every apparent outlier only to make the data look like it is normally distributed is wrong. $\endgroup$
    – Michael R. Chernick
    Commented Jan 1, 2017 at 18:27
  • $\begingroup$ @anony-mousse so why do people remove outliers if that's the case? i always thought that removing outliers improves normality and hence, improves the validity of the results? but i maybe wrong again... $\endgroup$
    – user143911
    Commented Jan 2, 2017 at 2:53

1 Answer 1

Reset to default
4
$\begingroup$

Okay. You are off to a good start. Rupert Miller Jr. in his book Beyond ANOVA: Basics of Applied Statistics published by Chapman and Hall 1997 as a reprint of an earlier publication, points out in fairly simple language the assumptions in ANOVA and what happens when there are departures from the underlying assumption and this includes specific robustness properties. So it may be that if ANOVA is robust enough in your case, you don't need to worry about the non-normality and so do not need to transform the data.

But in your case, you know that people fall asleep sometimes while they should be reacting to a stimulus. It would seem that the observer should be able to note when this happens and you can eliminate the cases when this occurs. If this is the case you do not need a statistical technique to identify and remove the outliers. You then need to be satisfied that what remains provides an adequate sample size for the ANOVA. Also, you want to be convinced that the people you eliminate for sleeping would not have very different reaction times compared to the ones that remain.

Some comments about outliers: I have a lot of experience researching outliers and have also applied outlier detection methods when validating Department of Energy databases. I think it is important to distinguish between outlier detection and outlier rejection. Outlier detection helps you identify when observations depart significantly from the bulk of the data.

I view outlier detection as being acceptable since it leads you to check for possible errors or other causes for the outlier. If the error can be confirmed, there is justification for removing it.

Detecting outliers in some situations can tell you a lot about possible problems with your experiment. But arbitrary removal is dangerous because you are forcing the data to conform to some preconceived notion as to what your data should look like.

As you become more comfortable with these subjects do look at some of the posts here about ANOVA, transformations and outliers. Also Wikipedia can often be helpful as well as some elementary books and articles on outliers. Besides ANOVA, Miller's book deals with transformations, and one sample and two sample problems.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thank you so much for your wonderful answer. I'm a little dubious about removing outliers as well, especially without a good justification. We can only assume that the participant is falling asleep but we can never really be sure as they are left alone in the room to do the experiment. Therefore, after reading your comment, is it acceptable if i do a transformation to my data to "correct" for outliers? Thanks again. $\endgroup$
    – user143911
    Commented Jan 2, 2017 at 2:48
  • $\begingroup$ @justanotherlayman Is it fair to say that falling asleep doesn't negate the reaction time but just makes it unusually long? In that case if you want to transform to normality a power transformation might be okay. You could try the Box-Cox approach. $\endgroup$
    – Michael R. Chernick
    Commented Jan 4, 2017 at 1:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.