8
$\begingroup$

This all pertains to my Psychology honours thesis.

I have two groups (Autism and control) and all participants completed four tasks. It is very important to my study that the groups do not differ on reaction time in each of the tasks. However, they do. The autism group responded faster than the control group. This confounds the results for the construct we actually want to investigate.

I thought I might correct the difference by excluding outliers from the study. I tried to identify outliers at both the univariate (Boxplots, SD = +/- 2.5, for each of the four tasks) and multivariate level (Mahalanobis Dsq). No participant comes up as an outlier. Then I thought I would exclude participants that have low average reaction times ('low' being an arbitrary value), but even so the difference between the two groups was significant.

  • Is there anything else I can do?
  • And how would I report such a process in my thesis?
Improve this question
$\endgroup$
5
  • 5
    $\begingroup$ Could you describe a bit why having fast RTs in these tasks confounds your results? $\endgroup$
    – John
    Commented Sep 13, 2011 at 2:44
  • 1
    $\begingroup$ As hard as it may sound: Assuming your analysis itself is correct, there is not a lot you can do without modifying your data... However, maybe first make a check on assumptions. How big is your sample size? What test did you use? In case of a test with assumptions, did you check them (e.g. normality if the t-test was used)? What is the resulting p-value? With a bit of luck it might be possible to argument against the results of your study ;) $\endgroup$
    – Thilo
    Commented Sep 13, 2011 at 3:22
  • $\begingroup$ Is it already known that subjects with autism who do this test have faster reaction time - have you been looking for one thing and found another? And if they do have a faster reaction time is this noteworthy? $\endgroup$
    – Andrew
    Commented Sep 13, 2011 at 8:28
  • 1
    $\begingroup$ How was the data collected? Are you sure it is accurate? Is there a data entry error / miscommunication that is fouling things up? Are the units of measurement consistent across respondents for your variable(s) of interest? I realize some of these may not be relevant, but the point is to get you thinking about the data itself and trying to understand why it may not fit in with the rest of the data. Perhaps the answer is that the phenomenon you are observing is more complex than initially thought and you're on to some ground breaking research! $\endgroup$
    – Chase
    Commented Sep 13, 2011 at 11:26
  • 3
    $\begingroup$ It sounds like you may have discovered something :-). Why not study the "outliers" rather than throw them away (and thereby lose any right to generalize your conclusions to anybody beyond the subjects you actually studied)? $\endgroup$
    – whuber ♦
    Commented Sep 13, 2011 at 13:17

3 Answers 3

Reset to default
14
$\begingroup$

It is very important that you consider the possibility that the categories of subject have a real difference in reaction times. If that is the case then anything that makes the difference go away will lead to potentially artifactual results. Don't assume that an inconvenient effect is a result of the presence of outliers.

Perhaps you could look for a relationship between reaction time and another outcome measure. The form of the relationship may differ between autistic subjects and normal subjects.

Improve this answer
$\endgroup$
8
$\begingroup$

You should not exclude outliers just because they cause problems, nor should you use a subset of your data because the full data causes problems. Neither of these solved the "problem" in your case, but even if they did, it wouldn't be right.

You haven't given a lot of detail about what you are trying to do or how you are doing it, but can you add reaction time as a covariate?

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Adding it as a covariate was my initial thought too, but I'm seeing a problem with that, one that is conceptual/logical rather than purely statistical. Taking that route would mean testing group differences as if each group possessed an average level of reaction time. Since this is something that is clearly not true in the populations, is it nonsensical to try this angle?...You probably will get additional useful answers if you explain why you believe comparable reaction time to be paramount in your study. $\endgroup$
    – rolando2
    Commented Sep 13, 2011 at 11:40
-1
$\begingroup$

It sounds like you need to explore your data a little more. Why don't you try some unsupervised techniques like clustering. Outliers would show up in their own groups. And you would think there'd be some kind of grouping of your controls.

Regardless, you can still have a thesis about not seeing an effect you expected to see. You'd have to explain how your data/method was not flawed. And add a section about what variables you might add to explain why your test subjects and controls are grouping together. This work still helps future researchers.

Improve this answer
$\endgroup$
2
  • 6
    $\begingroup$ I find it unfortunate that this question received two downvotes (as of writing this comment). While I do not necessarily agree with the respondent in this context, it definitely deserves some comments as to why it is inappropriate enough for a downvote. $\endgroup$
    – Andy W
    Commented Sep 15, 2011 at 19:23
  • $\begingroup$ Andy W : Well Said ! $\endgroup$
    – IrishStat
    Commented Sep 15, 2011 at 23:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.