2
$\begingroup$

Background

I have two conditions: A and B, with around 400 measurement points for each condition.

The most common result for both conditions is around '700', and neither one has a result below '600'.

Condition A has a long tail with results up to 10,000, while condition B has a longer tail with results up to 20,000.

Both the dependent variable and its residuals are not normally distributed (or even close to it).

What I'm trying to do

I want to test whether there is a difference in mean between the group, and whether there is a difference in the distribution, when outliers are taken into account.

Since the dependent variable/residuals are not normally distributed, I thought about using a non-parametric test such as Kruskal Wallis. However, since this test is rank-based, it means that I lose the difference due to outliers, which is crucial to the current analysis. The same happens when I perform a log-transformation (which also doesn't help reach a normal distribution anyway).

What can I do in this situation?

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you considered taking bootstrap-samples from both conditions and compute their means? That should give you a good picture of what you know about the means. Those bootstrapped means should be roughly normally distributed in both conditions, so a z-test should be possible. $\endgroup$
    – Bernhard
    Commented Aug 7, 2018 at 10:52
  • 1
    $\begingroup$ I could not understand this: "and whether there is a difference in the distribution, when outliers are taken into account". Do you want to compare groups whether their distributions are different shape? Or compare distributions original vs trimmed? How do you define "outliers" or what do you mean by "taking account" of them? $\endgroup$
    – ttnphns
    Commented Aug 7, 2018 at 12:38

2 Answers 2

Reset to default
3
$\begingroup$

You can still compare the means of the distributions, e.g. using the $t$-test, even if the distributions are highly skewed. This is because, according to the CLT, the means of samples drawn from a distribution will be normally distributed, even if the underlying distribution is not normal.

However, in skewed distributions, means are not very interesting, because they are strongly influenced by outliers, and can be far away from the distribution's central tendency. Therefore, in practice, skewed distributions are most commonly described by a median and an IQR, instead of a mean and a standard deviation. The standard test to compare two skewed distributions is the Wilcoxon-Mann-Whitney test.

Improve this answer
$\endgroup$
2
$\begingroup$

This is a standard application for a .

First, calculate the difference in means you observe.

Then, randomly permute the condition labels A and B. Calculate the difference in means between these randomly labeled data. Do this many times, e.g., 1,000 or 10,000 times. This process simulates the distribution of means under the null hypothesis of no effect of the condition.

Finally, insert the actually observed difference into this distribution of means. If you want a p value, calculate the tails of the empirical cumulative distribution of your permuted differences at the observed difference.

This is typically one of the first illustrations of permutation tests in standard textbooks. I believe this exact example is treated in the first chapters of Good's Permutation, Parametric, and Bootstrap Tests of Hypotheses.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.