8
$\begingroup$

I have a dataset of measurements (non-normally distributed). Each line of this dataset is composed of two measurements of the same variable at a specific time (i.e. XYY [time, meas1, meas2]). This dataset contains 24 lines.

Is it correct to use the Wilcoxon Signed-Rank Test to compare these two measurements? Is it correct to use this test to compare the mean value of these measurements between different experiments?

Thanks

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

One major concern is whether there's serial dependence in the differences, due to the observations being over time. The test assumes independence and if that doesn't hold, it could be badly affected. [The rest of my answer is predicated on independence being sufficiently close to true that the null distribution of the test statistic is not badly affected.]

If you want to test the null hypothesis that the pairs of measurements have the same distribution against the alternative that they don't, the Wilcoxon signed rank test will be suitable (there are some other pairs of hypotheses that would fit with the test).

If you want to specifically test for an alternative of a difference in means, the test will be sensitive to that difference (under a null of identical distributions), but the location-shift estimate it gives is not a shift in means (it's the Hodges-Lehmann estimate, the median of pairwise differences; if you like it's the median shift). If your alternative is explicitly a shift alternative, the Hodges-Lehmann estimate will be a reasonable estimator for the population mean of the pairwise shifts.

(In short, if I understand your description, then yes, it looks like a signed rank test would be suitable, but to make your inference about the means you have to make additional assumptions.)

If your interest is particularly in the mean of the differences, I'd suggest looking at a permutation test; with a reasonably efficient implementation, your sample is small enough to cover all possible permutations.

Improve this answer
$\endgroup$
7
  • $\begingroup$ I don't know if I have understood it right. $\endgroup$
    – ThiagoMAF
    Dec 9, 2013 at 16:05
  • 1
    $\begingroup$ Sorry... continuing... I don't know if I understood it right. My situation is that I'm dealing with biological experiments. I have 2 biological replicates and each one with 2 technical replicates. My goal is to statistically say if the data is valid for further analysis or not. Should I analyse each point (i.e. 4 values) or the whole dataset (i.e. 4 * 24 values).... or, even, just the mean values? $\endgroup$
    – ThiagoMAF
    Dec 9, 2013 at 16:15
  • $\begingroup$ I don't think that "observations come from the same distribution" is the correct description of the null hypothesis in the signed-rank test. $\endgroup$
    – AdamO
    Feb 6, 2014 at 23:55
  • 2
    $\begingroup$ One possible concern is there may be dependence between pair-differences over time (such as serial correlation). $\endgroup$
    – Glen_b
    Jun 11, 2014 at 6:56
  • 1
    $\begingroup$ Another potential concern is that the test is for continuous distributions. If that's not the case, there may be some problems, especially if most of the data occurs in a few values. $\endgroup$
    – Glen_b
    Aug 21, 2014 at 11:19
1
$\begingroup$

I have also been wondering about this. It turns out that this question has cropped up a few times in the recent past:

@Glen_b makes the point that autocorrelation in the data (e.g. the tendency for readings close in time to be close in value) breaks the central assumption of the Wilcoxon Signed-Rank Test that the pairs are independent in your data.

Put another way, if your 100 paired readings are highly correlated over time (as is often the case), then you are not testing 100 independent pairs. Your effective sample size is much smaller. If observation A >> observation B at time 1, the same is likely to be true at time 2, time 3 and so on... depending on the level of autocorrelation in the data.

Now Wilcoxon comes along and sees 100 independent samples that all have the property A >> B. Bingo! A differs from B with dramatically high statistical significance. But in reality, if you were to run the test again you might find that $A \approx B$ at time 1 and this propagates forwards in time as before. Wilcoxon now says that A and B have the same distribution.

So, having laboured this point, I now admit that I have no real idea how to deal with it! A recent study gives a modification to the test to deal with clustered data. For example a left eye - right eye (before and after) test. But our problem only has one 'cluster' - a single time series - so I don't think this will help.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ In general, it would be plausible to suppose that measurements of even a serially correlated variable would be conditionally independent of that variable (although of course whether this is actually the case depends on the type of measurement). Thus the autocorrelation in the data does not necessarily indicate there is any kind of a problem in comparing differences between the two measurements to zero (or any other dataset, for that matter). $\endgroup$
    – whuber ♦
    Mar 9, 2015 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.