I have two paired samples.

$\text{Group 1}: X_1,X_2, ...,X_n$
$\text{Group 2}: Y_1,\;Y_2,...,\;Y_n$.

$(X_1, Y_1)$, $(X_2, Y_2), ..., (X_n, Y_n)$ are paired respectively.

The null hypothesis is $\mu_{|X-Y|} = 0$,
the alternative hypothesis is $\mu_{|X-Y|} > 0$

What is the appropriate test for this purpose?

  • $\begingroup$ Like a (two-tailed) paired t-test, Wilcoxon signed rank test or sign test, for example? What aspect of the distribution of the difference are you most interested in? $\endgroup$ – Glen_b -Reinstate Monica Jun 6 '14 at 3:35
  • $\begingroup$ I am interested if the mean of the absolute difference significantly different from zero or not. $\endgroup$ – Q_Li Jun 6 '14 at 3:59
  • 1
    $\begingroup$ If you have data that's roughly normalish, I'd suggest a paired-t-test. If you don't, there are a variety of choices, but if the differences aren't too skew, I'd lean toward a permutation test. $\endgroup$ – Glen_b -Reinstate Monica Jun 6 '14 at 4:45
  • $\begingroup$ Could you remind me how permutation test can be used here $\endgroup$ – Q_Li Jun 6 '14 at 14:37
  • $\begingroup$ Sure I'll expand my comments to an answer. $\endgroup$ – Glen_b -Reinstate Monica Jun 7 '14 at 1:05

In comments you specify an interest in the alternative $\mu_D\neq 0$, I'll discuss several possibilities:

1) paired t-test: suitable if the differences are approximately normal. Moderately tolerant of non-normality.

2) Wilcoxon signed rank test: effectively a permutation test on the mean of the signed ranked differences. Assumes symmetry of the differences (and under that assumption, works as a test for the mean difference being zero).

3) Sign test. Does not assume symmetry, but is not a test for the mean difference being zero if you don't have symmetry. Since it tends to have lower power unless the tails are very heavy, if you're going to assume symmetry you'd usually be better off with (2).

4) permutation test. This has the same assumptions as (2) - independence, symmetry of differences, but it has better power at the normal than (2) (so if you're really having trouble working out which of (1) or (2) to do, you should probably do (4) instead. If your tails are heavier than normal, say logistic-ish, you might be slightly better off with (2) from a power point of view. On the other hand, the loss of power at the normal if you do (2) is surprisingly small, so there's not all that much to worry about either way.

The permutation test works as follows:

If the null is true and the distribution of differences is symmetric, then the differences would be as likely to have one sign as the other - under the null hypothesis, the signs are arbitrary.

So you can generate pseudosamples by taking the absolute differences and giving each one a random sign.

For each pseudosample you calculate some test statistic which will tend to be unusual compared to the null case (i.e. is sensitive to the alternative). One example is the sum of the signed differences.

If $n$ is the number of differences, then there are $2^n$ such sets of signs. If $2^n$ is tractable (say $n<20$ or so), you can compute all possible arrangements of signs.

If $n$ is too big to find them all, you can randomly sample the sets of signs (making it a randomization test rather than a permutation test).

Either way, you end up with a distribution for the test statistic. You find where your sample statistic lies in the distribution. If it's in either extreme tail (say the top of bottom $n\alpha/2$ values, you would reject the null.

  • $\begingroup$ Thanks for your answer! However, as I mentioned in the question, I am interested in the absolute change of these two groups. The tests in your answer are all used for testing the equality of the two group means. In the case that half of samples are increased and the other half of the samples are decreased, a test for testing the equality of two group means will probably give a non-significant p-value. However, I suppose if we test the ABSOLUTE CHANGE, the result will be very likely significant. $\endgroup$ – Q_Li Jun 7 '14 at 23:17
  • $\begingroup$ Your original phrasing was ambiguous. It suggested to me that you were after a difference in either direction - I even asked you to confirm the kind of thing with my initial comment, to which your response merely repeated the same somewhat ambiguous information (a simple 'no, I don't mean that' would have been more help), again leaving me guessing. Please edit your question to include an algebraic definition (in terms of what you have measured) of what you want to compare, and a clear statement of your null and alternative hypotheses, and I will modify my answer. $\endgroup$ – Glen_b -Reinstate Monica Jun 8 '14 at 0:07
  • $\begingroup$ It sounds like you want to compare mean deviation from the mean, as a way of comparing spreads, is that right? Can you define your variables, clarify how many there are and make explicit the comparison, please? $\endgroup$ – Glen_b -Reinstate Monica Jun 8 '14 at 0:10
  • $\begingroup$ I apologize for confusion. The original question is edited. Thank you! $\endgroup$ – Q_Li Jun 8 '14 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.