5
$\begingroup$

Let us say we perform Wilcoxon signed-rank test on paired samples $x_{1,i}$ and $x_{2,i}$. I am trying to understand the independence assumption of the test. My questions are:

  1. Which quantities must be independent? Is it $x_{1,i}-x_{2,i}$ or is it the ranks, $R_i$ of $x_{1,i}-x_{2,i}$? Alternatively, is it the signed ranks, $R_i \cdot sgn(x_{1,i}-x_{2,i})$?

  2. Why is there a requirement for independence at all? My understanding is that Wilcoxon signed-rank test is based on a permutation test which requires exchangeability of data points. Since exchangeability does not necessarily require independence, then why does Wilcoxon signed-rank test require independence?

  3. Does independence need to hold only for the null hypothesis or does it have to be true for alternative hypothesis as well?

  4. What would happen if independence assumption was not met? I understand that if a requirement is not met, the $p$-values might be erroneous. I am looking for a more specific answer that describes how a step in conducting the test can go wrong in the absence of independence. A description with an example of a data that lacks required independence would be much appreciated.

  5. How can we assess the impact of of autocorrelation in the data? For example, if there were autocorrelation of ~0.2 for lag=1 and ~0 for lag>1, how would it impact the significance level and power?

$\endgroup$
0

1 Answer 1

3
$\begingroup$

A permutation test will still be valid under exchangeability rather than independence. In this case I think it is exchangeability of $+$ and $-$ signs across the observations (the set of ranks if your test statistic is the usual signed rank one) that matters.

When performing the usual rank-based test it is common to assume that the pair-differences, $d_i=x_{1,i}-x_{2,i}$ are independent (the independence of the derived quantities like the ranks and the signed ranks will follow); while this is more than is necessary it is often a reasonable assumption to consider, while the less general assumptions are harder to consider (in that when independence of the $d$'s doesn't hold it may be difficult to identify whatever does hold that would still be sufficient. Calculation of the variance of the sum of the signed ranks is usually based on assuming independence (from which uncorrelatedness follows that is actually needed to get the variance formula).

Independence would not be required under the alternative; various forms of dependence under the alternative will impact the power of course (along with sample size calculations) but not the type I error rate.

While direct algebraic calculations may often be difficult, we can assess the impact of specific kinds of dependence on the properties of a test (like significance level and power) via simulation; when the form of dependence being considered is a function of a single parameter it's often convenient to produce a power curve.

[I will attempt to come back and give a specific example.]

$\endgroup$
6
  • $\begingroup$ Thanks. I will appreciate more details on some of your statements. Specifically, it is not clear to me why dependence under the alternative will impact the power but not the type I error rate. $\endgroup$
    – Mr K
    Aug 14, 2018 at 7:55
  • 1
    $\begingroup$ It won't impact the type I error rate because that only exists under the null. If you only have dependence under the alternative the type I error rate will be based on independence $\endgroup$
    – Glen_b
    Aug 14, 2018 at 10:30
  • $\begingroup$ That makes sense. I suppose power would be affected depending on characteristics of the alternative and a test that accounts for dependence under the alternative might have more power. $\endgroup$
    – Mr K
    Aug 14, 2018 at 18:54
  • $\begingroup$ Yes. More strictly it's the design of the test statistic that is important; it needs to be sensitive to the location difference in the presence of whatever the form of dependence is, given the distribution you're sampling. [If you believe you have the distribution under the null and alternative correct, a likelihood ratio test is a common choice. If you think it's close to correct, a robustified test may be a good choice. If you don't know, a nonparametric procedure may be a better choice but then the design is harder, because normally you can only do the power when you know the distribution] $\endgroup$
    – Glen_b
    Aug 14, 2018 at 23:21
  • 1
    $\begingroup$ It depends on what you mean by both "valid" and by "violated". I outline the circumstances for a particular pair of meanings is my answer above; if there is dependence (violation of independence) between the pair differences only under the alternative (e.g. if there was increasing dependence with effect size), the test will still have the correct type I error (so valid in that particular sense), but its power would be impacted. $\endgroup$
    – Glen_b
    Mar 30, 2020 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.