2
$\begingroup$

The Ansari-Bradley test appears to be ineffective at detecting scale differences when the two distributions have markedly different medians. However, when I subtract the median of each group, I get greatly increased power without maintaining the type I error rate.

The temptation is to recommend that we subtract out the medians before we do an Ansari-Bradley test of scale. If we always do this, then I do not see it as data snooping, and then it isolates the Ansari-Bradley test to testing the scale.

Why would I not want to subtract out the medians before I do the Ansari-Bradley test?

$\endgroup$
1
$\begingroup$

Here is the short of it:

You have the correct instinct. The original paper from the eponymous creators of the test make the assumption the population difference in medians is known, therefore not of concern because you can just center your data. So basically, the test ignores that issue.

In summary, the test is this. Assume you have a simple random sample from a population $X$ with probability distribution $F(X)$. And assume you have a sample from population $Y$ with probability distribution $G(Y)$. The hypotheses are:

  • $H_0$: $F$ and $G$ are identical distributions.
  • $H_1$: The only difference between $F$ and $G$ is the scale parameter differs.

You could estimate the difference in location parameters and center your data before you run the test in software. Some software may even do this (even though it's not a good idea). I would not call this data-snooping. You are trying to use your data to answer the question "Are the scales different? I don't care about locations." It's just that the tool you have isn't quite right for the job without the strong assumption that the true difference in population medians is known.

Here's the problem. Say you were to center the data based on an estimated median difference or location difference. The Ansari-Bradley test was formulated and all probabilistic results were derived based on knowing the true location difference. You do not know what the true difference is, you can only estimate it. This means that results of the test would not be reliable when you used the estimate to center the data. This would be especially problematic at small sample sizes.. At larger sample sizes it might not matter. But that would require investigation, simulations, etc. to verify.

This is similar to the one-sample z-test for the mean that gets taught in many introductory statistics courses. You assume you know the true standard deviation $\sigma$. This allows you to use the Z-distribution for inference. However, you almost always never know $\sigma$ so you estimate it with $s$. But, the Z-distribution would no longer be valid for the situation and any inference would be unreliable. However, we can fall back on the t-distribution and carry on like nothing's wrong. You do not have that option with the Ansari-Bradley test.

Here is the long of it

The test was a great idea from a theoretical non-parametric statistics perspective since the results of the Ansari-Bradley test were distribution free, i.e., which probability distribution the data came from didn't matter. We love distribution free tests in non-parametric statistics. This also brings up another somewhat problematic assumption, they assumed that not only was the location difference known, but that the centered distributions were identical except for scale (which is a similar assumption of the Mann-Whitney-Wilcoxon test for location but also assumes equal scale). Without these kinds of assumptions, we lose the distribution free properties of non-parametric tests.

I've done some investigating and tried to find an alternative method which doesn't require those problematic assumptions and... My initial findings aren't so great. At least I can't find a well suited alternative test implemented commonly, or at least not obscurely, in software. Maybe someone else knows of something...

The most straight forward brute force method is to make a permutation test. Create an appropriate test statistic that accounts for the difference in location, calculate it for the original data, then permute the data thousands of times under the assumption the scale is the same, and finally compare the permutation test statistics to the original and see what happens. The problem is that whole centering the data thing. It makes things a bit unreliable overall.

I did find a paper where the assumption is that the distributions have some quantile in common, not necessarily the median. Link here. However it looks like they just wrote some SAS code for it and there isn't a generic function.

$\endgroup$
3
  • $\begingroup$ "Given the test was formulated and all probabilistic results were derived based on knowing the true location difference, results of the test are not reliable if one used the estimated difference in location, especially so at small sample sizes." This seems to contradict the paragraph above it. Could you please clarify? $\text{//}$ This is not the first time that I have looked for a variance test and found myself disappointed. I have another question on here from the past month about how to examine for variance explicitly, even when the normality assumption is violated. $\endgroup$ – Dave Dec 23 '20 at 20:58
  • $\begingroup$ Also, reading the original Ansari/Bradley paper, their first page seems to say that it's okay to subtract out the medians. "Taking these location parameters to be zero without loss of generality, we test that hypothesis that $G(u) = F(u)$ against alternatives of the form $G(u) = F(\theta u)$, $\theta\ne 1$." $\text{//}$ My recent post where I was looking for a suitable variance test: stats.stackexchange.com/q/498718/247274. $\endgroup$ – Dave Dec 23 '20 at 21:06
  • $\begingroup$ I tried to add some clarification by adding an example about the one sample z-test for the mean, and rewriting some bits. What you point out in the paper initially tripped me up a bit, because I thought that meant there shouldn't be an issue. However, right above what you quote is what I am talking about. "It is required for the basic test that the difference in locations (medians) of the two populations be known...". That assumption allows for their without loss of generality statement. My main point is that you will not know the true difference, therefore the test isn't reliable. $\endgroup$ – Double Ought Not Dec 23 '20 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.