# Comparison of differences between pairs of samples of unequal size

I have four samples $x_1, x_2$ and $y_1, y_2$ with $n_{x1} \neq n_{x2} \neq n_{y1} \neq n_{y2}$. I calculated, using a Wilcoxon rank sum test, that $x_1$ is significantly different to $x_2$ and $y_1$ significantly different to $y_2$.

However, I would like to test whether the difference $x_1 - x_2$ differs significantly to $y_1 - y_2$ but I have no idea how do to that given the unequal sample sizes.

Any ideas or suggestions would be really appreciated.

If you consider all four groups in a Kruskal-Wallis (the rank based 'one-way anova'), you would be in the position of wanting to test a contrast there.

Now Kruskal-Wallis is basically a special case of the proportional odds ordinal logistic model.

You could get this contrast by testing a combination of coefficients in the proportional odds ordinal logistic model.

That is, the kinds of contrasts you'd tend to do in ANOVA pretty much can be done for a generalization of the Wilcoxon type of approach.

I think Frank Harrell's R package rms may be able to do this, for example.

That said, I agree with @NickCox's suggestion of considering modelling with glm's more generally; there may be GLMs that describe the mean, the mean-variance relationship and the general shape of your data fairly well, and in that case, your contrasts become not only easy to test, but perhaps also more directly interpretable in terms of relationships between means, especially if identity links were used.

You need a different analysis. What might be suitable depends on the variables you are using and the nature of the sampling.

Abstract descriptions with notation do not necessarily make a problem clearer to statistical people. Tell us more about the variables and how they were measured and the nature of the samples. Do the samples overlap, etc.?

Why did you choose Wilcoxon in the first place?

(LATER) I am adding a few extra suggestions in the light of extra comments.

Your variables $x$ and $y$ sound like the same variable (Fano factor metric) with categorical controls neuron and condition. That suggests to me starting with analysis of variance; note that the normality assumption is the least important assumption and at most applies to the conditional distributions, i.e. the four distributions conditional or neuron and condition. Unequal groups are not fatal although results won't be especially trustworthy if any group size is small.

If nonnormality or heteroscedasticity is a real problem, some kind of generalised linear model is likely to be the next step.

I am not clear what "overlap" means and thus what that implies.

• Sorry, but that really does not help; all it suggests that you may not understand the statistical meaning of independence. If samples of x and y are independent of each other the pairwise differences are expected to be noise, which would seem to make testing pointless except as a check on that expectation. I'm asking for concrete scientific details to make your example vivid and for why you chose a non-parametric test (was it a question of measurement scale or were you working on some supposed rule such as the variables not being normally distributed. Commented Jun 7, 2013 at 11:13
• Apologies for previous comment; it was posted by pressing return by accident. Data come from a neuroscience experiment. We recorded the responses of two categories of neurons (x and y) under two different conditions (1 and 2). Data in each sample vector correspond to a Fano factor metric (scalar) calculated for each neuron and condition. For some neurons the metric is not available for both conditions which results in samples of unequal sizes. Samples do overlap, but as I said x1 appears to be different to x2 and y1 to y2. Commented Jun 7, 2013 at 11:35
• Regarding the test, I am not sure whether variables are normally distributed or not so I took the conservative approach and chose the nonparametric test. Many thanks for your replies! Commented Jun 7, 2013 at 11:36
• Thanks again for your reply. I am not sure how analysis of variance will help me given that I am not interested in testing whether e.g. x1 is different to y1. What I want to test is whether the difference between the conditions for the first category of neurons x1-x2, is different to that of the second category of neurons y1-y2 (i.e. compare the "effects"). What I mean by "overlap" is that the distributions of e.g. x1, y1 appear to overlap and are probably not significantly different. However, x1 is significantly different to x2 and y1 to y2. Commented Jun 7, 2013 at 12:21
• What you say sounds perfectly consistent with analysis of variance to me. Sure, you are more interested in some comparisons than others; that's standard. Note that your Wilcoxon test results don't have equivalents in ANOVA and in any case don't address all the controls you have. Commented Jun 7, 2013 at 12:25