# Paired sign test, when expected difference is not 50:50

I have a matrix of around 1000 species abundances from 40 sites e.g.

         SpA  Spb  SpC  SpD
Sample1    3    0  330    1
Sample2    0    0   20    0
etc...


I have two versions of this matrix, produced using two different sampling methods (say Method X and Method Y). I've already tested whether the different methods affect species richness (the number of species detected), and now I want to test whether particular groups of species are more abundant using one sampling method more over the other.

For example, 100 of the species may belong to Genus A - do those species have higher abundance using Method X than Method Y.

This seems like a simple paired sample test to me. The distribution of the differences has extremely high kurtosis, caused by some species being extremely abundance, and many rare species - I decided not to use a paired t-test. As there is such huge variation in the differences I thought the best test might be a paired sign test.

So to do the test for Genus A mentioned above I find all the species that belong to Genus A, and all the samples that those species belong to, and look at whether the abundance is higher using Method X or Method Y. I then do a binomial test to get a p-value.

My problem is that if I just do a test on ALL of the species (not just those in Genus A), I find a significant result. Method X is more often higher abundance than Method Y, even though the sampling effort is the same (the actual ratio is around 52:48 rather than 50:50). So I'm worried this skews the results when I test for Genus A. Is there a way to do a sign test when the underlying data does not have a 50:50 ratio?

If I change the binomial test from p=0.5, should I change it to 0.48 or 0.52 if I want to test whether abundance is higher using Method X? Is this even appropriate, given that the difference between methods could be caused by the species in Genus A?

I think a permutation test would be better but can't think of an appropriate way to permute the data.

• When you do the sign test you use the binomial with $p=0.5$ but you could use nay value you wanted instead. But I somehow feel that is not what is bothering you here, would you like to edit your question to explain more if I have not understood you? Commented Jul 22, 2017 at 16:13
• I guess I'm not sure whether it's appropriate or not to change the p=0.5. Maybe it is the differences between species that is causing the underlying pattern to not be 505:0, so I should just test for a differences from 50:50. Or, maybe I need to test for a differences to the underlying pattern, say 48:52. Or maybe I should do a different kind of test - I think some kind of permutation test would be ideal, but I can't think of an appropriate way to permute the data.
– rw2
Commented Jul 23, 2017 at 10:40
• 1. Abundances would not be lognormal; that's continuous while abundances are discrete. One common choice for a distributional model seems to be the log-series distribution. 2. Can you be more explicit about what you're trying to find out at the lower level? Commented Jul 23, 2017 at 12:33
• At the lower level, I want to test whether certain species are detected by one method more than the other. So whenever Species A occurs, does it show higher abundance using Method A compared to Method B more than would be expected by chance. (I guess I mean a discrete log-normal distribution then - described here influentialpoints.com/Training/…)
– rw2
Commented Jul 23, 2017 at 19:47
• I re-worded the question to try and make it clearer what the problem is
– rw2
Commented Jul 24, 2017 at 10:07