Significance test between two samples when positions matter Let's say that I have two samples:
A    B  C    D
100  5  17   12  #<- Method 1
90   2  4    15  #<- Method 2

And want to compare to test whether method 1 is different from method 2. However, that observations in the samples are tied together, say 100 and 90 come from the same market A. What is the proper way to test this?
I am pretty sure that lumping everything together and doing a t-test or Mann-Whitney U test is not the way to go.
 A: If you are willing to assume normality of the differences (with your sample size, the assumption will be very important, but if your real data is much larger then the Central Limit Theorem makes this assumption less important) you can use a paired-T test, essentially take the difference between each pair (always the same direction) then do a 1 sample t test on the differences testing if the mean difference is 0.
There are non-parametric tests that will also test pairs and don't require the assumption of normality, but they are going to have very low power given your sample size (the sign test and exact permutation tests will have 0 power for any alpha level less than 0.0625).
A: Your observations appear to be counts (number of purchases). You would be best to use analyses that relate to that kind of data. A paired t-test won't be particularly suitable for a number of reasons (the variance of the difference isn't constant, for starters)
In this case, a chi-squared test of homogeneity will allow you to test for site-differences.
