# Statistical non-parametric test of order independence

I've ran an experiment in which different situations were presented to subjects in a random order, on which I measured a result out of $y \in {-1,0,1}$.

I'd like to show that $y$ is order-independent, meaning that the answer ratio was the same either if it was on the 1st trial or the last one.

My data is clearly shows it here:

What would be the right statistical test to prove this claim?

ANOVA feels almost there, since the groups (the number of trials) are linked.

Moreover, if the imageorder was an arbitrary (and not ordered), ANOVA would still unfit since the in-group distribution doesn't fit anova's assumptions, and therefore we need a non-parametric test.

Any suggestions?

• A few questions: Do you expect that the influence of position-in-the-order (PIO) should be monotonic if H1 is rejected, or do you need to test for idiosyncratic/non-monotonic effects of PIO? Is a response of 0 somehow "between" 1 and -1, or is the response purely categorical? Do you expect the influence of PIO on response, if it exists, will be similar across all of the (presumably 13) situations? – Jacob Socolar Sep 5 '17 at 4:32
• I'm not sure that monotonic test (linear effect if I understand you correctly) is the only thing I should test. The claim that I'm trying to prove is - no matter when the situation is presented, the answer distribution should be the same. – Dimgold Sep 6 '17 at 7:30
• Do the subjects repeat, in the sense of each subject seeing multiple situations each presented in random order, or does each subject only contribute one response to the overall experiment? – jbowman Sep 7 '17 at 18:10
• Yes, every subject (out of about 40) is observing 12 situations in a random order. – Dimgold Sep 8 '17 at 6:42
• I think some kind of permutation test would fit this situation. However, you have a bigger problem. The null hypothesis is that order does not matter. You are trying to confirm the null hypothesis. From the look of your data, you have very unbalanced classes of outcomes: almost all data points have $Y = -1$. This will reduce the power of your test. If you have a low-powered test, then "failing to reject the null hypothesis" does not mean much. – Lizzie Silver Sep 9 '17 at 10:06