# Testing statistical significance in two conditions

I am measuring two unpaired variables $x$ and $y$ in two different conditions ($x$ and $y$ are magnitudes of some special magnetic signals). In the first condition, my hypothesis is that $\bar{x} > \bar{y}$ and in the second condition that $\bar{x} < \bar{y}$. Now that I have $N$ samples from both variables, how can I test whether my hypotheses are true? I am not sure if I can safely assume that $x$ and $y$ are independent from each other. Neither do I know from what kind of distributions they are sampled from. The sample size I have is small. I have read several introductions to statistics for the past few days, but never saw a worked out example for this kind of situations. All help appreciated.

Edit: Like Michael Mayer wrote, there is a binary grouping variable "condition". Sorry for a bit unclear question.

• I have a high degree of belief that exactly one of your hypotheses is true :-). Your data might be able to cast doubt on one of them. But what is "small"? To some people that means $N\le 2$; to others, it might mean $N\le 10^6$.
– whuber
Commented Oct 7, 2013 at 21:34
• My sample size is $N = 10$. Commented Oct 7, 2013 at 21:39
• Hypotheses are about populations, not samples. You can check whether $\bar x < \bar y$ at a glance - no need for p-values or anything. Commented Oct 7, 2013 at 23:20
• Given your small sample and lack of knowledge about the distributions, I'd suggest a permutation test. Commented Oct 7, 2013 at 23:53
• @Peter See stats.stackexchange.com/a/1836/919 for why $N=1$ can work. $N=2$ is usually needed in order to estimate the variability. In applications where observations are sufficiently expensive, $N \gt 2$ may be considered extremely large. (I work in a field where (a) private parties pay for observations which are (b) required by government regulations that (c) are viewed as a burden and, in the worst situations, (d) an observation (actually a monitoring station) can cost \$100K or more. If you want to tell my clients they need a larger$N$, you had better have a great reason!) – whuber Commented Oct 8, 2013 at 3:10 ## 1 Answer A simple approach would be the following: 1) Take all observations sampled at random under condition A and obtain the relevant one sided p-value from Wilcoxon's rank sum test. 2) Do the same for the observations sampled under condition B. 3) If the smaller of the two p-values is below the level$\alpha/2$and the other p-value is below$\alpha$, then your claim holds at the$\alpha\$ level. (This would be the Bonferroni-Holm correction for multiple testing.)

Since the sample sizes are extremely low, you will get a "significant" result only if the signal is very strong.