# Validity of conclusion of statistical tests when comparing two groups of observations in an experiment

I am just reading the results of an experiment, and I am having a hard time estimating their validity.

In the experiment, several participant had to perform a certain task twice, once with the treatment and the other one without it. There were two possible tasks each participant could work on, but she could only pick one.

Several non-parametric statistical test were applied to compare the average result of the following configurations (or "hypothesis"):

Task A with treatment vs Task A without treatment (n = 8) Task B with treatment vs Task B without treatment (n = 8) Treatment vs Without treatment (n = 16)

The statistical tests for the first two configurations said that p >= 0.05, but the statistical test for the last configuration said that p < 0.05. Therefore, the conclusion was that there is in fact a difference between the results with treatment and the results without treatment (kind of ignoring the results from the first two configurations).

Is that conclusion correct?

I sincerely apologize for my prose. I am not really knowledgeable in statistics so my terminology is definitely vague and confusing.

Thanks for your help!!!

• You get points for "definitely vague" :-)...Otherwise, wording in these matters is very important. No one needs to conclude that "there is in fact a difference": clearly there is, among this sample. The question is, how often do such things occur by chance, and then, given that calculation (p), how to interpret the likelihood of a difference existing in the larger population to which you want to generalize. – rolando2 Jan 19 '17 at 16:03