Logical Hypothesis Modification I performed an experiment with a treatment and a control group. It just so happened that my treatment could be modified to either increase or decrease my dependent variable. So, I decided to test two hypothesis to strengthen my causal inference about the effect of my treatment:


*

*H1: $\mu_{treatment_1} > \mu_{control}$

*H2: $\mu_{treatment_2} < \mu_{control}$
In the end, I was able to reject the null for H1 with a p-value of .047, but I was unable to reject the null for H2. $\mu_{treatment_2}$ was smaller than $\mu_{control}$ but not by enough to reject chance. 
What I want to know, do people think I can modify my hypothesis at this point to: 


*

*$\mu_{treatment_1} > \mu_{control} > \mu_{treatment_2}$
This would allow me to use Jonckheere's test, which has a p-value of .0033 given my data.
On the one hand, it feels wrong since I'd be changing my hypothesis test to fit my data. On the other hand, it seems like it might be ok because the "logic" of my hypothesis is maintained (i.e., I'm not really changing the hypothesis). Or, perhaps, the solution is that I should just make sure to control for the FWER since I'd now be doing multiple tests to see if my original hypothesis is true.
 A: You can't reasonably change the test after you have seen the non-rejection from the test you used. That's significance-hunting in fairly pure form.
You could reasonably have changed it before you saw any data (and perhaps even have pre-registered the new analysis), based on the logic you outlined -- and the test would tend to be somewhat more powerful at picking up the combined alternative* in the case that both original alternatives were true (at the expense of not being able to pick up that only one were true).
[The trick is to ask about these issues - such as how to arrange things so that your alternative is as precise as possible and that you use a good choice of test, or to do power comparisons for various situations so you can choose the kind of performance you need - before you collect your data]
* than say a pair of one-sided Mann-Whitney tests (at a similar overall type I error rate), and even more powerful still compared to a Kruskal-Wallis, since that encompasses 6 such directional alternatives and you want to focus on one.
