I started with a hypothesis that says e.g. A is larger than B which would usually call for a one-tailed t-test. But after acquiring the data it turns out that the effect seems to be contrary to my hypothesis, as in B is larger than A. I can't still calculate my one-tailed t-test as my hypothesis is rejected. But I could technically calculate a two-tailed t-test with the explicit remark that this is merely exploratory, correct? In that case do I have to adjust my alpha and if so how? Thank you!
1
-
$\begingroup$ You can do the original one tailed test; you will not reject the null. $\endgroup$– Glen_bCommented Oct 26, 2019 at 6:43
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Technically, no, you have to stop analyzing these data and start over. This is the price you pay for having a one-tailed hypothesis and doing a one-tailed test. So, all you could do is say "the difference was in the opposite direction from the one we hypothesized and this is a subject for future research".
I sometimes use an analogy. Suppose you are told that there is \$1,000,000 in a house. The house has two floors. You have three hours to search. Now suppose that you are told it is on the first floor, so you don't look at all on the 2nd floor. After 3 hours, you haven't found it and then you are told it was really on the second floor. But it's too late now!
answered Oct 25, 2019 at 10:39
$\begingroup$
$\endgroup$
The advantage of one-sided testing is that, since you concentrate the probability in one tail, you don’t need as extreme of a test statistic for rejecting the null hypothesis. This will increase power if you know that only one direction is interesting to your particular problem. The disadvantage, as Peter Flom posted, is that you’ve used up all of your time to hunt for the money.
The advantage of two-sides testing is that you spread your probability over both tails (spread time over both floors, meaning that you will get to reject the null hypothesis if you get an extreme test statistic in either direction. The disadvantage is that you sacrifice power (need a larger test statistic) if you only have one direction that is interesting, so if you only care if A is greater than B, then you don’t need to waste power on checking the reverse.
It sounds like you care about both directions, so two-tailed testing is appropriate.
