Correct Interpretation for One-tailed Hypothesis with Inverse Result

Question

I am currently doing a research and have this hypothesis for a variable:

HA: "Variable A has a significant positive influence on Variable B"

However, the result was actually negative and statistically significant at 1%. So just want to confirm whether I can safely say that the result is actually significant but the sign is the opposite from the hypothesis, so we can reject H0? Thanks!

Answer 1

The whole point of statistical hypothesis testing is to set a sharp line to decide when an effect is large enough to "reject" the null hypothesis. In much of science, such sharp cutoffs aren't helpful and can get in the way. But if you want to use a sharp cutoff, you have to do so honestly.

Only choose a one-sided test (also called one-tailed), when you are sure that a true effect (difference) can only go in one direction, and that any observed effect in the other direction can only be explained by experimental error and random sampling. If that is the case, then your p value should be reported as >0.99. It is computed as 1.0 minus the calculated small one-sided p-value. You cannot reject the null hypothesis.

Answer 2

There are several possibilities:

• If you have made a mistake using software and tested against an alternative in the wrong tail from what you intended, then it is best to do the correct test.

• If the test in the "wrong" tail was what you really intended, then it is cheating to choose between left and right tail only after seeing the results. If you're going to use a one-tailed test you need to declare that in advance of data collection.

• If you have no idea the direction of the effect as data collection starts, then you should use a two-sided alternative. (The P-value of a 2-sided test is double the P-value of a test in the "correct" tail; changing direction of alternative after seeing (some) data is P-hacking).

Consider the following fictitious data and three t tests:

Generate fictitious data

set.seed(1234)
x = rnorm(40, 54, 6)
summary(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  39.93   48.50   50.96   51.52   54.49   68.50 

Test $H_0: \mu=50$ vs. $H_a: \mu > 50.$ Purpose: is to see if population mean exceeds 50.

t.test(x, mu=50, alt="gr")$p.val
[1] 0.04386101    # Significant at 5% level

Test $H_0: \mu=50$ vs. $H_a: \mu < 50.$ Unexpected data? Honest blunder using R?

t.test(x, mu=50, alt="less")$p.val
[1] 0.956139      # Very large P-value 
                  #   can signal wrong tail

Test $H_0: \mu=50$ vs. $H_a: \mu \ne 50.$ Correct test if you wondered at the start whether there is a difference from 50, with no idea in advance whether smaller or larger than 50.

t.test(x, mu=50)$p.val
[1] 0.08772203   # P-val twice the first; 
                 #  not signif at 5% level

Answer 3

If you have any interest in saying anything but "we fail to reject $H_0$" when the relationship is in the opposite direction to the one you predicted, you should not have been using a one-tailed test.

You don't get to change your mind about what cases count as rejection after you see the outcome; that's p-hacking. In effect you misrepresent (to put it nicely) your true significance level when you work this way.

Consider the gamut of possible outcomes (and your desired actions in those cases) before you conduct your test, not after. Indeed, it should come before you collect data.