0
$\begingroup$

When we perform two tailed test (z-test for proportion, ttest or even bootstrap) the null hypothesis is that there is no difference or samples come from same distribution and alternative hypothesis is that there is a difference or that samples come from different distributions.

Technically in case of two tailed test p-value is the probability of getting difference between two means / proportions greater or equal to the present difference from my experiment having that null hypothesis is true. And this difference could go both sides, it is an absolute difference. So basically two tailed test doesn't say which mean / proportion is greater. It doesn't show direction. It just says that there is a difference.

I know that there is a one tailed test. But it doesn't make sense to me: when I use one tailed test I get significant results twice as often. And I also know that actually it is almost never appropriate to use one tailed test. And in everyday life say with A/B testing we always use two tailed tests. I've personally never seen one tailed test in business practice.

My question is how it is possible to make decisions after two sided test and for example in case of A/B test to say that mean / proportion in one group is greater than other? Test doesn't show the direction of the difference, but we say that one greater than other and make our decision. Am I missing something?)

Improve this question
$\endgroup$
12
  • 4
    $\begingroup$ Once you have decided the difference is significant, look at what the data tell you. $\endgroup$
    – whuber ♦
    Commented Jul 19, 2021 at 12:55
  • 4
    $\begingroup$ This is not the formal approach, but intuitively you might look at a two-tailed test as two one-tailed tests, with a correction for doing two mutually exclusive tests simultaneously (so for example $5\%$ critical values nearer $1.96$ standard errors than $1.64$ standard errors). $\endgroup$
    – Henry
    Commented Jul 19, 2021 at 13:21
  • 1
    $\begingroup$ "And I also know that actually it is almost never appropriate to use one tailed test." False. For example tests of non-inferiority and tests of non-superiority are routinely one-sided tests. There's also the two one-sided tests framework for tests for equivalence, also routine. $\endgroup$
    – Alexis
    Commented Oct 19, 2023 at 22:22
  • 1
    $\begingroup$ Two relevant answers here and here. $\endgroup$
    – dimitriy
    Commented Oct 16 at 20:53
  • 1
    $\begingroup$ @MichaelLew Hi Mike not sure where we disagree. I imagine you agree that a one-tail p-value can only be calculated if you specify the hypothesized direction of the effect as part of the experimental design? And that a one-sided p-value will be >0.50 when the actual data goes in the opposite direction? So where is the disagreement? $\endgroup$
    – Harvey Motulsky
    Commented Nov 16 at 21:38

1 Answer 1

Reset to default
-1
$\begingroup$

Well, eventually Two Tailed test is mostly used when you want to say, wether the sample mean is equal to your population mean (Null Hypothesis).

  • Here you use an equality sign(=)

And, One tailed test is when you would like to say, sample mean is better/worse than your population mean(Null Hypothesis)

  • Here you use a greater/less than or equal to signs(>= , <=)

You can definitely say after two tailed test wether the sample which you used for testing came from the same population or not!? And Similarly, derive conclusion for one tail test.

Check this link: https://www.investopedia.com/terms/o/one-tailed-test.asp

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ What is your answer? It's unclear what you recommend. $\endgroup$
    – whuber ♦
    Commented Jul 20, 2021 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.