0
$\begingroup$

When we perform two tailed test (z-test for proportion, ttest or even bootstrap) the null hypothesis is that there is no difference or samples come from same distribution and alternative hypothesis is that there is a difference or that samples come from different distributions.

Technically in case of two tailed test p-value is the probability of getting difference between two means / proportions greater or equal to the present difference from my experiment having that null hypothesis is true. And this difference could go both sides, it is an absolute difference. So basically two tailed test doesn't say which mean / proportion is greater. It doesn't show direction. It just says that there is a difference.

I know that there is a one tailed test. But it doesn't make sense to me: when I use one tailed test I get significant results twice as often. And I also know that actually it is almost never appropriate to use one tailed test. And in everyday life say with A/B testing we always use two tailed tests. I've personally never seen one tailed test in business practice.

My question is how it is possible to make decisions after two sided test and for example in case of A/B test to say that mean / proportion in one group is greater than other? Test doesn't show the direction of the difference, but we say that one greater than other and make our decision. Am I missing something?)

$\endgroup$
2
  • 4
    $\begingroup$ Once you have decided the difference is significant, look at what the data tell you. $\endgroup$
    – whuber
    Jul 19, 2021 at 12:55
  • 2
    $\begingroup$ This is not the formal approach, but intuitively you might look at a two-tailed test as two one-tailed tests, with a correction for doing two mutually exclusive tests simultaneously (so for example $5\%$ critical values nearer $1.96$ standard errors than $1.64$ standard errors). $\endgroup$
    – Henry
    Jul 19, 2021 at 13:21

1 Answer 1

0
$\begingroup$

Well, eventually Two Tailed test is mostly used when you want to say, wether the sample mean is equal to your population mean (Null Hypothesis).

  • Here you use an equality sign(=)

And, One tailed test is when you would like to say, sample mean is better/worse than your population mean(Null Hypothesis)

  • Here you use a greater/less than or equal to signs(>= , <=)

You can definitely say after two tailed test wether the sample which you used for testing came from the same population or not!? And Similarly, derive conclusion for one tail test.

Check this link: https://www.investopedia.com/terms/o/one-tailed-test.asp

$\endgroup$
1
  • $\begingroup$ What is your answer? It's unclear what you recommend. $\endgroup$
    – whuber
    Jul 20, 2021 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.