Testing a null hypothesis using positive and negative z-score; accept or reject

Question

Theorem:: To test null hypothesis $H_0$: $p_0$=$p_{1}$ versus alternate hypothesis $H_1$: $p_0 \ne p_{1}$ at the $\alpha$ level of significance, $H_0$ should be rejected if $z$ is either $(1)\le -z_{\alpha/2} $ or $(2) \ge z_{\alpha/2}$.

Using this theorem, I calculated that $z=-0.924$ and am trying to understand whether to accept or reject the null hypothesis under significance level $\alpha=0.01$.

It seems to me that the null hypothesis should be rejected here because $z_{\alpha/2}=-2.58$, which is less than $-0.924$, however the answer is the opposite (that it should be accepted). Why could this be?

Thank you!

Answer 1

$-2.58$ is the rejection region, below which you will reject your null hypothesis with a test statistic calculated from your data (i.e. $-.924$). Think of it this way, if you are always using a two-sided hypothesis test with $\alpha$ set at .01, the rejection region is always going to be $|2.58|$ regardless of your data (the rejection regions stays constant). In other words $-2.58$ is a reference point. You should be testing whether your calculated statistic falls above or below this reference point. In this case, if it falls below, you have sufficient evidence to reject your null hypothesis.

You have the two terms switched around (you seem to be using your test statistic as a rejection region and your rejection region as a test-statistic).

Answer 2

If $\hat p_1$ and $\hat p_2$ are estimates of success probabilities in two binomial experiments, then you need to find the (estimated) standard error of $\hat p_1 - \hat p_2$ in order to do a z test.

Suppose Sample 1 has $X$ successes out of $n_1$ trials and Sample 2 has $Y$ successes out of $n_2$ trials, then $\hat p_1 = X.n_1$ and the variance of $\hat p_1$ is estimated by $\hat p_1(1-\hat p_1)/n_1,$ and similarly for $\hat p_2$ so the (estimated) standard error of $\hat p_1 - \hat p_2$ is

$$ \text{SE} = \sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1} + \frac{\hat p_2(1-\hat p_2)}{n_2}}.$$

Then the test statistic $Z = (\hat p_1 - \hat p_2)/\text{SE}$ is approximately distributed. So you would reject $H_0: p_1 = p_2$ against the two-sided alternative $H_a: p_1 \ne p_2$ at the 5% level if $|Z| \ge 1.96.$ (At the 1% level, you would reject if $|Z| > 2.576.)$

Here is an example of such a test from Minitab software. Note the reference to Fisher's exact test at the end, which is an alternative method of testing the hypothesis, without using a normal approximation.

Test and CI for Two Proportions 

Sample   X   N  Sample p
1       23  45  0.511111
2       29  40  0.725000


Difference = p (1) - p (2)
Estimate for difference:  -0.213889
95% CI for difference:  (-0.415081, -0.0126970)
Test for difference = 0 (vs ≠ 0):  Z = -2.08  P-Value = 0.037

Fisher’s exact test: P-Value = 0.049

With either test, you can reject at the 5% level but not at the 1% level.