Why, in this case, is the test using a confidence interval not equivalent to the test using hypothesis testing, and is it?
Hypothesis testing for
$H_0: \pi = \pi_0$
$H_1: \pi\neq\pi_0$
A single trial has a Bernoulli distribution with mean $\pi$ and variance $\pi(1-\pi)$. According to the central limit theorem, the average of $n$ trials will have (approximately) mean $\pi$ and variance $\frac{\pi(1-\pi)}n$. Therefore,
$U_1 = \frac{\hat \pi -\pi}{\sqrt\frac{\pi(1-\pi)}{n}}$ is from $\mathcal N(0;1)$
If $H_0$ is true, $\pi = \pi_0$, so if $H_0$ is true,
$U_2 = \frac{\hat \pi -\pi_0}{\sqrt\frac{\pi_0(1-\pi_0)}{n}}$ is from $\mathcal N(0;1)$.
So we reject $H_0$ iff ($u_\frac{\alpha}{2}> U_2$ or $u_{1-\frac{\alpha}{2}}< U_2$).
In other words, we reject $H_0$ iff ($u_\frac{\alpha}{2}> \frac{\hat \pi -\pi_0}{\sqrt\frac{\pi_0(1-\pi_0)}{n}}$ or $u_{1-\frac{\alpha}{2}}< \frac{\hat \pi -\pi_0}{\sqrt\frac{\pi_0(1-\pi_0)}{n}}$).
On the contrary, testing according to confidence intervals:
The confidence interval is
$\left[\hat\pi - u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}};\hat\pi + u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}\right]$
We reject $H_0$ iff ($\pi_0< \hat\pi - u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$ or $\pi_0> \hat\pi + u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$).
In other words, we reject $H_0$ iff ($\pi_0-\hat\pi < - u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$ or $\pi_0-\hat {\pi} > u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$).
Rewriting the parenthesis:
($\hat\pi -\pi_0 > u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$ or $\hat {\pi} -\pi_0 < -u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$)
($\hat\pi -\pi_0 > u_{1-\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$ or $\hat {\pi} -\pi_0 < u_{\frac{\alpha}{2}}\sqrt{\frac{(1-\hat{\pi})\hat{\pi}}{n}}$)
($u_\frac{\alpha}{2}> \frac{\hat \pi -\pi_0}{\sqrt\frac{\hat\pi(1-\hat\pi)}{n}}$ or $u_{1-\frac{\alpha}{2}}< \frac{\hat \pi -\pi_0}{\sqrt\frac{\hat\pi(1-\hat\pi)}{n}}$)
The difference:
When using hypothesis testing, we reject $H_0$ iff ($u_\frac{\alpha}{2}> \frac{\hat \pi -\pi_0}{\sqrt\frac{\pi_0(1-\pi_0)}{n}}$ or $u_{1-\frac{\alpha}{2}}< \frac{\hat \pi -\pi_0}{\sqrt\frac{\pi_0(1-\pi_0)}{n}}$).
On the contrary, when using the confidence interval, we reject $H_0$ iff ($u_\frac{\alpha}{2}> \frac{\hat \pi -\pi_0}{\sqrt\frac{\hat\pi(1-\hat\pi)}{n}}$ or $u_{1-\frac{\alpha}{2}}< \frac{\hat \pi -\pi_0}{\sqrt\frac{\hat\pi(1-\hat\pi)}{n}}$).
Is the difference caused by replacing $\pi$ with $\hat\pi$ while constructing the confidence interval, making it just an approximate confidence interval? Or by something else?
(There are other questions on the site about the equivalence between confidence intervals and hypothesis testing, but none of them helped me with this.)
