Confidence interval for normal mean. Suppose we have a random sample $X_1, X_2, \dots X_n$ from a normal population. Let's look at the confidence interval for normal mean $\mu$ in terms of hypothesis testing. If $\sigma$ is known, then a two-sided test of $H_0:\mu = \mu_0$ against $H_a: \mu \ne \mu_0$ is based on the statistic $Z = \frac{\bar X - \mu_0}{\sigma/\sqrt{n}}.$ When $H_0$ is true, $Z \sim \mathsf{Norm}(0,1),$ so we reject $H_0$ at the 5% level if $|Z| \ge 1.96.$
Then 'inverting the test', we say that a 95% CI for $\mu$ consists of the values $\mu_0$ that do not lead to rejection--the 'believable' values of $\mu.$ The CI is of the form $\bar X \pm 1.96\sigma/\sqrt{n},$ where $\pm 1.96$ cut probability
0.025 from the upper and lower tails, respectively, of the standard normal distribution.
If the population standard deviation $\sigma$ is unknown and estimated by by the sample standard deviation $S,$ then we use the statistic $T=\frac{\bar X - \mu_0}{S/\sqrt{n}}.$ Before the early 1900's people supposed that $T$ is approximately standard normal for $n$ large enough and used $S$ as a substitute for unknown $\sigma.$ There was debate about how large counts as large enough.
Eventually, it was known that $T \sim \mathsf{T}(\nu = n-1),$ Student's t distribution with $n-1$ degrees of
freedom. Accordingly, when $\sigma$ is not known, we use $\bar X \pm t^*S/\sqrt{n},$ where $\pm t^*$ cut probability 0.025 from the upper and lower tails, respectively, of $\mathsf{T}(n-1).$
[Note: For $n > 30,$ people have noticed that for 95% CIs $t^* \approx 2 \approx 1.96.$ Thus the century-old idea that you can "get by" just substituting $S$ for $\sigma$ when $\sigma$ is unknown and $n > 30,$ has persisted even in some recently-published books.]
Confidence interval for binomial proportion. In the binomial case, suppose we have observed $X$ successes in a binomial experiment with $n$ independent trials. Then we use $\hat p =X/n$ as an estimate of the binomial success probability $p.$
In order to test $H_0:p = p_0$ vs $H_a: p \ne p>0,$ we use the statitic $Z = \frac{\hat p - p_0}{\sqrt{p_0(1-p_0)/n}}.$ Under $H_0,$ we know that $Z \stackrel{aprx}{\sim} \mathsf{Norm}(0,1).$ So we reject $H_0$ if $|Z| \ge 1.96.$
If we seek to invert this test to get a 95% CI for $p,$ we run into some difficulties. The 'easy' way to invert the test is to
start by writing $\hat p \pm 1.96\sqrt{\frac{p(1-p)}{n}}.$ But his is useless because the value of $p$ under the square root is unknown. The traditional Wald CI assumes that, for sufficiently large $n,$ it is OK to substitute $\hat p$ for unknown $p.$ Thus the Wald CI is of the form $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$ [Unfortunately, the Wald interval works well only if the number of trials $n$ is at least several hundred.]
More carefully, one can solve a somewhat messy quadratic inequality to 'invert the test'. The result is the Wilson interval. (See Wikipedia.) For a 95% confidence interval a somewhat simplified version of this result comes from
defining $\check n = n+4$ and $\check p = (X+2)/\check n$ and then computing the interval as $\check p \pm 1.96\sqrt{\frac{\check p(1-\check p)}{\check n}}.$
This style of binomial confidence interval is widely known as the Agresti-Coull interval; it has been widely advocated in elementary textbooks for about the last 20 years.
In summary, one way to look at your question is that CIs for normal $\mu$ and binomial $p$ can be viewed as inversions of tests.
(a) The t distribution provides an exact solution to the problem of needing to use $S$ for $\sigma$ when $\sigma$ is unknown.
(b) Using $\hat p$ for $p$ requires some care because the mean and variance of $\hat p$ both depend on $p.$ The Agresti-Coull CI provides one serviceable way to get CIs for binomial $p$ that are reasonably accurate even for moderately small $n.$