Confidence intervals for the mean $\mu$ of a normal population based on a random sample of size $n$ from the population are of the type you mention.
A 95% z-interval, if the population standard deviation
$\sigma$ is known is of the form $\bar X \pm 1.96\,\sigma/\sqrt{n}.$ where $\bar X$ is the sample mean and $\sigma/\sqrt{n}$ is called the _standard error of the mean.
A 95% t-interval, if $\sigma$ is unknown and estimated by the sample standard deviation $S,$ is of the form $\bar X \pm t^*\,S/\sqrt{n},$ where $t^*$ cuts probability $0.025$ from the upper tail of the (symmetrical) Student distribution with $n-1$ degrees of freedom, and $S/\sqrt{n}$ is the (estimated) standard error of the mean.
Consider the following sample of size $n = 50$ from
$\mathsf{Norm}(\mu,\sigma).$ (Using R for sampling and computation.)
set.seed(2022)
x = rnorm(50, 100, 15)
mean(x)
[1] 98.06686
Then the z-interval for $\mu,$ where we know $\sigma=15$ is $(93.9,\,102.2):$
CI = mean(x) + qnorm(c(.025,.975)*\15/\sqrt{50}
[1] 93.90915 102.22457
By contrast, the t-interval for $\mu,$ where $\sigma$ is estimated by $S$ is $(94.34,\, 101.790).$ The t.test procedure
in R makes this interval. You can verify if for yourself, using the formula above.
t.test(x)$conf.int
[1] 94.34387 101.78984
attr(,"conf.level")
[1] 0.95
However, for various populations other than normal
and population parameters other than $\mu,$ confidence intervals for various population parameters
may be of different styles. Here are examples of
yet other styles of CIs that do not explicitly
use endpoints based on a margin of error (based on standard error) above and
below the sample mean
CI for normal variance. The confidence interval for the variance $\sigma^2$ of a normal population is based on the fact that
$\frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(\nu=n-1).$ The CI is of the form
$\left(\frac{(n-1)S^2}{U},\frac{(n-1)S^2}{L}\right),$
where $L$ and $U$ cut probability $0.025$ from the lower and upper tails, respectively, of $\mathsf{Chisq}(n-1).$ The 95% CI for $\sigma^2$ is $(119.7,\,266.5).)$ Notice that the point estimate
$\widehat{\sigma^2} = 171.6.$ does not lie at the center of this CI (because the chi-squared distribution is not symmetrical). Take square roots of the
endpoint of the CI for $\sigma^2$ to get a 95% CI
$(10.84,\,16.32)$ for the population standard deviation
$\sigma.$
var(x)
[1] 171.6105
49*var(x)/qchisq(c(.975,.025), 49)
[1] 119.7469 266.4851
sqrt(49*var(x)/qchisq(c(.975,.025), 49))
[1] 10.94289 16.32437
Bootstrap CI for gamma mean. The population distribution $\mathsf{Gamma}(\mathrm{shape}=3,\mathrm{rate}=0.2)$ has
mean $\mu = 3/0.2 = 15.$
There ere theoretical CIs for $\mu$ based on estimates of parameters
(the shape and rate) of such a distribution,
but the formulas are not as simple as the ones we
have seen above.
Moreover, in a real application, if we have a sample
of size $n = 500$ from a population with this
distribution, we may not even
know that the population is gamma distributed.
Then
a 95% nonparametric bootstrap CI can give useful
information. Here is how one style of bootstrap CI
can be computed. By repeated re-sampling with
replacement from the sample, we can get an idea
of the variability of $\bar X$ as a point estimate
of $\mu$ and use that information to make a 95%
CI $(4.28,\, 15.74)$ for $\mu.$
# simulate fictitious gamma data
set.seed(519)
y = rgamma(500, 3, 1/5)
summary(y); sd(y)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.3141 8.4593 13.1476 14.9594 19.4401 52.2321
[1] 8.913885
# 95% nonparametric bootstrap CI
set.seed(1234)
a.obs = mean(y)
d = replicate(200, mean(sample(y,500,rep=T))-a.obs)
UL = quantile(d, c(.975,.025))
a.obs - UL
97.5% 2.5%
14.27896 15.74119
