If you have a random sample of size $n$ from $\mathsf{Exp}(\lambda = 1/\mu),$ then $\bar X \sim \mathsf{Gamma}(n,\, \text{rate}=n\lambda).$
Thus, $P(L \le \bar X\lambda = \bar X/\mu \le U)=0.95,$
where $L$ and $U$ cut probability 0.025 from the lower and upper
tails of $\mathsf{Gamma}(\text{shape} = n,\, \text{rate}=n),$ respectively.
This implies that $P(\bar X/U \le \mu < \bar X/L) = .95,$ so that
a 95% CI for $\mu$ is of the form $(\bar X/U, \bar X/L).$
For example, if $\mu = 1/\lambda = 5$ and $n = 10,$ we might get $\bar X = 3.80.$ Then
a 95% CI for $\mu$ is $(2.22, 7.92).$ Notice that $\bar X$ is contained in this CI, but the sample mean does not lie
at the center of the CI. Computations in R:
set.seed(1234); a = mean(rexp(10, 1/5)); a
[1] 3.800074
a/qgamma(c(.975,.025), 10, 10)
[1] 2.224242 7.924434
Notes: (1) The Wikipedia article on exponential distributions
discusses inference in some detail; under 'Confidence Interval'
the article has a confidence interval equivalent to the one
shown above, but in terms of a chi-squared distribution. (This makes
it possible to find confidence limits using printed chi-squared tables.)
(2) To get an exact 95% CI for rate $\lambda,$ take reciprocals: $(L/\bar X,\, U/\bar X).$ However, notice that $1/\bar X$ is a biased point estimator of $\lambda,$ with bias becoming negligible for large $n.$
(3) Because the exponential distribution is highly skewed it is inappropriate to use a symmetrical CI of the form $\bar X \pm M,$
were $M$ is a margin of error based on a (symmetrical) normal distribution, unless the sample size is sufficiently large.
For sufficiently large $n,$ the mean $\bar X$ of an exponential
sample becomes approximately normal, and a symmetrical CI is
a reasonable approximation.
(4) Below is a simulation of a million $\bar X$'s based on random samples of size $n = 10$ from $\mathsf{Exp}(\mu = 5).$ The histogram illustrates
that $Q = \bar X/\mu \sim \mathsf{Gamma}(\text{shape}=n, \text{rate}=n),$ as claimed above. A formal proof uses moment generating functions.
set.seed(2019)
a = replicate(10^6, mean(rexp(10, 1/5)))
hist(a/5, prob=T, col="skyblue2", xlab="Q",
main="GAMMA(10,10)")
curve(dgamma(x,10,10), add=T, lwd=2)
abline(v = qgamma(c(.025,.975), 10, 10),
lwd=2, col="red", lty="dashed")
