Return to Answer

For the actual computation of the confidence interval we use the vertical direction. We compute the boundaries for each $\theta$ as a hypothesis test. This computation will be the same for a transformed $\theta$.

For the actual computation of the confidence interval we often use the vertical direction. We compute the boundaries for each $\theta$ as a hypothesis test. This computation will be the same for a transformed $\theta$.

Making the boundaries smallest in the horizontal direction is more difficult, because there is no good way to define/measure it. It could be possible, maybe, if you use some prior for the distribution of $\theta$. In that case one could shift the choice of the boundaries (which still must be in the vertical direction to ensure 95% coverage, conditional on $\theta$, but they do not need to be optimal in vertical direction) in order to optimise some measure for the length of the interval. In that case, the transformation does indeed change the situation. But this way of constructing confidence intervals is not very typical.

Making the boundaries smallest in the horizontal direction is more difficult, because there is no good way to define/measure it (making the interval shorter for one observation requires making the interval larger for another, and one would need some way to weigh the different observations). It could be possible, maybe, if you use some prior for the distribution of $\theta$. In that case one could shift the choice of the boundaries (which still must be in the vertical direction to ensure 95% coverage, conditional on $\theta$, but they do not need to be optimal in vertical direction) in order to optimise some measure for the length of the interval. In that case, the transformation does indeed change the situation. But this way of constructing confidence intervals is not very typical.

Confidence intervals are based on probabilities conditional on the parameters, and do not transform like regular probabilities (on which credible intervals are based), if you transform the parameters. See for instance in this question: If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval? a confidence interval is not just like a credible interval with a flat prior. For a confidence interval we have:

That is, we choose the range of $\theta$ (as a function of $X$) based on the condition:

the range of all hypotheses $\theta$ for which the observation is inside the (narrowest) $\alpha\%$ hypothesis test (in this example definition it is a two-tailed test with tails that have equal probability mass, which is narrowest for Gaussian distribution observations, but it might be changed for other shapes conditional probability functions, depending on the situation)

This condition, the hypotheses, does not change with the transformation. The hypothesis $\theta = 1$, is the same as the hypothesis $\log(\theta) = 0$.

Confidence intervals are based on probabilities conditional on the parameters, and do not transform if you transform the parameters. Unlike (Bayesian) probabilities of the parameters (on which credible intervals are based). See for instance in this question: If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval? a confidence interval is not just like a credible interval with a flat prior. For a confidence interval we have:

That is, we choose the range of $\theta$ (as a function of $X$) based on a probability conditional on the $\theta$'s. For instance

the range of all hypotheses $\theta$ for which the observation is inside a two-tailed $\alpha\%$ hypothesis test.

This condition, the hypotheses, does not change with the transformation. For instance, the hypothesis $\theta = 1$, is the same as the hypothesis $\log(\theta) = 0$.