32
$\begingroup$

Suppose the 95% confidence interval for $\ln(x)$ is $[l,u]$. Is it true that the 95% CI for $x$ is simply $[e^l, e^u]$?

I have the intuition the answer is yes, because $\ln$ is a continuous function. Is there some theorem that supports/refutes my intuition?

$\endgroup$
  • 1
    $\begingroup$ What is $X$? An estimate of the mean? And so if you know the CIs of the mean, you would like to find the CIs of the log mean? Or did you mean that you know the percentiles of a random variable $X$, and want to find the corresponding percentiles of the random variable $\log(X)$? $\endgroup$ – tchakravarty Jul 12 at 0:13
  • 1
    $\begingroup$ The latter—though would the answer differ if I had the former in mind? $\endgroup$ – Tamay Jul 12 at 0:14
  • 6
    $\begingroup$ Great first question! Welcome to Cross Validated. $\endgroup$ – Neil G Jul 12 at 18:39
  • 3
    $\begingroup$ Continuity is not directly relevant: only monotonicity is. $\endgroup$ – whuber Jul 14 at 21:00
  • $\begingroup$ @Tamay, although you could consider confidence intervals as part of a confidence distribution, it is not very common to consider the confidence interval as percentiles of that distribution. Do you really mean this latter case of tchakravarty? If you are looking for transformation rules for percentiles, then could you elaborate on what distribution you connect this to (a probability distribution or a confidence distribution)? $\endgroup$ – Sextus Empiricus Jul 15 at 7:30
49
$\begingroup$

That is a 95% confidence interval for $x$, but not the 95% confidence interval. For any continuous monotonic transformation, your method is a legitimate way to get a confidence interval for the transformed value. (For monotonically decreasing functions, you reverse the bounds.) The other excellent answer by tchakravarty shows that the quantiles match up for these transformations, which shows how you can prove this result.

Generally speaking, there are an infinite number of possible 95% confidence intervals you could formulate for $x$, and while this is one of them, it is not generally the shortest possible interval with this level of confidence. When formulating a confidence interval, it is usually best to try to optimise to produce the shortest possible interval with the required level of coverage --- that ensures that you can make the most accurate inference possible at the required confidence level. You can find an explanation of how to do this in a related question here.

Taking a nonlinear transformation of an existing interval does not give you the optimum (shortest) confidence interval (unless by an incredible coincidence!). The general method used to obtain the shortest confidence interval is to go back and look at the initial probability statement operating on the pivotal quantity used to formulate the interval. Instead of using "equal tails" in the probability statement, you set the relative tail sizes as a control variable, and then you find the formula for the length of the confidence interval conditional on that variable. Finally, you use calculus methods to determine the value of the control variable that minimises the interval length. Often this method can be programmed for broad classes of problems, allowing you to rapidly compute optimal confidence intervals for an object of interest.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ (+1) This is a nice and clear answer. Is there general framework for finding the shortest possible 95% confidence interval? Does the delta method do this? $\endgroup$ – Noah Jul 12 at 1:08
  • 1
    $\begingroup$ @Ben-ReinstateMonica I think that the main question here is to prove that the CIs are valid, which you have assumed in your answer. It might also help to provide some intuition as to why the nonlinear transformation might not lead to the shortest CI, and in what scenarios it would coincide. This obviously assuming that the OP understands why one should care about the shortest CIs. $\endgroup$ – tchakravarty Jul 12 at 17:53
  • 3
    $\begingroup$ It seems to me that CIs are generally chosen based on centering them on the estimating statistic, rather than minimizing length, although those may coincide in some cases. $\endgroup$ – Acccumulation Jul 12 at 18:31
  • $\begingroup$ @tchakravarty: I think your answer covers the validity issue well, so I am happy to focus on this other aspect of the problem. I have edited the answer to elaborate on why shortest confidence intervals are desirable. $\endgroup$ – Ben Jul 13 at 3:09
  • 3
    $\begingroup$ Here is an illustrative example of the issue: suppose $Y=\log_e(X) \sim \mathcal N(10,1)$. Then $\mathbb P(8.08 \le Y \le 11.96) \approx 0.95$ so we can consider the log-normally distributed $X$ in the way suggested and say $\mathbb P\left(e^{8.08} \le e^Y \le e^{11.96}\right) \approx \mathbb P\left(3102.7 \le X \le 156367.5\right) \approx 0.95$. But it is also true that $\mathbb P\left(574.7 \le X \le 114250.1\right) \approx 0.95$ and that is a narrower interval $\endgroup$ – Henry Jul 13 at 10:50
21
$\begingroup$

You can easily show that this is the case.

Let $Y\equiv \log(X)$. Then, the $\alpha$-quantile of $Y$ is $y\in\mathbb{R}$, such that $\mathbb{P}[Y \leq y] = \alpha$. Similarly, the $\alpha$-quantile of $X$ is $x \in \mathbb{R}^+$, such that $\mathbb{P}[X \leq x] = \alpha$, or, $\mathbb{P}[\log(X) \leq y] = \alpha$, or, $\mathbb{P}[X \leq \exp(y)] = \alpha$. Thus, $y = \exp(x)$. Note that there are regularity conditions relating to the continuity and monotonicity of the transformation function $\log$ that you need to be careful about when applying this result more generally.

| cite | improve this answer | |
$\endgroup$
7
$\begingroup$

Confidence intervals do not change when you transform the parameters (with a monotonic transformation)

Confidence intervals are based on probabilities conditional on the parameters, and do not transform if you transform the parameters. Unlike (Bayesian) probabilities of the parameters (on which credible intervals are based). See for instance in this question: If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval? a confidence interval is not just like a credible interval with a flat prior. For a confidence interval we have:

  • The boundaries of probabilities (credibility intervals) will be different when you transform the variable, (for likelihood functions this is not the case). E.g for some parameter $a$ and a monotonic transformation $f(a)$ (e.g. logarithm) you get the equivalent likelihood intervals $$\begin{array}{ccccc} a_{\min} &<& a &<& a_{\max}\\ f(a_{\min}) &<& f(a) &<& f(a_{\max}) \end{array}$$

Why is this?

See in this question Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

  • You might see the confidence intervals as being constructed as a range of values for which an $\alpha$ level hypothesis test would succeed and outside the range an $\alpha$ level hypothesis test would fail.

That is, we choose the range of $\theta$ (as a function of $X$) based on a probability conditional on the $\theta$'s. For instance

$$I_{\alpha}(X) = \lbrace \theta: F_X(\alpha/2,\theta) \leq X \leq F_X(1-\alpha/2,\theta) \rbrace$$

the range of all hypotheses $\theta$ for which the observation is inside a two-tailed $\alpha\%$ hypothesis test.

This condition, the hypotheses, does not change with the transformation. For instance, the hypothesis $\theta = 1$, is the same as the hypothesis $\log(\theta) = 0$.

Graphical intuition

You could consider a 2d view of hypotheses on the x-axis and observations on the y-axis (see also The basic logic of constructing a confidence interval):

confidence intervals

You could define a $\alpha$-% confidence region in two ways:

  • in vertical direction $L(\theta) < X < U(\theta)$ the probability for the data $X$, conditional on the parameter being truly $\theta$, to fall inside these bounds is $\alpha$ .

  • in horizontal direction $L(X) < \theta < U(X)$ the probability that an experiment will have the true parameter inside the confidence interval is $\alpha$%.

For the actual computation of the confidence interval we often use the vertical direction. We compute the boundaries for each $\theta$ as a hypothesis test. This computation will be the same for a transformed $\theta$.

So when you transform the parameter, then the image will just look the same, and only the scale on the x-axis will change. For a transformation of a probability density this is not the same and the transformation is more than just a change of the scale.

However,

Indeed like Ben has answered. There is not a single confidence interval, and there are many ways to choose the boundaries. However, whenever the decision is to make the confidence interval based on probabilities conditional on the parameters, then the transformation does not matter (like the before mentioned $I_{\alpha}(X) = \lbrace \theta: F_X(\alpha/2,\theta) \leq X \leq F_X(1-\alpha/2,\theta) \rbrace$).

I would disagree that there is a shortest possible interval.

Or at least this can not be defined in a unique way, or possibly it can be defined based on the conditional distribution of observations, but in that case transformation (of the conditional part) does not matter.

In that case (based on conditional distribution) you define the boundaries such that the vertical direction is smallest (e.g. how people often make the smallest decision boundaries for a hypothesis test). This is the most common way to determine the confidence interval. Optimizing the confidence interval such that you get the smallest interval in the vertical direction is independent from transformations of the parameter (you can see this as stretching/deforming the figure in horizontal direction, which does not change the distance between the boundaries in vertical direction).

Making the boundaries smallest in the horizontal direction is more difficult, because there is no good way to define/measure it (making the interval shorter for one observation requires making the interval larger for another, and one would need some way to weigh the different observations). It could be possible, maybe, if you use some prior for the distribution of $\theta$. In that case one could shift the choice of the boundaries (which still must be in the vertical direction to ensure 95% coverage, conditional on $\theta$, but they do not need to be optimal in vertical direction) in order to optimise some measure for the length of the interval. In that case, the transformation does indeed change the situation. But this way of constructing confidence intervals is not very typical.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ At the outset it would help to describe what you mean by "transform" a variable because many familiar functions will otherwise provide counterexamples to your claims. $\endgroup$ – whuber Jul 14 at 21:01
  • 1
    $\begingroup$ Since the question explicitly concerns an interval, the class of injective functions is too large. As far as "not very typical" goes, we do occasionally see things like a square or a trigonometric function. $\endgroup$ – whuber Jul 15 at 12:28
  • $\begingroup$ It is necessary when the intention is to map arbitrary intervals into intervals. $\endgroup$ – whuber Jul 15 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.