Estimating confidence interval of a parameter from the MLE of another parameter

Question

Let's say I have a maximum likelihood estimate (MLE) for my parameter $\theta$ as in $$\hat{\theta}\sim N\left(\theta, \frac{\theta^2}{n}\right) $$ and we have $$\mu = \sqrt{\frac{\theta}{2}}$$

If I estimate my $CI(\theta)$ in the form of an interval $(min_{theta}, max_{theta})$, can I then use the following as a confidence interval for $\mu,$

$$ CI(\mu) = \left(\sqrt{\frac{min_{theta}}{2}}\: , \: \sqrt{\frac{max_{theta}}{2}}\right),$$

or am I doing something terribly wrong?

I've tested this using the CLT, so having $\mu\sim N(\mu, \frac{\sigma^2}{n}),$ and I'm a getting pretty good confidence interval, but I'd like to know if it's just a special case.

Answer 1

The question, as clarified in comments, is a general one about confidence intervals. It is best framed generally, because the generality strips away irrelevant details to bring out the main idea.

So, imagine a statistical estimation setting in which a sample $\mathbf X = (X_1,X_2,\ldots,X_n)$ is assumed to be governed by a probability law $F\in\Theta$ (a specified space of such laws). A property is a real-valued function $\theta:\Theta\to\mathbb R.$ This is convenient language to talk about things we might want to know about $F$ and, by extension, any sample $\mathbf X$ governed by $F,$ while allowing for very general applications.

By definition, a confidence interval procedure (of size $\alpha$) for a property $\theta$ is a pair of functions $l, u,$ defined on the set of all possible samples, for which

$$\inf_{F\in\Theta}{\Pr}_F\left[l(\mathbf X) \le \theta(F) \le u(\mathbf X)\right] = 1-\alpha.\tag{*}$$

(It is my purpose not to mention issues related to measurability or how things might depend on the sample size $n,$ because these would just be distractions from the main idea.)

This looks painfully abstract but conveys a relatively straightforward idea. The left hand side concerns the chance that the interval $[l(\mathbf X), u(\mathbf X)],$ as computed from the sample, covers the true (but unknown) value $\theta(F).$ The infimum applied to it means that the coverage probability might depend on $F,$ but is never less than $1-\alpha$ and can approach it arbitrarily closely (or even equal it).

The question concerns a transformation $h$ of the property. That is, $h:\mathbb R \to \mathbb R$ is some function or partial function, such as the square root (which is defined only for non-negative numbers). When the property $\theta$ takes on values in the domain of $h,$ then the composition of $\theta$ followed by $h,$ $h\circ \theta:F\to h(\theta(F)),$ is also a property. The question asks:

Given a property $\theta,$ a confidence interval procedure $(l,u),$ and a transformation $h,$ is it the case that $(h\circ l, h\circ u)$ is a confidence interval procedure for $h\circ\theta$?

In some special but common cases--usually not including the square root function in the question--the answer is yes. Specifically, assume

1. $h$ is a strictly increasing function and

2. The values of all upper and lower limits are in the domain of $h.$

Condition (2) assures the question even makes sense, while condition (1) guarantees that the event $l(\mathbf X) \le \theta(F) \le u(\mathbf X)$ is equivalent to the event $h\circ l(\mathbf X) \le h\circ\theta(F) \le h\circ u(\mathbf X).$ Since for any given $F$ these are the same events (as implied by the strict monotonicity of $h$), they have the same probability. Thus, the infimum in the defining equation $(*)$ is still $1-\alpha.$

That's all there is to it. But let's restate the result in English:

Because when a transformation $h$ is strictly increasing, any interval $[\lambda, \upsilon]$ covers $\theta$ if and only if $[h(\lambda), h(\upsilon)]$ covers $h(\theta),$ a procedure $l,u$ is a $1-\alpha$ confidence interval for a property $\theta:\Theta\to\mathbb R$ if and only if $h\circ l, h\circ u$ is a $1-\alpha$ confidence interval procedure for the property $h\circ\theta.$

---

Wait a second, you might be asking: aren't there cases where this result obviously is false? Consider a confidence interval for the mean of a Lognormal distribution. (We say $X$ has a Lognormal distribution when $\log(X)=Y$ has a Normal distribution.) This is a challenging problem with various complicated solutions. But if we let $h$ be the logarithm, then upon applying $h$ to $X$ we seem to have reduced the question to finding a confidence interval for the mean of a Normal distribution, which is simple and well known.

The resolution of this conundrum is that it's an apparent paradox of terminology, not of estimation. The preceding analysis has shown only that exponentiating the endpoints of a confidence interval for a Normal mean will give a confidence interval for the exponential of the Normal mean. However, the exponential of that mean is not the mean of the corresponding lognormal distribution. Indeed, when $Y$ has a Normal$(\mu,\sigma^2)$ distribution, the mean of $X=\exp(Y)$ is not $\exp(\mu):$ instead, it is $\exp(\mu + \sigma^2/2).$

To put it the other way: $\exp(\mu)$ is the geometric mean of the Lognormal distribution. We conclude that the exponentials of the limits of a CI for the mean of a Normal distribution give a CI for the geometric mean of the corresponding Lognormal distribution.

Thus, the paradox is resolved by pointing out that

$h(\theta)$ and $\theta,$ as properties of $h(\mathbf X)$ and $\mathbf X,$ respectively, might not necessarily be called by the same names.

Finally, just to conclude this example: A confidence interval for the arithmetic mean of a sample $\mathbf X$ from a Lognormal$(\mu,\sigma^2)$ distribution is equivalent to a confidence interval for the property $\mu+\sigma^2/2$ in a sample $\mathbf Y$ from a Normal$(\mu,\sigma^2)$ distribution. One merely has to exponentiate the latter. Proof: the exponential is defined everywhere and is strictly increasing.