4
$\begingroup$

Let's say I have a maximum likelihood estimate (MLE) for my parameter $\theta$ as in $$\hat{\theta}\sim N\left(\theta, \frac{\theta^2}{n}\right) $$ and we have $$\mu = \sqrt{\frac{\theta}{2}}$$

If I estimate my $CI(\theta)$ in the form of an interval $(min_{theta}, max_{theta})$, can I then use the following as a confidence interval for $\mu,$

$$ CI(\mu) = \left(\sqrt{\frac{min_{theta}}{2}}\: , \: \sqrt{\frac{max_{theta}}{2}}\right),$$

or am I doing something terribly wrong?

I've tested this using the CLT, so having $\mu\sim N(\mu, \frac{\sigma^2}{n}),$ and I'm a getting pretty good confidence interval, but I'd like to know if it's just a special case.

$\endgroup$
8
  • $\begingroup$ Could you please explain what your notation "$min_{theta},$" etc means? $\endgroup$
    – whuber
    Jan 17 at 18:21
  • $\begingroup$ @whuber thank you for asking, it's just the lower bound of the CI and the upper bound... i gave them some name just to reference them at the end $\endgroup$ Jan 17 at 18:22
  • $\begingroup$ Thank you. How do you plan to deal with situations where either or both of those bounds is negative? This can happen even when $\theta$ itself is positive (although the chance of that happening depends on the value of $n,$ which you haven't described). It's also curious that you attempt to describe your estimate $\hat\theta$ in terms of itself -- it appears on the right hand side of your first formula. What do you really mean by that? $\endgroup$
    – whuber
    Jan 17 at 18:28
  • $\begingroup$ @whuber yes the function for $mu$ is just an example, and the $\hat{theta}$ it's for the Slutsky theorem (if i remember correctly), that "states" that we can use a "good approximation" and it's fine for example for convergence of CLT $\endgroup$ Jan 17 at 18:35
  • $\begingroup$ @whuber this wants to be an example, saying "if we have a parameter $\theta$ estimated with MLE, and a second parameter $\gamma = h(\theta)$", and we use the confidence interval of $\theta$ to estimate the confidence interval of $\gamma$? $\endgroup$ Jan 17 at 18:37

1 Answer 1

8
$\begingroup$

The question, as clarified in comments, is a general one about confidence intervals. It is best framed generally, because the generality strips away irrelevant details to bring out the main idea.

So, imagine a statistical estimation setting in which a sample $\mathbf X = (X_1,X_2,\ldots,X_n)$ is assumed to be governed by a probability law $F\in\Theta$ (a specified space of such laws). A property is a real-valued function $\theta:\Theta\to\mathbb R.$ This is convenient language to talk about things we might want to know about $F$ and, by extension, any sample $\mathbf X$ governed by $F,$ while allowing for very general applications.

By definition, a confidence interval procedure (of size $\alpha$) for a property $\theta$ is a pair of functions $l, u,$ defined on the set of all possible samples, for which

$$\inf_{F\in\Theta}{\Pr}_F\left[l(\mathbf X) \le \theta(F) \le u(\mathbf X)\right] = 1-\alpha.\tag{*}$$

(It is my purpose not to mention issues related to measurability or how things might depend on the sample size $n,$ because these would just be distractions from the main idea.)

This looks painfully abstract but conveys a relatively straightforward idea. The left hand side concerns the chance that the interval $[l(\mathbf X), u(\mathbf X)],$ as computed from the sample, covers the true (but unknown) value $\theta(F).$ The infimum applied to it means that the coverage probability might depend on $F,$ but is never less than $1-\alpha$ and can approach it arbitrarily closely (or even equal it).

The question concerns a transformation $h$ of the property. That is, $h:\mathbb R \to \mathbb R$ is some function or partial function, such as the square root (which is defined only for non-negative numbers). When the property $\theta$ takes on values in the domain of $h,$ then the composition of $\theta$ followed by $h,$ $h\circ \theta:F\to h(\theta(F)),$ is also a property. The question asks:

Given a property $\theta,$ a confidence interval procedure $(l,u),$ and a transformation $h,$ is it the case that $(h\circ l, h\circ u)$ is a confidence interval procedure for $h\circ\theta$?

In some special but common cases--usually not including the square root function in the question--the answer is yes. Specifically, assume

  1. $h$ is a strictly increasing function and

  2. The values of all upper and lower limits are in the domain of $h.$

Condition (2) assures the question even makes sense, while condition (1) guarantees that the event $l(\mathbf X) \le \theta(F) \le u(\mathbf X)$ is equivalent to the event $h\circ l(\mathbf X) \le h\circ\theta(F) \le h\circ u(\mathbf X).$ Since for any given $F$ these are the same events (as implied by the strict monotonicity of $h$), they have the same probability. Thus, the infimum in the defining equation $(*)$ is still $1-\alpha.$

That's all there is to it. But let's restate the result in English:

Because when a transformation $h$ is strictly increasing, any interval $[\lambda, \upsilon]$ covers $\theta$ if and only if $[h(\lambda), h(\upsilon)]$ covers $h(\theta),$ a procedure $l,u$ is a $1-\alpha$ confidence interval for a property $\theta:\Theta\to\mathbb R$ if and only if $h\circ l, h\circ u$ is a $1-\alpha$ confidence interval procedure for the property $h\circ\theta.$


Wait a second, you might be asking: aren't there cases where this result obviously is false? Consider a confidence interval for the mean of a Lognormal distribution. (We say $X$ has a Lognormal distribution when $\log(X)=Y$ has a Normal distribution.) This is a challenging problem with various complicated solutions. But if we let $h$ be the logarithm, then upon applying $h$ to $X$ we seem to have reduced the question to finding a confidence interval for the mean of a Normal distribution, which is simple and well known.

The resolution of this conundrum is that it's an apparent paradox of terminology, not of estimation. The preceding analysis has shown only that exponentiating the endpoints of a confidence interval for a Normal mean will give a confidence interval for the exponential of the Normal mean. However, the exponential of that mean is not the mean of the corresponding lognormal distribution. Indeed, when $Y$ has a Normal$(\mu,\sigma^2)$ distribution, the mean of $X=\exp(Y)$ is not $\exp(\mu):$ instead, it is $\exp(\mu + \sigma^2/2).$

To put it the other way: $\exp(\mu)$ is the geometric mean of the Lognormal distribution. We conclude that the exponentials of the limits of a CI for the mean of a Normal distribution give a CI for the geometric mean of the corresponding Lognormal distribution.

Thus, the paradox is resolved by pointing out that

$h(\theta)$ and $\theta,$ as properties of $h(\mathbf X)$ and $\mathbf X,$ respectively, might not necessarily be called by the same names.

Finally, just to conclude this example: A confidence interval for the arithmetic mean of a sample $\mathbf X$ from a Lognormal$(\mu,\sigma^2)$ distribution is equivalent to a confidence interval for the property $\mu+\sigma^2/2$ in a sample $\mathbf Y$ from a Normal$(\mu,\sigma^2)$ distribution. One merely has to exponentiate the latter. Proof: the exponential is defined everywhere and is strictly increasing.

$\endgroup$
6
  • $\begingroup$ thank you so much, this is amazing $\endgroup$ Jan 17 at 22:19
  • $\begingroup$ just to conclude the answer, for the 2 points you listed, then in the example show, we actually can do the transformation of the CI from one parameter to the other, since the sqrt is defined for each sup and inf of $theta/2$, and because h is a strictly increasing function (but we would have to define $\theta > 0$ to avoid having the sqrt of negatives) $\endgroup$ Jan 17 at 22:23
  • $\begingroup$ Don't confuse the values of the limits with the parameter! Although the parameter $\theta$ might be positive, there is some chance that one or even both limits will be negative in your situation. Thus you cannot leap to the conclusion that your approach works. What is needed is an argument that it is very improbable for the lower limit to be negative. That will be the case as $n$ grows large. $\endgroup$
    – whuber
    Jan 17 at 22:24
  • $\begingroup$ can you please point to some reference to the lim sup and lim inf you are referring to? i might be confusing the definition or what thing you are referring to in the phrase The values of all upper and lower limits $\endgroup$ Jan 17 at 22:33
  • $\begingroup$ I am not referring to lim inf or lim sup: I am discussing the usual confidence limits. You have posited a situation in which data are drawn from some kind of Normal distribution. It is perfectly possible for a lower confidence limit to be negative even when the mean of the underlying distribution itself is positive. $\endgroup$
    – whuber
    Jan 17 at 22:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.