Assuming I know the density of my random sample - for example:

$$ f_\Theta(x) = \frac{e^{-x/\Theta^2}}{\Theta^2} $$

(but I care about general single parameter case)

How can I obtain $1-\alpha$ confidence interval of $\Theta$? I've read lots of resources that describe estimation of expected value and/or variance. Alternatively general $\Theta$ of normal distribution. However, I have non-normal one.

The process is as follows. I'm looking for $T_1(x_1,\ldots,x_n)$ and $T_2(x_1,\ldots,x_n)$ such that

$$ P(T_1(x_1,\ldots,x_n) < \Theta < T_2(x_1,\ldots,x_n)) = 1 - \alpha $$

Obviously, all I need to do is decide which statistics to use. Is there any general solution? I though of CLT - but what if I have only 20 samples?

  • $\begingroup$ This is an exponential distribution. Googling "confidence interval exponential distribution" should be fruitful. $\endgroup$ – Stéphane Laurent May 23 '15 at 9:07
  • $\begingroup$ Is this for some subject? It's a different (non-standard) parameterization of an exponential; forming an interval in either of the standard parameterizations is straightforward, and then it's easy to back out an interval for $\Theta$. If you want to do it from scratch, it's a simple matter to write down a pivotal quantity and back out an interval for $\Theta^2$.. $\endgroup$ – Glen_b May 23 '15 at 9:12
  • $\begingroup$ @StéphaneLaurent Yes, of course. But what if the given distribution isn't well known such as the provided example? It may be very "wild". How to estimate it in that case? $\endgroup$ – petrbel May 23 '15 at 9:17

An exact confidence interval is possible here. To see this, for the moment let's take $\theta=\theta^2$ to get something we are familiar with. Indeed, in this case the density becomes

$$f_{X} \left(x;\theta \right)=\begin{cases} \frac{1}{\theta} \exp\left\{-\frac{x}{\theta} \right\} & 0<x<\infty \\ 0 & \text{otherwise} \end{cases} $$

which we immediately recognise as a $Gamma\left(1,\theta \right)$ density. Assuming now you have a random sample of $X_1,X_2, \ldots, X_n $, then the random variable $Y=\sum_{i=1}^n X_i$ is a Gamma random variable with parameters $n $ and $\theta$. You might notice that we are working with the sufficient statistic.

But now we see that the distribution of $\frac{Y}{\theta}$ is actually $\Gamma \left(n,1 \right)$ and does not depend on $\theta$. This observation allows us a to construct a confidence interval for $\theta$. Indeed, choosing appropriate quantiles we get

$$P \left[ \Gamma\left(\alpha/2,n,1 \right) <\frac{Y}{\theta}<\Gamma\left(1-\alpha/2,n,1 \right) \right]=1-\alpha $$

which implies that

$$P \left[ \frac{Y}{\Gamma\left(1-\alpha/2,n,1 \right)} <\theta<\frac{Y}{\Gamma\left(\alpha/2,n,1 \right)}\right]=1-\alpha $$

and we have a Confidence Interval.

If you now take square roots, you will be back to your original scale. That's good, right? If I were you, I would comppare the asymptotic CLT based CI with this one for a sequence of sample sizes. You will of course see that for small $n$ the difference can be quite large, the asymptotic CI being symmetric and all.

As a general rule, whenever you are confronted with such a problem try to obtain a pivot, that is a quantity whose distribution does not depend on the parameter of interest. If you can find a function of that quantity that has a well-known distribution, or at least a recognizable distribution, a CI may be constructed using a similar procedure. It is often helpful to start with the sufficient statistic.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.