Confidence interval for known non-normal estimation? Assuming I know the density of my random sample - for example:
$$
f_\Theta(x) = \frac{e^{-x/\Theta^2}}{\Theta^2}
$$
(but I care about general single parameter case)
How can I obtain $1-\alpha$ confidence interval of $\Theta$? I've read lots of resources that describe estimation of expected value and/or variance. Alternatively general $\Theta$ of normal distribution. However, I have non-normal one.
The process is as follows. I'm looking for $T_1(x_1,\ldots,x_n)$ and $T_2(x_1,\ldots,x_n)$ such that
$$
P(T_1(x_1,\ldots,x_n) < \Theta < T_2(x_1,\ldots,x_n)) = 1 - \alpha
$$
Obviously, all I need to do is decide which statistics to use. Is there any general solution? I though of CLT - but what if I have only 20 samples?
 A: An exact confidence interval is possible here. To see this, for the moment let's take $\theta=\theta^2$ to get something we are familiar with. Indeed, in this case the density becomes
$$f_{X} \left(x;\theta \right)=\begin{cases} \frac{1}{\theta} \exp\left\{-\frac{x}{\theta} \right\} & 0<x<\infty \\ 0 & \text{otherwise} \end{cases} $$
which we immediately recognise as a $Gamma\left(1,\theta \right)$ density. Assuming now you have a random sample of $X_1,X_2, \ldots, X_n $, then the random variable $Y=\sum_{i=1}^n X_i$ is a Gamma random variable with parameters $n $ and $\theta$. You might notice that we are working with the sufficient statistic.
But now we see that the distribution of $\frac{Y}{\theta}$ is actually $\Gamma \left(n,1 \right)$ and does not depend on $\theta$. This observation allows us a to construct a confidence interval for $\theta$. Indeed, choosing appropriate quantiles we get
$$P \left[ \Gamma\left(\alpha/2,n,1 \right) <\frac{Y}{\theta}<\Gamma\left(1-\alpha/2,n,1 \right) \right]=1-\alpha $$
which implies that
$$P \left[ \frac{Y}{\Gamma\left(1-\alpha/2,n,1 \right)} <\theta<\frac{Y}{\Gamma\left(\alpha/2,n,1 \right)}\right]=1-\alpha $$ 
and we have a Confidence Interval.
If you now take square roots, you will be back to your original scale. That's good, right? If I were you, I would comppare the asymptotic CLT based CI with this one for a sequence of sample sizes. You will of course see that for small $n$ the difference can be quite large, the asymptotic CI being symmetric and all.
As a general rule, whenever you are confronted with such a problem try to obtain a pivot, that is a quantity whose distribution does not depend on the parameter of interest. If you can find a function of that quantity that has a well-known distribution, or at least a recognizable distribution, a CI may be constructed using a similar procedure. It is often helpful to start with the sufficient statistic.
