2
$\begingroup$

I have been trying to construct a $1-\alpha$ confidence interval for the mean. The distribution from which a sample is drawn is exponential distribution with density:

$$p(x) = \frac{1}{\beta}e^{-x/\beta}$$

The exponential distribution has mean $\mu=\beta$ and standard deviation $\sigma=\beta$. As far as my knowledge is concerned, for a sample $X_1, X_2, \cdots, X_n$ of size $n$, the confidence interval :

$$\left(\bar{X}_n-z_{\alpha/2}\frac{\sigma}{\sqrt{n}}, \bar{X}_n+z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\right)$$

should trap the true $\mu$ with $1-\alpha$ chance (I am not saying that this interval contains $\mu$ with probability $1-\alpha$; that will be wrong since $\mu$ is not a random variable).

However, this assumes that population standard deviation $\sigma$ is known. Since I want to simulate a situation where that too needs to be estimated, I constructed the forllowing interval:

$$\left(\bar{X}_n-t_{\alpha/2}\frac{\hat{\sigma}}{\sqrt{n}}, \bar{X}_n+t_{\alpha/2}\frac{\hat{\sigma}}{\sqrt{n}}\right)$$

where, $$\hat{\sigma}=\sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar{X}_n)^2}$$

and $t_{\alpha/2}$ is the critical t-value obtained for the Student's t-distribution for $n-1$ degrees of freedom.

I wrote a small python script to verify that with this modification, the constructed interval is indeed $95\%$ confidence interval when $\alpha=0.05$. For my simulations, I chose $n=10$, so that we have $9$ degrees of freedom, $t_{\alpha/2}=2.262$. I could verify that the first interval where population variance is known indeed contains true $\mu$ around $95\%$ of the time. (I generate $10000$ intervals and count how many contain the known $\mu$). However, the second one seems to contain $\mu$ only $90\%$ of the time. I am not sure why even after correctly choosing the correct critical $t$ this is happening. If I increase $t_{\alpha/2}=3.182$ corresponding to 3 degrees of freedom, I do get $95\%$ interval again! But of course this makes no sense because the degrees of freedom is $9$ in this case. Any idea what is happening?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Your confidence interval formula is inappropriate for your probability model. It will be approximately correct for very large samples -- much larger than $n=10$ -- due to the Central Limit Theorem. Try your script with, say, $n=500.$ $\endgroup$
    – whuber ♦
    Commented Apr 18, 2021 at 15:28
  • $\begingroup$ The way you’ve written your exponential PDF, the mean is $\beta$, and the variance is $\beta^2$. You can’t know one without knowing the other. You seem to get this, but others might not pick up on this fact.) $\endgroup$
    – Dave
    Commented Apr 18, 2021 at 15:40
  • $\begingroup$ @whuber : I didn't get you. What is the appropriate way in this case? $\endgroup$
    – Peaceful
    Commented Apr 18, 2021 at 15:52

1 Answer 1

Reset to default
2
$\begingroup$

If you have a random sample of size $n$ from $\mathsf{Exp}(\mathrm{rate}=1/\beta),$ (with mean $\beta),$ then $\bar X/\beta \sim \mathsf{Gamma}(\mathrm{shape}=n,\mathrm{rate} = n).$ Thus a 95% CI for $\beta$ is of the form $\left(\frac{\bar X}{U},\, \frac{\bar X}{L}\right),$ where $L$ and $U$ cut probability $0.025$ from lower and upper tails, respectively of $\mathsf{Gamma}(n,n).$

For example, let's take a random sample of size $n=30$ from an exponential distribution with mean $\mu = 15.$ In R below, I got $\bar X =14.08$ and 95% CI $(10.14,\, 20.86),$ which does happen to contain $\mu = 15.$ Of course, with real data from a real population, one never knows for sure if the 95% CI covers the unknown $\mu.$

set.seed(1234)
x = rexp(30, 1/15)
summary(x)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
 0.05992  4.10425 11.88384 14.07736 24.64535 45.78687 
mean(x)/qgamma(c(.975,.025), 30, 30)
[1] 10.14004 20.86475

Three additional runs with no seed specified, gave intervals $(12.89,\, 26.52),$ $(7.36,\, 15.15),$ and $(11.02,\, 22.67).$ But a run using todays date as set.seed(418) missed the mark with CI $(15.29,\, 31.46).$

Improve this answer
$\endgroup$
8
  • $\begingroup$ @BrudeET: So do you mean that my assumption of t-distribution is violated for exponential? $\endgroup$
    – Peaceful
    Commented Apr 18, 2021 at 16:11
  • $\begingroup$ You wrongly assume normal data when you use a z-interval. In my method you can start with $P(L \le \bar X/\beta \le U) = 0.95,$ with $L$ and $U$ chosen as stated. Then 'pivot' to isolate $\beta$ in the inequality to get $P(\bar X/U \le \beta \le \bar X/L) = 0.95,$ which provides the basis for the form of CI that I suggested. $\endgroup$
    – BruceET
    Commented Apr 18, 2021 at 16:14
  • $\begingroup$ Thanks! I should also pay attention to how fast $\bar{X}_n$ approaches normal distribution. For different underlying distributions, convergence is different. $\endgroup$
    – Peaceful
    Commented Apr 18, 2021 at 16:20
  • 2
    $\begingroup$ Better yet, use exactly the right CI for the situation at hand, if you can find it. No use needlessly relying of the CLT or other methods of approximation. // The CLT works OK when $\sigma$ is known because $Z = \frac{\bar X -\mu}{\sigma/\sqrt{n}}$ converges to standard normal as $n$ becomes large. But you have to be somewhat more cautious assuming that the dist'n of $T = \frac{\bar X -\mu}{S/\sqrt{n}}$ is approximated by Student's dist'n. // Some people seem to think t intervals are automatically OK if $n > 30,$ but that is not really correct. $\endgroup$
    – BruceET
    Commented Apr 18, 2021 at 16:33
  • $\begingroup$ That's a very useful advice. Thanks. So when exactly is t-distribution helpful? When n is large, we anyway don't need it. When n is small, you seem to imply that only when the underlying distribution is normal, t-distribution is of any use. $\endgroup$
    – Peaceful
    Commented Apr 19, 2021 at 2:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.