I was thinking about a very basic matter, and arrived at the conclusion I know a lot less about it than I thought.

When people have data $ X_1^N $ sampled i.i.d. from a distribution: $ X_1^N \sim X $, it is customary to estimate the mean by $ \bar{X} = \frac{1}{N} \sum_{i=1}^N X_i $ and to estimate the confidence interval for the true mean $ \mu $ in the 95% confidence interval:

$$ \mu \in \left[\bar{X} - 1.96 \cdot \frac{\hat{std}(X)}{\sqrt{N}}, \bar{X} + 1.96 \cdot \frac{\hat{std}(X)}{\sqrt{N}}\right] $$

Where $ \hat{std}(X) $ is some estimate for the standard deviation of X.

Now, this comes from an assumption that X follows a normal distribution. Otherwise, this confidence interval is only valid asymptotically.

But, if the variable $ X $ belongs to $ \mathcal{L}^2 $, that is, if the standard deviation exists and is finite, could we derive a precise, non-asymptotic interval?

The answer is yes. By Markov inequality,

$$ \mathbf{P} \left[ \left| \frac{1}{N} \sum_{i=1}^N X_i - \mu \right| \gt \epsilon \right] \leq \frac{ \mathbf{P} \left[ \left( \frac{1}{N} \sum_{i=1}^N X_i - \mu \right)^2 \right] }{\epsilon^2} $$ $$ = \frac{Var(X)}{N \epsilon^2} $$

So if we want a 95% confidence interval, (that is, $ \mathbf{P} \left[ \left| \frac{1}{N} \sum_{i=1}^N X_i - \mu \right| \gt \epsilon \right] \leq 0.05 $), we might take:

$$ \epsilon = \sqrt{ \frac{Var(X)}{0.05 N}} \sim 4.47 \frac{std(X)}{\sqrt{N}} $$

So, if we want an exact (except by the error in $ \hat{std(X)} $), non-asymptotic interval for $ \mu $, instead of the one above, we should do:

$$ \mu \in \left[\bar{X} -4.47 \cdot \frac{\hat{std}(X)}{\sqrt{N}}, \bar{X} + 4.47 \cdot \frac{\hat{std}(X)}{\sqrt{N}}\right] $$

Is it a "bad" non-asymptotic confidence interval? That is, is it possible to find a better one, valid for any distribution in $ \mathcal{L}^2 $?

I imagine there could be a multiplicative factor that is a function of N, such that the number 4.47 progressively turns into 1.96 while N grows. But, for a variable in $ \mathcal{L}^2 $, I am not aware of any better bound than the one coming from Markov's inequality.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.