# What “nice” property of a confidence set is being violated

Suppose that I have $$X_{i} \overset{i.i.d.}{\sim} P$$ with $$E[X_{i}]=\mu$$ and $$V[X_{i}^{2}] = \sigma^{2}<\infty$$.

Then by the central limit theorem I know that: \begin{align} \sqrt{n} (\bar{X}_{n} - \mu) \overset{d}{\to} N(0,\sigma^{2}) \end{align} where $$\bar{X}_{n}$$ is the sample average. Suppose for some silly reason I know the value of $$\sigma^{2}$$. Then this asymptotic approximation allows me to justify confidence sets for $$\mu$$ of the form: \begin{align} \bar{X}_{n} \pm q_{\alpha/2} \sqrt{\frac{\sigma^{2}}{n}} \end{align} where $$q_{\alpha/2}$$ is the $$\alpha/2^{th}$$ quantile of the standard normal. In particular: \begin{align} \lim_{n \to \infty} P \left(q_{\alpha/2} \leq \sqrt{n} \frac{(\bar{X}_{n} - \mu)}{\sigma} \leq -q_{\alpha/2} \right) = 1-\alpha\\ \implies \lim_{n \to \infty} P \left(\bar{X}_{n} + q_{\alpha/2}\frac{\sigma}{\sqrt{n}} \leq \mu \leq \bar{X}_{n} -q_{\alpha/2}\frac{\sigma}{\sqrt{n}} \right) = 1-\alpha\\ \end{align} For simplicity, let: $$CI_{1} = \left[\bar{X}_{n} + q_{\alpha/2}\frac{\sigma}{\sqrt{n}} , \bar{X}_{n} -q_{\alpha/2}\frac{\sigma}{\sqrt{n}} \right]$$ Now suppose that I am a strange statistician, and that rather than the confidence interval constructed above, I prefer a confidence interval (for whatever reason) of my own making: $$CI_{2} = \left[\bar{X}_{n} + q_{\alpha/2}\frac{\sigma}{\sqrt{n}}+b_{n} , \bar{X}_{n} -q_{\alpha/2}\frac{\sigma}{\sqrt{n}} -b_{n}\right]$$ where $$b_{n} = o(n^{-1/2})$$ is some vanishing deterministic sequence. Note that $$CI_{2}$$ also provides $$1-\alpha$$ coverage probability asymptotically.

My question: is there any reason to prefer $$CI_{1}$$ to $$CI_{2}$$? Asymptotically they are the same, so I suspect any reason would need to appeal to finite sample arguments. For example, I can always construct the sequence $$b_{n}$$ such that $$CI_{1}$$ and $$CI_{2}$$ are VERY different in finite sample. So what statistical justification would lead someone to use $$CI_{1}$$ versus $$CI_{2}$$? Is there a name for the desirable property $$CI_{1}$$ possesses that $$CI_{2}$$ does not?

Thanks so much!

• $CI_1$ is shorter, therefore (in some sense) more informative. If you have no reason to assume $CI_1$ is significantly biased towards being too short, why use a longer interval that conveys less information about where the actual value may be? – jbowman Apr 4 '19 at 17:43
• @jbowman why is $CI_1$ shorter? If the sequence $b_n$ is positive that will not be the case. – möbius Apr 4 '19 at 18:14
• @jbowman why does it not have the stated coverage probability? The asymptotic probability that $\mu$ is in $CI_2$ by construction of the vanishing sequence $b_n$. The point of this question is that both confidence have the same coverage probability asymptotically, as was stated in the question. – möbius Apr 4 '19 at 20:33
• @jbowman I think you're missing the point of the question. I interpreted it as "both $CI_1$ and $CI_2$ are asymptotically level $\alpha$ confidence intervals, so why should $CI_1$ be used over $CI_2$?" I think this is a good question (+1), since the utility of $CI_1$ is usually justified by its asymptotic coverage. Meanwhile there are few coverage guarantees for $CI_1$ for finite samples. Maybe something like the Berry-Esseen theorem can be used to give bounds for the coverage probability of $CI_1$, but nevertheless $CI_1$ and $CI_2$ do not give exactly $1-\alpha$ coverage for finite samples – Artem Mavrin Apr 4 '19 at 21:30
• Still, I do see your point; but it does look to me like it comes down to "I don't care about finite sample properties of my CIs, so why shouldn't I use any CI I want as long as it has the same asymptotic properties?" I could construct an estimate of the mean of a Normal distribution as $\sum x_i/(1000000000+n)$, too, as asymptotically it's the same as $\bar{x}$, but few people care so little about the finite sample properties of their estimators as to do that. – jbowman Apr 4 '19 at 21:39

Non sunt multiplicanda entia sine necessitate.

Entities are not to be multiplied without necessity.

Occam's razor, in other words.

...which in our situation implies that it is not the $$CI_1$$ that has to defend itself, rather, it is the $$CI_2$$ that has to convince us of the necessity of including the $$b_n$$ sequence.

The "for whatever reason" offered by the OP as the reason to include the $$\{b_n\}$$ sequence does not cut it, not by a long shot.

This may appear to be a very abstract philosophical desideratum that is lost rather than a tangible and desirable statistical property, but this is not the case, as the following example indicates:

Consider the sequence

$$\begin{cases} {b_n} = \text{Graham's number}\;\;\;\;\;\; n=1,..., 10000 \\b_n=o(1) \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; n>10000\end{cases}$$

Assume that our sample size is, hmm, $$n=7358$$. Then we would add to our confidence interval Graham's number, rendering it completely trivial.

This is not a straw-man argument. It tells us that if there is a single example where we can with confidence reject a proposed $$\{b_n\}$$ sequence, then it follows that the $$\{b_n\}$$ sequence that we will actually propose would be faced with the eternal question "$$Why\, ?$$"... and in science, the answer "$$Why \;not\,?$$ is a great way to start but a terrible way to end research and inference.

And justifying the use of a $$\{b_n\}$$ sequence is bound to be, case-specific, sample-specific, reasearch-purpose-specific, etc.

I would like to say a "Good" statistician can prefer $$CI_1$$ or $$CI_2$$ depending on the context. For example, if the underlying distribution is symmetric (or even better it is Gaussian), symmetric confidence interval would be preferred. From the same reason, if we know the underlying distribution is skewed (e.g., Gamma distribution), we can prefer a properly skewed confidence set.

If we seriously concern about finite-sample performance, neither $$CI_1$$ or $$CI_2$$ is good. We can always use finite-sample confidence sets based on a proper tail condition on the distribution. (For example, Chernoff bounds under sub-Gaussian condition.)

As a remark, if we want to use $$CI_2$$, it is always important to provide reasonable justification of the choice of $$b_n$$. If we cannot provide a proper justification, there is no reason to use it. To choose $$b_n$$ properly, we may need a decent amount of information about the underlying distribution. If we have no such information, and if we do not like the $$CI_1$$, there are many reasonable alternatives (e.g., Bootstrap confidence interval)

• I do not think your first point addresses the question. For example, both $CI_{1}$ and $CI_2$ are centered on the sample mean, so now what? How I can I prefer one to the other? I also think you must implicitly have a finite sample property in mind in your first point, since confidence sets are constructed with respect to the sampling distribution of the sample mean, which is always normal (thus symmetric) under weak conditions. – möbius Apr 14 '19 at 12:45
• I am sympathetic to your second point, but again your third point does not address the question. "If we cannot provide a justification there is no reason to use it." By similar logic, I might reasonably say "if we cannot provide a justification not to use it, there is no reason not to use it." – möbius Apr 14 '19 at 12:46