Lagrange multipliers and confidence interval

Question

Suppose $X_1,...X_n$ is a random sample from $X$~$N(\mu,\sigma^2)$, with $\sigma^2$ unknow. If $$[\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}};\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}}]$$ is a confidence interval for $\mu$ where $a_1+a_2=a$ and $z_w$ is such that $P(X\leq z_w)=w$ where $Z$~$N(0,1)$. Show that the length of the invertal is shortest when $a_1=a_2=\frac{a}{2}$

I know that the lenght of interval $L$ is $$\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}}-(\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}})=-\frac{\sigma}{\sqrt{n}}(z_{a_1}+z_{a_2})$$

First I have that $$P(\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}}\leq\mu\leq\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}})=1-\alpha$$ developing this I have that $$\phi(z_{a_1})+\phi(z_{a_2})=\alpha$$

Suppose that $f(z_{a_1},z_{a_2})=z_{a_1}+z_{a_2}$

and that the condition I have is $g(z_{a_1},z_{a_2})=\phi(z_{a_1})+\phi(z_{a_2})=\alpha$

Now I have that $$(1)f_{z_{a_1}}=\lambda g_{z_{a_1}} \Leftrightarrow 1=\lambda\frac{dg}{dz_{a_1}}\phi(z_{a_1})$$ $$(2)f_{z_{a_2}}=\lambda g_{z_{a_2}} \Leftrightarrow 1=\lambda\frac{dg}{dz_{a_2}}\phi(z_{a_2})$$ $$(3)g(z_{a_1},z_{a_2})=\phi(z_{a_1})+\phi(z_{a_2})=\alpha$$

Making $(1)-(2)$ $$(4)\lambda[\frac{dg}{dz_{a_1}}\phi(z_{a_1})-\frac{dg}{dz_{a_2}}\phi(z_{a_2})]=0$$

$(4)$ is valid if $\lambda=1$ and $\frac{dg}{dz_{a_1}}\phi(z_{a_1})=\frac{dg}{dz_{a_2}}\phi(z_{a_2})$, from this I have that

$$\phi(z_{a_1})+\phi(z_{a_2})=\phi(z_{a_1})+\phi(z_{a_1})=a_1+a_1=2a_1=\alpha\Rightarrow a_1=\frac{\alpha}{2}$$ and similary $a_2=\frac{\alpha}{2}$

Answer 1

Let me elaborate somewhat over the comment of @AaronZeng:

Let $f$ be a unimodal density function of a continuous r.v. (the latter for simplicity). If the interval $[a,b]$ satisfies

1. $\int_a^bf(x)dx=1-\alpha$
2. $f(a)=f(b)>0$
3. $a\leqslant x^* \leqslant b$, where $x^*$ is the mode of $f(x)$,

then $[a,b]$ is the shortest interval satisfying 1.

Proof: For some fixed $c$, Leibniz' Theorem gives that $$ \frac{d}{da}\int_a^{a+c}f(x)dx=f(a+c)-f(a) $$ Setting this derivative to zero gives the first order condition $f(a+c)=f(a)$. This is a maximum, because unimodality and continuity imply that $a\leqslant x^*\leqslant a+c$ and continuity implies that for any $a'>(<)a$, $\int_a^{a'}f(x)dx>(<)\int_{a+c}^{a'+c}f(x)dx$, such that $$ \int_a^{a+c}f(x)dx>\int_{a'}^{a'+c}f(x)dx\qquad\text{for any }a' $$ Now, choose $a^*$ and $c^*$ such that $\int_{a^*}^{a^*+c^*}f(x)dx=1-\alpha$ and $f(a^*+c^*)=f(a^*)$. By the previous results, that maximizes the area under the integral, to the desired area $1-\alpha$. Hence $\int_{a^*+\epsilon}^{a^*+c^*+\epsilon}f(x)dx<1-\alpha$, so that we would need to widen the interval to some $[a^*+\epsilon,a^*+c^*+\epsilon+\delta]$ to maintain a coverage probability of $1-\alpha$.

Here is a figure trying to illustrate: First, we have a 90% equitailed confidence interval between the 5%-quantile $-z_{\alpha/2}$ and 95%-quantile $z_{\alpha/2}$. Second, the interval between the 9%- and 99%-quantile is also a 90%-c.i., but we can see that it is longer: we must move the right endpoint by more to the right than the distance between the two left endpoints to ensure the same area under the density, as the density increases to the right of $-z_{\alpha/2}$, but decreases to the right of $z_{\alpha/2}$.

Answer 2

I avoid discussing "z-scores" and "$p$-values" because I have very little understanding of what these things, so dear to the statistical heart, actually mean.

You have $n$ independent identically distributed normal random variables $X_i, 1 \leq i \leq n$ with unknown mean $\mu$ but known variance $\sigma^2$. Their average $$\bar{X} = \frac 1n \sum_{i=1}^n X_i$$ is also a normal random variable with $\mu$ but smaller variance $\sigma^2/n$.

---

Now, for a $N(\mu,\tau^2)$ random variable $Y$, $$P\{\mu-1.96 \tau \leq Y \leq \mu+1.96\tau\} = 0.95\tag{1}$$ If we knew the value of $\mu$, we could construct the interval $\mathcal I = [\mu-1.96 \tau, \leq \mu+1.96\tau]$ and be assured that when we do the experiment over and over, on $95\%$ of such trials, the value of $Y$ will lie in $\mathcal I$. We don't know $\mu$, however. But we have observed that $Y$ has taken on value $y$ in the trial just concluded. Now, if $y \in \mathcal I$, then it must be that $\mu \in [y-1.96 \tau, y + 1.96 \tau]$. We don't know whether or not the specific $y$ that we observed lies in $\mathcal I$, but we might feel some confidence, perhaps 95% confidence, that $y$ is one of the "good guys" that belongs to belongs to $\mathcal I$, and so we are 95% confident that $\mu \in [y-1.96 \tau, y + 1.96 \tau]$.

---

The midpoint of the interval $\mathcal I$ defined by $(1)$ is $\mu$. Now, there are uncountably many real numbers $a$ such that $$P\{a \leq Y \leq b(a)\} = 0.95\tag{2}$$ (Here $b(a)$ is a number, obviously dependent on $a$ such that the equality holds in $(2)$. The function $b(a)$ has value approximately $\mu+1.65\tau$ when $a = -\infty$. It increases slowly and has value $\mu+1.96\tau$ when $a=\mu-1.96\tau$. It then increases very rapidly and approaches $\infty$ as $a$ gets close to $\mu-1.65\tau$. There are no solutions to $(2)$ if $a > \mu-1.65\tau$. So, when does the length $b(a)-a$ of the interval have a minimum? Symmetry suggests that the minimum occurs when $a$ and $b(a)$ are equally far from the mean $\mu$, and we can do a formal proof via calculus as in Christoph Hanck's answer.

---

I have not thought about applying Lagrange multipliers to the problem.