# Lagrange multipliers and confidence interval

Suppose $X_1,...X_n$ is a random sample from $X$~$N(\mu,\sigma^2)$, with $\sigma^2$ unknow. If $$[\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}};\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}}]$$ is a confidence interval for $\mu$ where $a_1+a_2=a$ and $z_w$ is such that $P(X\leq z_w)=w$ where $Z$~$N(0,1)$. Show that the length of the invertal is shortest when $a_1=a_2=\frac{a}{2}$

I know that the lenght of interval $L$ is $$\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}}-(\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}})=-\frac{\sigma}{\sqrt{n}}(z_{a_1}+z_{a_2})$$

First I have that $$P(\overline{X}+z_{a_2}\frac{\sigma}{\sqrt{n}}\leq\mu\leq\overline{X}-z_{a_1}\frac{\sigma}{\sqrt{n}})=1-\alpha$$ developing this I have that $$\phi(z_{a_1})+\phi(z_{a_2})=\alpha$$

Suppose that $f(z_{a_1},z_{a_2})=z_{a_1}+z_{a_2}$

and that the condition I have is $g(z_{a_1},z_{a_2})=\phi(z_{a_1})+\phi(z_{a_2})=\alpha$

Now I have that $$(1)f_{z_{a_1}}=\lambda g_{z_{a_1}} \Leftrightarrow 1=\lambda\frac{dg}{dz_{a_1}}\phi(z_{a_1})$$ $$(2)f_{z_{a_2}}=\lambda g_{z_{a_2}} \Leftrightarrow 1=\lambda\frac{dg}{dz_{a_2}}\phi(z_{a_2})$$ $$(3)g(z_{a_1},z_{a_2})=\phi(z_{a_1})+\phi(z_{a_2})=\alpha$$

Making $(1)-(2)$ $$(4)\lambda[\frac{dg}{dz_{a_1}}\phi(z_{a_1})-\frac{dg}{dz_{a_2}}\phi(z_{a_2})]=0$$

$(4)$ is valid if $\lambda=1$ and $\frac{dg}{dz_{a_1}}\phi(z_{a_1})=\frac{dg}{dz_{a_2}}\phi(z_{a_2})$, from this I have that

$$\phi(z_{a_1})+\phi(z_{a_2})=\phi(z_{a_1})+\phi(z_{a_1})=a_1+a_1=2a_1=\alpha\Rightarrow a_1=\frac{\alpha}{2}$$ and similary $a_2=\frac{\alpha}{2}$

• I don't understand your question here. Based on CLT, $\bar{X}$ follows a normal distribution asymptotically, which is a symmetric distribution. Then give the confidence level, you will be easily figure out the IC, and $q_1$ will be equal to $q_2$, which leads to lowest range. – Aaron Zeng Jun 3 '15 at 3:39
• @AaronZeng No reference to CLT or asymptotics is needed here. The $X_i$ are given to be independent normal random variables and so $\bar{X}$ is known to be $N(\mu,\sigma^2/n)$ for all positive integers $n$ right there in Chapter 6, long before one gets to Chapter 8 and the CLT and asymptotics. – Dilip Sarwate Jun 4 '15 at 21:02
• You're right. I wasn't paying attention on the iid normal random variables. Thanks for pointing out that. – Aaron Zeng Jun 4 '15 at 21:46
• @DilipSarwate you can give a hand? – user72621 Jun 4 '15 at 23:23

Let me elaborate somewhat over the comment of @AaronZeng:

Let $f$ be a unimodal density function of a continuous r.v. (the latter for simplicity). If the interval $[a,b]$ satisfies

1. $\int_a^bf(x)dx=1-\alpha$
2. $f(a)=f(b)>0$
3. $a\leqslant x^* \leqslant b$, where $x^*$ is the mode of $f(x)$,

then $[a,b]$ is the shortest interval satisfying 1.

Proof: For some fixed $c$, Leibniz' Theorem gives that $$\frac{d}{da}\int_a^{a+c}f(x)dx=f(a+c)-f(a)$$ Setting this derivative to zero gives the first order condition $f(a+c)=f(a)$. This is a maximum, because unimodality and continuity imply that $a\leqslant x^*\leqslant a+c$ and continuity implies that for any $a'>(<)a$, $\int_a^{a'}f(x)dx>(<)\int_{a+c}^{a'+c}f(x)dx$, such that $$\int_a^{a+c}f(x)dx>\int_{a'}^{a'+c}f(x)dx\qquad\text{for any }a'$$ Now, choose $a^*$ and $c^*$ such that $\int_{a^*}^{a^*+c^*}f(x)dx=1-\alpha$ and $f(a^*+c^*)=f(a^*)$. By the previous results, that maximizes the area under the integral, to the desired area $1-\alpha$. Hence $\int_{a^*+\epsilon}^{a^*+c^*+\epsilon}f(x)dx<1-\alpha$, so that we would need to widen the interval to some $[a^*+\epsilon,a^*+c^*+\epsilon+\delta]$ to maintain a coverage probability of $1-\alpha$.

Here is a figure trying to illustrate: First, we have a 90% equitailed confidence interval between the 5%-quantile $-z_{\alpha/2}$ and 95%-quantile $z_{\alpha/2}$. Second, the interval between the 9%- and 99%-quantile is also a 90%-c.i., but we can see that it is longer: we must move the right endpoint by more to the right than the distance between the two left endpoints to ensure the same area under the density, as the density increases to the right of $-z_{\alpha/2}$, but decreases to the right of $z_{\alpha/2}$.

• You know how to find the constants that minimize the confidence interval length using Lagrange multipliers? – user72621 Jun 4 '15 at 20:30
• Never looked at it from that angle yet, no. – Christoph Hanck Jun 6 '15 at 8:58

I avoid discussing "z-scores" and "$p$-values" because I have very little understanding of what these things, so dear to the statistical heart, actually mean.

You have $n$ independent identically distributed normal random variables $X_i, 1 \leq i \leq n$ with unknown mean $\mu$ but known variance $\sigma^2$. Their average $$\bar{X} = \frac 1n \sum_{i=1}^n X_i$$ is also a normal random variable with $\mu$ but smaller variance $\sigma^2/n$.

Now, for a $N(\mu,\tau^2)$ random variable $Y$, $$P\{\mu-1.96 \tau \leq Y \leq \mu+1.96\tau\} = 0.95\tag{1}$$ If we knew the value of $\mu$, we could construct the interval $\mathcal I = [\mu-1.96 \tau, \leq \mu+1.96\tau]$ and be assured that when we do the experiment over and over, on $95\%$ of such trials, the value of $Y$ will lie in $\mathcal I$. We don't know $\mu$, however. But we have observed that $Y$ has taken on value $y$ in the trial just concluded. Now, if $y \in \mathcal I$, then it must be that $\mu \in [y-1.96 \tau, y + 1.96 \tau]$. We don't know whether or not the specific $y$ that we observed lies in $\mathcal I$, but we might feel some confidence, perhaps 95% confidence, that $y$ is one of the "good guys" that belongs to belongs to $\mathcal I$, and so we are 95% confident that $\mu \in [y-1.96 \tau, y + 1.96 \tau]$.

The midpoint of the interval $\mathcal I$ defined by $(1)$ is $\mu$. Now, there are uncountably many real numbers $a$ such that $$P\{a \leq Y \leq b(a)\} = 0.95\tag{2}$$ (Here $b(a)$ is a number, obviously dependent on $a$ such that the equality holds in $(2)$. The function $b(a)$ has value approximately $\mu+1.65\tau$ when $a = -\infty$. It increases slowly and has value $\mu+1.96\tau$ when $a=\mu-1.96\tau$. It then increases very rapidly and approaches $\infty$ as $a$ gets close to $\mu-1.65\tau$. There are no solutions to $(2)$ if $a > \mu-1.65\tau$. So, when does the length $b(a)-a$ of the interval have a minimum? Symmetry suggests that the minimum occurs when $a$ and $b(a)$ are equally far from the mean $\mu$, and we can do a formal proof via calculus as in Christoph Hanck's answer.

I have not thought about applying Lagrange multipliers to the problem.