Given $\{\vec{x}_i\}$ i.i.d $N(\vec{\mu}, \Sigma)$ find confidence ellipse for $\mu$ (unknown $\Sigma$)

Searching online, I was not able to find construction of a confidence ellipse for $\mu$ in this case. Any help would be appreciated. Below is my attempt to construct $1- \alpha$ confidence ellipse.

(For simplicity suppose dim$(\vec{x})=2$) I try to use the fact that the statistics:

$$T := (\overline{x} - \vec{\mu})^T \hat{\Sigma}^{-1} (\overline{x} - \vec{\mu})\,.$$

is distributed $T \sim \text{Hotteling's } T^2$ and draw ellipses

$$(\bar{x} - \vec{r})^T\hat{\Sigma}^{-1}(\bar{x} - \vec{r}) = T^2_{\alpha}\, \quad\left(\text{with }T^2_{\alpha} := F_{T_{2, n-1}}^{-1}(1- \alpha)\right)\,,\tag{1}$$

(with the usual estimates $\bar{x} := \frac{1}{n}\sum_{i = 1}^n \vec{x}$ and $\hat{\Sigma} := \frac{1}{n-1}\sum_{i = 1}^n \left(\bar{x} - \vec{x}_i \right) \left(\bar{x} - \vec{x}_i \right)^T$)

But unlike the case where $\Sigma$ is known this seems not to work. So in the case where $\Sigma$ is know, then
$$\chi:= (\overline{x} - \vec{\mu})^T {\Sigma}^{-1} (\overline{x} - \vec{\mu}) ~~\sim~~ \chi_2^2\,.$$

Take $\chi_{\alpha} = F_{\chi_2^2}^{-1}(1- \alpha)$, and draw the following random ellipses in $\vec{r}$ centered on $\overline{x}$.

$$(\bar{x} - \vec{r})^T \Sigma^{-1}(\bar{x} - \vec{r}) = \chi_{\alpha}\,.\tag{2}$$

those random ellipses will cover $\vec{\mu}$, exactly $1 -\alpha$ percent of the time. This can be verified by the following observation: every time the realization of $\overline{x}$ happens to be inside the ellipse $(\vec{r} - \mu)^T\Sigma^{-1}(\vec{r} - \mu)$ (same ellipse but centered on unknown $\mu$) the ellipse (2) will covers $\mu$, now how often does this happen? With probability of $1 - \alpha$ therefor it yields the $1 - \alpha$ confidence procedure I was looking for.

Back to unknown $\Sigma$, then the ellipses (1), do not have constant principal axis, as those depend on random $\hat{\Sigma}$ and therefor it is not clear to me whether drawing (1) will actually cover the "center", $\mu$, $1 -\alpha$ percent of the time.

• The 3D case is covered in detail at mathematica.stackexchange.com/questions/21396. What you mean by "do not have constant axis" is mysterious. Consider the 1D case: the confidence ellipse is the usual confidence interval. Of course its endpoints vary because they depend on the random data--but so what? – whuber Dec 15 '16 at 0:36
• @whuber thanks! I will think about your comment, I will develop the 1d case with unknown variance... meanwhile I just can not convince myself that the random ellipses cover the center 1-$\alpha$% of the time when the matrix $\hat{\Sigma}$ ("axis") is random, is there an easy way to see this? – them Dec 15 '16 at 0:39