Suppose $X_1,\cdots, X_n$ are sampled iid from a multivariate Gaussian $\mathcal N(\mu, \Sigma)$. We denote the sample mean and covariance as follows \begin{align} \bar X &= \frac{1}{n} \sum_{i=1}^n X_i\\ S &= \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar X)(X_i - \bar X)^\mathrm{T }\end{align} One can prove the following theorem~(see Theorem 3.3.2 and Corollary 7.2.3 of [1]). Here $\mathcal W(V, n)$ is a Wishart distribution with $n$ degrees of freedom and scale matrix $V$.

Theorem 1. $\bar X$ and $S$ are independent and \begin{align} \bar X &\sim \mathcal N(\mu, \frac{\Sigma}{n}), \\ S &\sim \mathcal W(\frac{1}{n-1}\Sigma, n-1). \end{align}

Now we change a problem setting. Suppose $Y_i$ is sampled independently from a multivariate Gaussian $\mathcal N(\mu, \Sigma + T_i)$, $T_i$ PSD, for any $i = 1,\cdots, n$. We denote two estimators as follows \begin{align} \bar Y &= \frac{1}{n} \sum_{i=1}^n Y_i\\ Z &= \frac{1}{n-1} \sum_{i=1}^n (Y_i - \bar Y)(Y_i - \bar Y)^\mathrm{T } \end{align} Does the following theorem hold?

Theorem 2. $\bar Y$ and $Z$ are independent and \begin{align} \bar Y &\sim \mathcal N(\mu, \frac{1}{n}(\Sigma + \frac{1}{n}\sum_{i=1}^n T_i), \\ Z &\sim \mathcal W(\frac{1}{n-1}(\Sigma + \frac{1}{n} \sum_{i=1}^n T_i), n-1). \end{align}

If not, what is the distribution of $Z$? I know the covariance between $\bar Y$ and $Z$ is 0 [2]. But are they independent?

From Anderson's textbook [1], it seems like one strategy of the proof is to construct $Z = \sum_{i=1}^{n-1} A_iA_i^{\mathrm{T}}.$ But can we apply the same construction as section 3.2, page 63-65 of [1] (1984 version)?

[1] Theodore Wilbur Anderson. An Introduction to Multivariate Statistical Analysis. Wiley New York, 1958.

[2] Mukhopadhyay, Nitis, and Mun S. Son. On the covariance between the sample mean and variance. Communications in Statistics—Theory and Methods 40.7 (2011): 1142-1148.

  • $\begingroup$ It is IDD because independently differently distributed. I don't know why some people decided to change it to IID. There are partial solutions at math.stackexchange.com/questions/2329954/…. But it is still unclear if there is a distribution describing Z in an elegant way. In particular, will there be concentration bounds for Z? $\endgroup$ – Zi Wang May 17 '18 at 21:32
  • $\begingroup$ No doubt that was done because "IDD" isn't a widely used/ standard abbreviation and you mention "iid" in the body text right near the start; my own reaction was to guess you had probably made a typo. I suggest you simply remove "IDD" from your title $\endgroup$ – Glen_b May 17 '18 at 23:48
  • $\begingroup$ I think IDD is pretty obvious.. It was used in many papers in statistics. $\endgroup$ – Zi Wang Jul 26 '18 at 17:09
  • $\begingroup$ It was certainly not obvious to me; my training is in statistics. If it's used in many papers in statistics, could you please point to a couple of papers in reasonably well-known statistics journals that use it? (My primary interest being -- when they use it, do they consider it obvious enough to use it without defining it?) $\endgroup$ – Glen_b Jul 26 '18 at 23:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.