2
$\begingroup$

Here on page 7, example 2.7. The claim is that sufficient statistics for $d$ dimensional multivariate normal $\mathbf{x}_i \sim N(\vec{\mu}, \Sigma)$ is $$\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i, \hat{\Sigma} \right)\,,$$ where $\hat{\Sigma} := \sum_{i=1}^n\mathbf{x}_i\mathbf{x}^T_i - \left(n^{-1}\sum_{i=1}^n \mathbf{x}_i\right)\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i\right)^T$.

Where can I find derivation of this result? Thanks!

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ If you consider what "sufficient" means, your definition would likely lead directly to a simple method of deriving this result. Which definition do you have in mind? $\endgroup$ – whuber♦ Aug 25 '16 at 16:39
  • $\begingroup$ @whuber, Sufficiency in the sense that when conditioned on the statistic the distribution is independent of the parameters. Here I probably want to use the factorization theorem for the joint probability density of $n$ samples. That is to construct statistics $T(\mathbf{x}_1, \ldots, \mathbf{x}_n)$ so that $f(\mathbf{x}_1, \ldots, \mathbf{x}_n; \vec{\mu}, \Sigma) = g(T(\mathbf{x}_1, \ldots, \mathbf{x}_n);\vec{\mu}, \Sigma) \cdot h(\mathbf{x}_1, \ldots, \mathbf{x}_n)$. I wasn't able to factorize this in the desired way, would appreciate if you could elaborate how I can do this. Thanks. $\endgroup$ – them Aug 25 '16 at 16:59
  • $\begingroup$ As a side, It would be nice to know if the statistic is also minimal sufficient. $\endgroup$ – them Aug 25 '16 at 17:18
  • 1
    $\begingroup$ (1) You don't have to construct statistics: you are given two of them and only need to show the joint probability can be expressed in terms of them. (2) If these two are not minimal sufficient, then a single statistic would be minimal and sufficient. It's easy to show that's not the case when $d=1$; the demonstration applies immediately to higher $d$. $\endgroup$ – whuber♦ Aug 25 '16 at 17:45
  • 3
    $\begingroup$ Work it out for the case $d=1$ first so you can see what the algebra ought to look like without getting bogged down with matrices and vectors. $\endgroup$ – whuber♦ Aug 25 '16 at 17:58
8
$\begingroup$

As W. Huber tried to lead you to conclude, the sufficiency is a simple consequence of looking at the likelihood: \begin{align*} f(\mathbf{x}_1,\ldots,\mathbf{x}_n|\boldsymbol{\mu},\boldsymbol{\Sigma}) &\propto |\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\boldsymbol{\mu})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\right\}\\ &\quad \times\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{\sum_{i=1}^n \boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})(\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{ \boldsymbol{\Sigma}^{-1}\widehat{\boldsymbol{\Sigma}}\right\}\\ \end{align*} This likelihood thus only depends on two functions of the sample, $\bar{\mathbf{x}}$ and $\widehat{\boldsymbol{\Sigma}}$. $\qquad{ }$Q.E.D.

Note: the author of the attached document should be identified. The more when he is the stellar statistician Jon Wellner!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.