# Sufficient statistic for bivariate or multivariate normal

Here on page 7, example 2.7. The claim is that sufficient statistics for $d$ dimensional multivariate normal $\mathbf{x}_i \sim N(\vec{\mu}, \Sigma)$ is $$\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i, \hat{\Sigma} \right)\,,$$ where $\hat{\Sigma} := \sum_{i=1}^n\mathbf{x}_i\mathbf{x}^T_i - \left(n^{-1}\sum_{i=1}^n \mathbf{x}_i\right)\left(n^{-1}\sum_{i=1}^n \mathbf{x}_i\right)^T$.

Where can I find derivation of this result? Thanks!

• If you consider what "sufficient" means, your definition would likely lead directly to a simple method of deriving this result. Which definition do you have in mind? – whuber Aug 25 '16 at 16:39
• @whuber, Sufficiency in the sense that when conditioned on the statistic the distribution is independent of the parameters. Here I probably want to use the factorization theorem for the joint probability density of $n$ samples. That is to construct statistics $T(\mathbf{x}_1, \ldots, \mathbf{x}_n)$ so that $f(\mathbf{x}_1, \ldots, \mathbf{x}_n; \vec{\mu}, \Sigma) = g(T(\mathbf{x}_1, \ldots, \mathbf{x}_n);\vec{\mu}, \Sigma) \cdot h(\mathbf{x}_1, \ldots, \mathbf{x}_n)$. I wasn't able to factorize this in the desired way, would appreciate if you could elaborate how I can do this. Thanks. – them Aug 25 '16 at 16:59
• As a side, It would be nice to know if the statistic is also minimal sufficient. – them Aug 25 '16 at 17:18
• (1) You don't have to construct statistics: you are given two of them and only need to show the joint probability can be expressed in terms of them. (2) If these two are not minimal sufficient, then a single statistic would be minimal and sufficient. It's easy to show that's not the case when $d=1$; the demonstration applies immediately to higher $d$. – whuber Aug 25 '16 at 17:45
• Work it out for the case $d=1$ first so you can see what the algebra ought to look like without getting bogged down with matrices and vectors. – whuber Aug 25 '16 at 17:58

As W. Huber tried to lead you to conclude, the sufficiency is a simple consequence of looking at the likelihood: \begin{align*} f(\mathbf{x}_1,\ldots,\mathbf{x}_n|\boldsymbol{\mu},\boldsymbol{\Sigma}) &\propto |\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\boldsymbol{\mu})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\right\}\\ &\quad \times\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{\sum_{i=1}^n (\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{\sum_{i=1}^n \boldsymbol{\Sigma}^{-1}(\mathbf{x}_i-\bar{\mathbf{x}})(\mathbf{x}_i-\bar{\mathbf{x}})^\text{T}\right\}\\ &=|\boldsymbol{\Sigma}|^{-n/2}\,\exp\frac{-n}{2}(\bar{\mathbf{x}}-\boldsymbol{\mu})^\text{T}\boldsymbol{\Sigma}^{-1}(\bar{\mathbf{x}}-\boldsymbol{\mu})\\ &\quad \times\exp\frac{-1}{2}\text{tr}\left\{ \boldsymbol{\Sigma}^{-1}\widehat{\boldsymbol{\Sigma}}\right\}\\ \end{align*} This likelihood thus only depends on two functions of the sample, $\bar{\mathbf{x}}$ and $\widehat{\boldsymbol{\Sigma}}$. $\qquad{ }$Q.E.D.