I do not understand this notation for the sample covariance matrix (from Artificial Intelligence: A Modern Approach, Peter Norvig and Stuart J. Russell, Section 20.3, EM algorithm):

$\Sigma_{i} = \frac{\sum_{j}p_{ij}(\mathbf{x}_{j}-\mathbf{\mu}_{i})(\mathbf{x}_{j}-\mathbf{\mu}_{i})^{\top}}{n_{i}}$

As far as I know, matrix dimensions do not match. From what I understand, $\mathbf{x}_{j}$ and $\mathbf{\mu}_{i}$ are row vectors of dimension $1\times d$. How can this yield a $d\times d$ matrix? Isn't $\Sigma_{i}$ the covariance matrix of mixed Gaussian distribution component $i$? But, isn't $(\mathbf{x}_{j}-\mathbf{\mu}_{i})(\mathbf{x}_{j}-\mathbf{\mu}_{i})^{\top}$ a scalar?

I also looked up Wikipedia (https://en.wikipedia.org/wiki/Sample_mean_and_covariance). I understand this notation:


for elements of the sample covariance matrix (Q). But again, not this one:


What am I missing here?

  • 5
    $\begingroup$ What gives you the impression that they must be row vectors? Doesn't everything fit if they are in fact column vectors? $\endgroup$ – Ruben van Bergen Jul 9 '18 at 10:51
  • $\begingroup$ If $a$ is a (column) vector, then $aa^\top$ is definitely a matrix, not a scalar. $\endgroup$ – StubbornAtom Jul 9 '18 at 11:32
  • $\begingroup$ Cross posted on math.se : math.stackexchange.com/q/2844766/321264 $\endgroup$ – StubbornAtom Jul 9 '18 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.