Covariance matrix in Gaussian Discriminant Analysis

Question

I am at a point where I stumbled upon something I don't fully understand.

I have to compute the covariance matrix using this formula:

$$\Sigma=\frac{1}{m}\sum_{i=1}^{m}(x^{(i)}-\mu_{y^{(i)}})(x^{(i)}-\mu_{y^{(i)}})^{T}$$

with $$\mu_0=\frac{\sum_{i=1}^{m}1\{y^{(i)}=0\}x^{(i)}}{\sum_{i=1}^{m}1\{y^{(i)}=0\}}$$ and $$\mu_1=\frac{\sum_{i=1}^{m}1\{y^{(i)}=1\}x^{(i)}}{\sum_{i=1}^{m}1\{y^{(i)}=1\}}.$$

My question, $\Sigma$ is a matrix right? $\mu_0$ and $\mu_1$ are also matrices. And here comes what I don't get. $x^{(i)}$ is a training example, let's say with 2 features, say width and height. $\Sigma$ should be a square matrix.

Answer 1

Yes, $\sum$ is definitely a square matrix.

$x^{(i)}$ is indeed the ith data point, and if it has 2 features in it we can consider it a $2 \times 1$ vector. Likewise, if we look at the formulas for $\mu$, we can see that $\sum 1\{y^{(i)} = 0\}x^{(i)}$ is the sum of a bunch of $2 \times 1$ vectors, so the $\mu$'s are also $2 \times 1$ vectors. Now, $\sum$ is the sum of the product of a $2 \times 1$ vector and a $1 \times 2$ vector, so $\sum$ is $2 \times 2$