# Finding out the generative log-likelihood $\log(p(x|t))$

The question ask to build classifiers to label images of handwritten digits. Each image is $$8$$ by $$8$$ pixels and is represented as a vector of dimension $$64$$ by listing all the pixel values in raster scan order. The images are grayscale and the pixel values are between $$0$$ and $$1$$. The labels $$t$$ are $$0$$, $$1$$, $$2$$, ..., $$9$$ corresponding to which character was written in the image. There are $$700$$ training cases and $$400$$ test cases for each digit.
Using maximum likelihood, fit a set of 10 class-conditional Gaussians with a separate, full covariance matrix for each class. Remember that the conditional multivariate Gaussian probability density is given by, $$p(x |t = k) = (2\pi)^{-D/2}|\Sigma_k|^{-1/2}\text{exp}\bigg\{-\frac{1}{2}(\text{x}-\pmb{\mu}_k)^{\text{T}}\Sigma_k^{-1}(\text{x}-\pmb{\mu}_k)\bigg\}$$ where $$\pmb{\mu}_k \in \mathbb{R}^{1\times D}$$, $$\Sigma_k \in \mathbb{R}^{D\times D}$$ and positive-definite. Assume that $$p(t=k)=\frac{1}{10}$$.

Given that $$D=64$$ and $$N=7000$$, and $$\text{x}\in\mathbb{R}^{N\times D}$$ which $$\text{x}$$ is a $$7000 \times 64$$ matrix.

$$\textbf{(1)}$$ First compute parameter $$\mu_{kj}$$ and $$\Sigma_K$$ for each $$k\in{0,..,9}$$ and $$j\in\{1,...,64\}$$. Note: To ensure numerical stability you may have to add a small multiple of the identity to each covariance matrix. For this question should add 0.01I to each covariance matrix.
After use numpy in python I find out the $$\mu_{kj}$$ which is a $$1 \times 64$$ vector, and $$\Sigma_K$$ is a $$64 \times 64$$ matrix, for each $$k\in{0,..,9}$$ and $$j\in\{1,...,64\}$$.

$$\textbf{(2)}$$ Compute $$\log(p(x|t))$$.
Notice that $$\log(p(x|t)) = -\frac{D}{2}\log(2\pi) - \frac{1}{2}\log(|\Sigma_k^{-1}|) -\frac{1}{2}(\text{x}-\pmb{\mu}_k)^{\text{T}}\Sigma_k^{-1}(\text{x}-\pmb{\mu}_k)$$

But the problem is that $$(\text{x}-\pmb{\mu}_k)^{\text{T}}$$ is a $$64 \times 7000$$ matrix in python
and $$\Sigma_k^{-1}$$ is a $$64 \times 64$$ matrix in python
and $$(\text{x}-\pmb{\mu}_k)$$ is a $$7000 \times 64$$ matrix in python
So, that is impossible to compute
$$(\text{x}-\pmb{\mu}_k)^{\text{T}}\Sigma_k^{-1}(\text{x}-\pmb{\mu}_k)$$ So, I am stuck on computing $$\log(p(x|t))$$
So, my question is that are there any other ways to compute $$\log(p(x|t))$$ ?

However, your set-up is flawed. Let $$\boldsymbol{x}_{ik} \in \mathbb{R}^{64}$$ denote the $$i$$th vectorized 8x8 image for the $$k$$th label. The last term of the full log-likelihood should be $$\sum_{k=0}^9 \sum_{i=1}^{700} \left(\boldsymbol{x}_{ik} - \boldsymbol{\mu}_k\right)^{\prime} \Sigma_k^{-1} \left(\boldsymbol{x}_{ik} - \boldsymbol{\mu}_k\right)$$.
You can group all the terms together as well. For instance, let $$\begin{eqnarray*} \boldsymbol{X}_k = \begin{pmatrix} \boldsymbol{x}_{1,k}^{\prime} \\ \ldots \\ \boldsymbol{x}_{700,k}^{\prime} \end{pmatrix}. \end{eqnarray*}$$ Then we may write the last term of the full log-likelihood as $$\sum_{k=0}^9 \left(\mbox{vec} \left(\boldsymbol{X}_k^{\prime}\right) - \boldsymbol{1}_{700} \otimes \boldsymbol{\mu}_k\right)^{\prime} \Sigma_k^{-1} \left(\mbox{vec} \left(\boldsymbol{X}_k^{\prime}\right) - \boldsymbol{1}_{700} \otimes \boldsymbol{\mu}_k\right)$$, where $$\boldsymbol{1}_{700}$$ is a vector of seven hundred 1s.
Finally, the second term is also incorrect in your full log-likelihood. It should be $$-350\sum_{k=0}^9 \ln \left|\Sigma_k\right|$$