# Computing probability density function at a point, given the covariance matrix and mean

(Edited for clarity.)

Say I have the variance-covariance matrix $$\mathbf{V}$$ and mean $$\mathbf{\mu}$$ of a multivariate normal distribution. Given a sample, $$\mathbf{s}$$, can I compute/estimate the value of the probability density function at that sample?

I'm trying to determine how well a given sample "fits" the various distributions determined by some $$\mathbf{V}_i$$ and $$\mathbf{\mu}_i$$ in order to label the sample with $$\underset{i \in 0, 1, ...}{\arg\max\,}{f(\mathbf{s}; \mathbf{V}_i, \mathbf{\mu}_i)}$$.

• For anyone else who comes with the same question: yes, there is an analytic form for the PDF of the multivariate normal, and it's fairly convenient; see, for example, the scipy docs here Jul 12, 2019 at 19:18

The mean vector and covariance are not sufficient to give the entire probability distribution, so this is a case where you have not prescribed enough information to yield a probability density/mass function, or a likelihood function. If you were to prescribe a class of distributions parameterised by the mean and covariance matrix (e.g., the normal distribution) then that would be sufficient to give you a likelihood function. For example, if you use the normal distribution, the log-likelihood function is:

$$\ell_\mathbf{s}(\mathbf{V}) = - \frac{1}{2} \text{det}(\mathbf{V}) -\frac{1}{2} \mathbf{s}^\text{T} \mathbf{V}^{-1} \mathbf{s} \quad \quad \quad \text{for all } \mathbf{V} \in \mathcal{V},$$

where $$\mathcal{V}$$ is the set of real square positive-definite matrices of the required dimension.

• In practice, I'm using the maximum-likelihood empirical covariance estimator to get $\mathrm{V}$ in the first place, so making that same assumption (multivariate Gaussian) here is no worse than making it in the first place, right? Jul 11, 2019 at 0:28
• You can't be using any kind of "maximum-likelihood" estimator unless you have a likelihood function -- what would you be maximising? Thus, if you are using a procedure like this, there must be some specification (possibly implicit) of what the likelihood function actually is. Most likely the procedure you are using is (implicitly) assuming a Gaussian distribution, but you should check this.
– Ben
Jul 11, 2019 at 0:31
• Maybe I'm not saying what I think I'm saying. I have some set of observations from which I'm computing the covariance matrix (from sklearn). This computes the covariance assuming a multivariate normal distribution, I think. Jul 11, 2019 at 0:34
• Further, I think I'm looking for a $\ell_\mathbf{V}(\mathbf{s})$, for lack of a better way to put it. Does that have the same form? The likelihood of the sample $\mathbf{s}$ given the mean, covariance matrix, and the assumption of multivariate normal distribution. Jul 11, 2019 at 0:38
• Okay, well in that case you can use the formula above. (Note that I have omitted the constant term $-\tfrac{n}{2} \cdot \ln (2 \pi)$ from the function, where $k$ is the length of the vector $\mathbf{s}$. The computer program you are using would probably include this term.)
– Ben
Jul 11, 2019 at 1:01