2
$\begingroup$

(Edited for clarity.)

Say I have the variance-covariance matrix $\mathbf{V}$ and mean $\mathbf{\mu}$ of a multivariate normal distribution. Given a sample, $\mathbf{s}$, can I compute/estimate the value of the probability density function at that sample?

I'm trying to determine how well a given sample "fits" the various distributions determined by some $\mathbf{V}_i$ and $\mathbf{\mu}_i$ in order to label the sample with $\underset{i \in 0, 1, ...}{\arg\max\,}{f(\mathbf{s}; \mathbf{V}_i, \mathbf{\mu}_i)}$.

$\endgroup$
  • $\begingroup$ For anyone else who comes with the same question: yes, there is an analytic form for the PDF of the multivariate normal, and it's fairly convenient; see, for example, the scipy docs here $\endgroup$ – Linuxios Jul 12 at 19:18
2
$\begingroup$

The mean vector and covariance are not sufficient to give the entire probability distribution, so this is a case where you have not prescribed enough information to yield a probability density/mass function, or a likelihood function. If you were to prescribe a class of distributions parameterised by the mean and covariance matrix (e.g., the normal distribution) then that would be sufficient to give you a likelihood function. For example, if you use the normal distribution, the log-likelihood function is:

$$\ell_\mathbf{s}(\mathbf{V}) = - \frac{1}{2} \text{det}(\mathbf{V}) -\frac{1}{2} \mathbf{s}^\text{T} \mathbf{V}^{-1} \mathbf{s} \quad \quad \quad \text{for all } \mathbf{V} \in \mathcal{V},$$

where $\mathcal{V}$ is the set of real square positive-definite matrices of the required dimension.

$\endgroup$
  • $\begingroup$ In practice, I'm using the maximum-likelihood empirical covariance estimator to get $\mathrm{V}$ in the first place, so making that same assumption (multivariate Gaussian) here is no worse than making it in the first place, right? $\endgroup$ – Linuxios Jul 11 at 0:28
  • 1
    $\begingroup$ You can't be using any kind of "maximum-likelihood" estimator unless you have a likelihood function -- what would you be maximising? Thus, if you are using a procedure like this, there must be some specification (possibly implicit) of what the likelihood function actually is. Most likely the procedure you are using is (implicitly) assuming a Gaussian distribution, but you should check this. $\endgroup$ – Ben Jul 11 at 0:31
  • $\begingroup$ Maybe I'm not saying what I think I'm saying. I have some set of observations from which I'm computing the covariance matrix (from sklearn). This computes the covariance assuming a multivariate normal distribution, I think. $\endgroup$ – Linuxios Jul 11 at 0:34
  • $\begingroup$ Further, I think I'm looking for a $\ell_\mathbf{V}(\mathbf{s})$, for lack of a better way to put it. Does that have the same form? The likelihood of the sample $\mathbf{s}$ given the mean, covariance matrix, and the assumption of multivariate normal distribution. $\endgroup$ – Linuxios Jul 11 at 0:38
  • $\begingroup$ Okay, well in that case you can use the formula above. (Note that I have omitted the constant term $-\tfrac{n}{2} \cdot \ln (2 \pi)$ from the function, where $k$ is the length of the vector $\mathbf{s}$. The computer program you are using would probably include this term.) $\endgroup$ – Ben Jul 11 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.