2
$\begingroup$

I am having trouble comprehending the log-likelihood of a multivariate normal distribution.

For an n-dimensional vector $\mathbf{r}$ of N i.i.d. data points $\mathbf{r}=(r_1,...,r_N)$, the log-likelihood of the Gaussian pdf should be

$$ -2 \ln L = \mathbf{r}^{T} C^{-1} \mathbf{r} + \ln\det C + N\ln(2\pi) $$

where $C$ is an $N\times N$ dimensional covariance matrix which includes the model parameters.

The likelihood function describes the probability density of the data (i.e. the observations) given the parameters (i.e. the model).

Question: what is the dimension of this expression for the log-likelihood?

$N\ln(2\pi)$ is a constant, i.e. it is just a number.

$\ln\det C$ also sums to a single number

The vector $\mathbf{r}$ should be a vector of the dimension (rows, columns) = (N, 1), and the tranpose should be a vector of the dimension (1,N).

If I multiply $\mathbf{r}^{T} C^{-1} \mathbf{r}$, I get (1,N)(N,N)(N,1) = a single number.

So, it seems to me that this function is a constant....but it's a function. A function of fixed data to the parameters.

What is my mistake in comprehension?

$\endgroup$

2 Answers 2

1
$\begingroup$

Might be easier to interpret if you think of $r$ as $$ r = (x - \mu) $$ where mu is a fixed vector and $x$ is a vector for which you are asking the question, "how likely is $x$, given a fixed $\mu$ (and $C$)?"

For example, if you label some pixels (vectors in 3D) in a picture as being skin or not-skin, then you can construct 2 multivariate normals for each label. Each of these distributions will have their own $\mu$ (3D) and covariance (3x3). Then, you can evaluate unlabeled pixels under each state. When combined with prior information about skin/non-skin, you can use a Bayesian approach to determine the posterior probability that an unlabeled pixel is skin.

$\endgroup$
0
$\begingroup$

The log likelihood is a function of $C$, which is the parameter that you're estimating.

$\endgroup$
2
  • $\begingroup$ What confuses me is the dimensions involved. We plot the likelihood function (which is a function of covariance matrix C), against a subset of parameter values. The data vector could be size N. $\endgroup$ Jun 19, 2015 at 3:36
  • $\begingroup$ My question that for an array of data points N, C could be an N by N matrix. That is N x N number of matrix entries, i.e. N^2 number of parameters. How could I possible estimate that? $\endgroup$ Jun 19, 2015 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.