1
$\begingroup$

I have $N$ datapoints that have $d$ features in a GP and their covariance matrix $K$ and I want to calculate the differential entropy of that GP.

Is this formula right?

$E(I)= \frac{1}{2} \log((2πe)^d \det(K))$

Moreover, given $L$ the cholesky decomposition of $K$ is there a faster way?

Thank you

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Yes. You can see the formula here. At least, I am assuming that you have performed the "kernel trick" and are doing the analysis in feature space. K would have $d$ dimensions.

There is a good discussion of the best way to calculate the determinant here.

My answer would be that if you have to ask that question, you need to use one of the well established linear algebra code libraries - say Eigen or Armadillo.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you! then the $2πe$ is powered in the number of datapoints and not the number of features? Finally do you take the absolute value of the determinant? $\endgroup$
    – Yannis Assael
    Commented Jun 22, 2014 at 13:29
  • 2
    $\begingroup$ If the determinant is negative then $K$ cannot be a covariance matrix, so no absolute value is needed or even recommended. $\endgroup$
    – whuber ♦
    Commented Jun 22, 2014 at 15:04
  • 1
    $\begingroup$ The exponent is powered by the dimension of K. If K is the dxd covariance matrix of the feature space, then d is the exponent you need. $\endgroup$
    – Placidia
    Commented Jun 22, 2014 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.