I have $N$ datapoints that have $d$ features in a GP and their covariance matrix $K$ and I want to calculate the differential entropy of that GP.

Is this formula right?

$E(I)= \frac{1}{2} \log((2πe)^d \det(K))$

Moreover, given $L$ the cholesky decomposition of $K$ is there a faster way?

Thank you


Yes. You can see the formula here. At least, I am assuming that you have performed the "kernel trick" and are doing the analysis in feature space. K would have $d$ dimensions.

There is a good discussion of the best way to calculate the determinant here.

My answer would be that if you have to ask that question, you need to use one of the well established linear algebra code libraries - say Eigen or Armadillo.

  • $\begingroup$ Thank you! then the $2πe$ is powered in the number of datapoints and not the number of features? Finally do you take the absolute value of the determinant? $\endgroup$ – Yannis Assael Jun 22 '14 at 13:29
  • 2
    $\begingroup$ If the determinant is negative then $K$ cannot be a covariance matrix, so no absolute value is needed or even recommended. $\endgroup$ – whuber Jun 22 '14 at 15:04
  • 1
    $\begingroup$ The exponent is powered by the dimension of K. If K is the dxd covariance matrix of the feature space, then d is the exponent you need. $\endgroup$ – Placidia Jun 22 '14 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.