I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to duplicated rows in the kernel matrix which will result in both non-invertible kernel matrix and an infinite complexity term. I end up with an infinite log marginal likelihood which I guess due to the problem I explained above. Are there any ideas that could be used to avoid such a behavior?


1 Answer 1


Your covariance function apparently does not allow for noise. Maybe your problem does not even exhibit noise, which would mean that the same two inputs would always yield the exact same outputs. In any case you may want to add e.g. a (possibly tiny) constant noise term to the diagonal of the covariance matrix for regularization.

  • $\begingroup$ This is the way that most libraries will deal with it by calling a function called jitchol (in gpy /pygps etc) however in that case you are actually decreasing the certainty of both data points- this is handy in order to prevent extensive searching through the covariance matrix but if you can I think it is better to delete duplicate points before training. $\endgroup$
    – j__
    Apr 18, 2015 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.