0
$\begingroup$

I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to duplicated rows in the kernel matrix which will result in both non-invertible kernel matrix and an infinite complexity term. I end up with an infinite log marginal likelihood which I guess due to the problem I explained above. Are there any ideas that could be used to avoid such a behavior?

$\endgroup$
3
$\begingroup$

Your covariance function apparently does not allow for noise. Maybe your problem does not even exhibit noise, which would mean that the same two inputs would always yield the exact same outputs. In any case you may want to add e.g. a (possibly tiny) constant noise term to the diagonal of the covariance matrix for regularization.

$\endgroup$
1
  • $\begingroup$ This is the way that most libraries will deal with it by calling a function called jitchol (in gpy /pygps etc) however in that case you are actually decreasing the certainty of both data points- this is handy in order to prevent extensive searching through the covariance matrix but if you can I think it is better to delete duplicate points before training. $\endgroup$ – j__ Apr 18 '15 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.