0
$\begingroup$

I am having a problem with the GPML (Gaussian Processes for Machine Learning) tool in MATLAB. I have distilled my actual problem down to this basic demonstration below. Although the training points are linear, the GPML regression has oscillations that make the estimates significantly inaccurate, especially near the end points. The following plots are with length scales 0.25 and 1.75 respectively. ell = 0.25 ell = 1.75

The code I am using to create these plots is

meanfunc = {@meanConst};  hyp.mean = 2;
covfunc = {'covSEiso'}; ell = 1.75; sf = 0.2; hyp.cov = log([ell; sf]);
likfunc = @likGauss; sn = 0.01; hyp.lik = log(sn);

nlml = gp(hyp, @infExact, meanfunc, covfunc, likfunc, x2, error2);

There is no length scale that will yield a linear prediction. Is there another covariance function or likelihood function that I should be using for linear data?

Note: My dataset is actually more complicated, but it exibits these problems near the end points.

$\endgroup$
2
$\begingroup$

If you are using covSEiso (aka RBF) covariance function, you cannot get rid of this condition. Something like Gibbs and Runge phenomenon is happening for interpolation using RBF kernel. I would try a less smooth kernel.

$\endgroup$
  • $\begingroup$ Thank you for your help. Is there another type of kernel that would be more appropriate than the radial basis function? $\endgroup$ – Dillon Thomison Nov 22 '16 at 18:18
  • $\begingroup$ I was unfamiliar with the Gibbs and Runge phenomena when I posted my previous comment. I have done some research, and I think you are exactly right about that being the cause of my problem. Are you familiar with techniques that can mitigate this issue? $\endgroup$ – Dillon Thomison Nov 22 '16 at 19:10
  • $\begingroup$ GPML has a built-in Matern kernel where smoothness can be controlled via a parameter choice (1, 3 or 5). Try Matern with different paramaters. $\endgroup$ – Seeda Nov 22 '16 at 19:13
  • $\begingroup$ The Matern covariance function with Automatic Relevance Determination improved my results tremendously. The hyperparameters are much different than the other function, but it works. Thank you for all of your help. $\endgroup$ – Dillon Thomison Nov 23 '16 at 22:16
  • $\begingroup$ Which smoothness parameter "d" did you choose? 1, 3 or 5? $\endgroup$ – Seeda Nov 24 '16 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.