# How to Select Proper Functions and Hyperparameters for GPML in MATLAB

First time Stack Exchange user, so please correct me if I am not following any guidelines. I am having trouble tuning my Gaussian process regression in MATLAB. I use the GPML code, which I just upgraded to v4.0. I have spent weeks of trial an error testing various mean, covariance, and likelihood functions, but my regression is not a smooth curve as shown in the picture.

The curve I would expect would be more like this that I generated with the MATLAB Curve Fitting toolbox:

How should I tune my GPML parameters to produce a smoother surface? I have tried increasing the length scale. Also, updating to version 4.0 has caused an error, "Inference method failed. Matrix must be positive definite." Here is my code:

%% Load Data
x(:,1) = XFOIL_Data(:,1);
x(:,2) = XFOIL_Data(:,3);
y = XFOIL_Data(:,5);
x1 = linspace(.15,0.76);
x2 = linspace(-2.2,13.3);

%% Calculate GP
meanfunc = {'meanZero'};
covfunc = {'covSEiso'}; ell = 1.5; sf = 0.1; hyp.cov = log([ell; sf]);
likfunc = @likGauss; sn = 0; hyp.lik = log(sn);
nlml = gp(hyp, @infExact, meanfunc, covfunc, likfunc, x, y)

[i, j] = meshgrid(x1,x2);
z(:,1) = reshape(i,[],1);
z(:,2) = reshape(j,[],1);

[mean, var] = gp(hyp, @infExact, meanfunc, covfunc, likfunc, x, y, z);

m = reshape(mean,length(x2),length(x1));
CI_Upper = reshape(mean+2*sqrt(var),length(x2),length(x1));
CI_Lower = reshape(mean-2*sqrt(var),length(x2),length(x1));

%% Restrict Plotting Values
m(19.545*i+j-18.886>0) = NaN;
CI_Upper(19.545*i+j-18.886>0) = NaN;
CI_Lower(19.545*i+j-18.886>0) = NaN;

%% Plotting
figure
hold on
p1 = scatter3(x(:,1),x(:,2),y,100,'k','.');
p2 = surf(x1,x2,m);
%p3=mesh(x1,x2,CI_Upper,'FaceAlpha',0,'EdgeColor','k','LineWidth',.001);
% p4 = mesh(x1,x2,CI_Lower,'FaceAlpha',0,'EdgeColor','k','LineWidth',.001);

• It would help to briefly explain in words what you are doing for non-MATLAB users. – Vihari Piratla Nov 17 '16 at 6:33
• In Gaussian Process it can be done automatically by marginal likelihood maximization. Look at the GPML example in doc/demoRegression.m. This is the corresponding line: hyp2 = minimize(hyp2, @gp, -100, @infExact, [], covfunc, likfunc, x, y); It finds hyp2 by minimizing negative log marginal likelihood. – Seeda Nov 17 '16 at 13:44
• @VihariPiratla My apologies, Stack Exchange did not notify me of comments. I have 60 data points of the form (X1, X2, Y). I am trying to use a Gaussian process regression to yield a mean and variance at all points in the design space. The issue is that the regression is not smooth at all, and I'm struggling to tune it. – Dillon Thomison Nov 22 '16 at 1:26
• @Seeda Thank you for your suggestion. Unfortunately I have already tried using the optimization feature. It does do a good job of minimizing the negative log marginal likelihood. However, the plot is very 'bumpy' like the one that I posted. Is there any 'smoothness' measurement that can be optimized for as well? – Dillon Thomison Nov 22 '16 at 1:29
• I would like to add that I am desperate grad student with less than 2 weeks to publish a conference paper. I would be infinitely grateful to anybody that can help me. – Dillon Thomison Nov 22 '16 at 1:30