I am using Rasmussen's GPML code in Matlab R2011a_student. I have training data (2560x29707) w/ labels (6 classes), and test data (640x29707). To prep the data I have

  1. converted from sparse to full,
  2. binarized the classes (ie. All classes that = 1 are 1, everything else is -1).

I planned on running this 6 times to accommodate all the classes.

I ran the following code (taken right from the documentation, but replaced the x,y,t values with my data):

meanfunc = @meanConst; hyp.mean = 0;
covfunc = @covSEard; ell = 1.0; sf = 1.0; hyp.cov = log([ell ell sf]);
likfunc = @likErf;

hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, x, y);
[a b c d lp] = gp(hyp, @infEP, meanfunc, covfunc, likfunc, x, y, t, ones(n, 1));

I get the following error, and I'm not sure what it means. Any help would be greatly appreciated.

??? Error using ==> gp at 76
Number of cov function hyperparameters disagree with cov function

Error in ==> minimize at 75
[f0 df0] = feval(f, X, varargin{:});          % get function value and gradient

Error in ==> ds1pleasework at 7
hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, full_TrainSet_feature, L_train);

Thanks so much.


The problem is that you are not supplying the right number of initial hyper-parameter values for the covariance function you are using. In the example the data set has two attributes, so in that case the ARD covariance function needs three hyper-parameters (two scale parameters, one for each attribute and an overall scale factor for the covariance). For this covariance function (covSEard), you need one more hyper-parameter than the number of attributes.

I would suggest changing covfunc to @covSEiso and hyp.cov to log([ell sf])

For a dataset with as many attributes as yours I would forget trying to use an ARD covariance, there are just so many hyper-parameters that you will just end up over-fitting the marginal likelihood in model selection and end up with a very poor model.

I just wanted to add, GPML is a great piece of kit, I'd strongly recommend it to anyone interested in non-linear regression or machine learning approaches to pattern recognition. Rasmussen and Williams book is similarly excellent.

  • $\begingroup$ Incredible, worked like a charm though Matlab seems to struggle with the volume of data. Really appreciate the help. $\endgroup$ – BryanH Apr 18 '12 at 14:48
  • $\begingroup$ no problem, datasets of that size will take a while, but hopefully the results will be worthwhile! $\endgroup$ – Dikran Marsupial Apr 18 '12 at 14:54
  • $\begingroup$ Actually, Matlab ran for a bit, then threw a major error. Looking at scaling the features down now with relieff. $\endgroup$ – BryanH Apr 18 '12 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.