# Using GPML in Matlab for MultiClass Classification

I am using Rasmussen's GPML code in Matlab R2011a_student. I have training data (2560x29707) w/ labels (6 classes), and test data (640x29707). To prep the data I have

1. converted from sparse to full,
2. binarized the classes (ie. All classes that = 1 are 1, everything else is -1).

I planned on running this 6 times to accommodate all the classes.

I ran the following code (taken right from the documentation, but replaced the x,y,t values with my data):

meanfunc = @meanConst; hyp.mean = 0;
covfunc = @covSEard; ell = 1.0; sf = 1.0; hyp.cov = log([ell ell sf]);
likfunc = @likErf;

hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, x, y);
[a b c d lp] = gp(hyp, @infEP, meanfunc, covfunc, likfunc, x, y, t, ones(n, 1));


I get the following error, and I'm not sure what it means. Any help would be greatly appreciated.

??? Error using ==> gp at 76
Number of cov function hyperparameters disagree with cov function

Error in ==> minimize at 75
[f0 df0] = feval(f, X, varargin{:});          % get function value and gradient

Error in ==> ds1pleasework at 7
hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, full_TrainSet_feature, L_train);


Thanks so much.

-

## 1 Answer

The problem is that you are not supplying the right number of initial hyper-parameter values for the covariance function you are using. In the example the data set has two attributes, so in that case the ARD covariance function needs three hyper-parameters (two scale parameters, one for each attribute and an overall scale factor for the covariance). For this covariance function (covSEard), you need one more hyper-parameter than the number of attributes.

I would suggest changing covfunc to @covSEiso and hyp.cov to log([ell sf])

For a dataset with as many attributes as yours I would forget trying to use an ARD covariance, there are just so many hyper-parameters that you will just end up over-fitting the marginal likelihood in model selection and end up with a very poor model.

I just wanted to add, GPML is a great piece of kit, I'd strongly recommend it to anyone interested in non-linear regression or machine learning approaches to pattern recognition. Rasmussen and Williams book is similarly excellent.

-
Incredible, worked like a charm though Matlab seems to struggle with the volume of data. Really appreciate the help. –  BryanH Apr 18 '12 at 14:48
no problem, datasets of that size will take a while, but hopefully the results will be worthwhile! –  Dikran Marsupial Apr 18 '12 at 14:54
Actually, Matlab ran for a bit, then threw a major error. Looking at scaling the features down now with relieff. –  BryanH Apr 18 '12 at 17:22