Using GPML in Matlab for MultiClass Classification

Question

I am using Rasmussen's GPML code in Matlab R2011a_student. I have training data (2560x29707) w/ labels (6 classes), and test data (640x29707). To prep the data I have 1. converted from sparse to full, 2. binarized the classes (ie. All classes that = 1 are 1, everything else is -1). I planned on running this 6 times to accommodate all the classes.

I ran the following code (taken right from the documentation, but replaced the x,y,t values with my data): meanfunc = @meanConst; hyp.mean = 0; covfunc = @covSEard; ell = 1.0; sf = 1.0; hyp.cov = log([ell ell sf]); likfunc = @likErf;

hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, x, y); [a b c d lp] = gp(hyp, @infEP, meanfunc, covfunc, likfunc, x, y, t, ones(n, 1));

---

I get the following error, and I'm not sure what it means. Any help would be greatly appreciated. ??? Error using ==> gp at 76 Number of cov function hyperparameters disagree with cov function

Error in ==> minimize at 75 [f0 df0] = feval(f, X, varargin{:}); % get function value and gradient

Error in ==> ds1pleasework at 7 hyp = minimize(hyp, @gp, -40, @infEP, meanfunc, covfunc, likfunc, full_TrainSet_feature, L_train);

Thanks so much.

Answer 1

The problem is that you are not supplying the right number of initial hyper-parameter values for the covariance function you are using. In the example the data set has two attributes, so in that case the ARD covariance function needs three hyper-parameters (two scale parameters, one for each attribute and an overall scale factor for the covariance). For this covariance function (covSEard), you need one more hyper-parameter than the number of attributes.

I would suggest changing covfunc to @covSEiso and hyp.cov to log([ell sf])

For a dataset with as many attributes as yours I would forget trying to use an ARD covariance, there are just so many hyper-parameters that you will just end up over-fitting the marginal likelihood in model selection and end up with a very poor model.

I just wanted to add, GPML is a great piece of kit, I'd strongly recommend it to anyone interested in non-linear regression or machine learning approaches to pattern recognition. Rasmussen and Williams book is similarly excellent.