Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

I have been asked by a journal reviewer to provide the my final trained SVM models so that others can try to replicate or make use of the models.

Is there an accepted way to do this?

I was thinking of providing a spreadsheet detailing the selected features, the support vectors, the hyperplane bias and he support vector weights for each model. I am using Matlab's *SVM_train* function for this purpose.

If have done this in the past for regression and regularized dircriminant models however this seems rather unwieldy for SVMs.

share|improve this question
Very weird request, I was never asked to supply something like this. Of course the training data and SVM run parameters should be made available (so it is possible to replicate), and perhaps some version of the trained model. – Bitwise Sep 27 '12 at 15:31
Well I am not allowed to make the data publically available but the run parameters and decription of all the features etc were included in the manucript – BGreene Sep 27 '12 at 15:43
Not allowed by who? If you publish data it is usually a requirement that it be made available. – Bitwise Sep 27 '12 at 15:50
This is not usually the case in medicine. Data are often restricted for ethical or commercial reasons – BGreene Sep 27 '12 at 15:59
1  
Interesting. So there is no way to validate your results? – Bitwise Sep 27 '12 at 16:24
show 2 more comments

2 Answers

up vote 1 down vote accepted

I can't see why you can't just save the model in a .mat file and make that available. It would be far more sensible though for the reviewer to request the code that you used to make the model, that would be much more useful from a reproducibility perspective. A MATLAB function that takes the data as input and deterministically generates the model that you used would be an excellent way of "providing the model" as anyone who is interested can easily try it out with their version of the data.

For my papers I try to integrate the code that generates the results with the LaTeX source of the paper, so that in principle, anyone can reproduce the work by typing "make" and it will re-run the experiment, patch the results into the LaTeX source and recompile the document. Rather a lot of work is involved in this, but I think it leads to better experimental work in the long run.

share|improve this answer
1  
For commercial reasons I am not allowed to share the code, and for ethical reasons I am not allowed to share the data (this may change). Saving to a mat file means only Matlab users can interpret... – BGreene Sep 28 '12 at 11:26
1  
I am actually citing your 2010 JMLR paper based on a relevant answer you gave recently – BGreene Sep 28 '12 at 11:27
Cool, I'm glad you found the paper useful! Octave is free and can read many MATLAB files, so it is still a reasonable approach, and has the benefit of not needing to worry about the numerical representation issues etc. It is tough not being able to share code or data, but being asked to share the model! I think just putting all the component parts in a spreadsheet is probably the best you will be able to do. The current study I am working on used 30,000,000,000+ models, I hope the reviewers don't ask for them! – Dikran Marsupial Sep 28 '12 at 11:41
Thanks, I might try the mat file approach with the the relevant parameters and structrues saved. If that doesn't work I go with the dodgy spreadsheet option. Pper wa helpful, I am doing this a while but the model selection bias thing came as a surprise and really only came to the fore when I changed to an SVM from a RD classifier – BGreene Sep 28 '12 at 11:45
1  
It was a surprise to me as well just how large the bias can be. I only found out about it because I had recently acquired a larger computer and so the nested approach became a feasible option and was wondering why I couldn't get results nearly as good as some of those published using similar models, but with the median protocol. Shows the value of being thorough even when you think cutting the corner won't make a significant difference! – Dikran Marsupial Sep 28 '12 at 11:49
up vote 1 down vote

That is kind of an odd request, but I did something similar when participating with a BioCreative classification task. I basically requested a port facing outside of the University firewall and pointed an XMLRPC server at that port. My classifier lived on the University server, but users were still able to send classification requests to it, once they knew the address. This is actually quite easy to do with Python (SimpleXMLRPCServer), and then you can just publish some code snippets for users to use to connect with your classifier onto GitHub!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.