5
$\begingroup$

Through the last year I have been working with support vector machines for a binary text classification task. Having used software such as R and Rapidminer I have not spent much time on understanding what actually goes on inside support vector machines. This I have now started looking into in the hope of getting a better understanding of this classification/regression method.

I have spent a lot of time looking for calculation examples as it tends to enhance my understanding of a concept quite well if I can actually setup a problem in Excel. Therefore I hope to get guidance by asking this question here, as I have not been able to find any step-by-step calculation examples. One can easily find descriptions of the math and optimization problems one need to understand and solve computationally, but a step-by-step calculation example I have not been able to find.

If the forum approves my idea of producing such an example I will do the editing and in the end produce a nice and clear Excel sheet and a guide for future use.

I suggest that we use the Iris dataset (even though it is a multiclass dataset) and simply try to separate Iris setosa from Iris versicolor.

I provide three links. Link one is theory of application of SVMs which I thought one could use as a scaffold. Link two provides a regression example of how I was thinking our product would look in the end. Link three will take you to the Iris dataset.

Theory and application of SVMs

A guide for regressions

Link for the Iris dataset

Below I will try to formulate the problem more neatly.

Problem description:

How can one apply Excel and the technique of a linear support vector machine with soft margins in order to solve a binomial classification task given by separating Iris setosa and Iris versicolor from the Iris dataset using all available features?

Improve this question
$\endgroup$
3
  • 17
    $\begingroup$ Good god why are you doing this in Excel?? $\endgroup$
    – Stumpy Joe Pete
    Commented Jun 9, 2013 at 5:45
  • 1
    $\begingroup$ I totally get why you would want to do this. I have been using Excel for over 10 years but I have only been using R for about a year and am also just as new to datamining (have only used multiple and logistic regression and decision trees including random forest). For example when I wanted to validate my understanding of ROC and lift and calibration charts I took datasets in Excel and recreated the plots by hands. I suppose of course that relatively to SVM that was easier to do. $\endgroup$
    – daniellopez46
    Commented Dec 3, 2013 at 1:20
  • $\begingroup$ See also this excellent tutorial from QuantMacro $\endgroup$
    – purbani
    Commented Oct 10, 2017 at 14:44

4 Answers 4

Reset to default
5
$\begingroup$

Honestly, I am not sure why you want to do this in Excel. Nonetheless, ...

A linear SVM requires solving a quadratic program with several linear constraints. You can check this answer [1] to find out how the quadratic program is setup. Once you setup the quadratic program and find a solver that can help you solve it in Excel, then you are good to go.

On the other hand, the corresponding quadratic program has a dual that gives rise to the notion of kernels. The objective function for the dual can be found here [2]. If you can find a quadratic program solver in Excel, you might as well solve the dual, which will allow you to solve problems beyond linear kernels.

If you don't have a QP solver at hand, then you can write the SMO algorithm [3] which solves the SVM dual. The provided link gives you a pseudocode. SMO is one of the simplest algorithms to solve the SVM dual, but also the slowest. For a small number of training data, it should be pretty fast, however.

[1] Given a set of points in two dimensional space, how can one design decision function for SVM?

[2] Non-linear SVM classification with RBF kernel

[3] http://cs229.stanford.edu/materials/smo.pdf

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for your answer. I realize it might seem stupid to do this in excel. However I like using excel when I need to understand something in depth... $\endgroup$
    – Kasper Christensen
    Commented Jun 9, 2013 at 19:38
  • $\begingroup$ I am not saying it is stupid, just that the amount of work involved seems a bit too much :-) $\endgroup$
    – TenaliRaman
    Commented Jun 10, 2013 at 5:59
  • $\begingroup$ I agree. I do know how to setup a SVM in both Rapidminer and R but I am not really getting a deeper understanding of how they work, which is my goal at this point :) $\endgroup$
    – Kasper Christensen
    Commented Jun 10, 2013 at 19:03
  • 1
    $\begingroup$ I hope the links above help you. If you have done linear programs and quadratic programs before, SVMs are not very hard to understand at all. The only hard part is solving the quadratic program and most papers on SVM just deal with fast solutions to the SVM quadratic problem. However, you can just go ahead and try out the SMO algorithm. It might be be beneficial for you to read about Lagrange duality and Karush-Kuhn-Tucker conditions. $\endgroup$
    – TenaliRaman
    Commented Jun 10, 2013 at 19:48
  • $\begingroup$ @KasperChristensen while your goal is a good one which I genuinely support, using Excel for this is like frying eggs on a beer cooler. $\endgroup$
    – Marc Claesen
    Commented Jul 20, 2013 at 19:10
2
$\begingroup$

This looks like a good tutorial, and has a downloadable Excel example: http://people.revoledu.com/kardi/tutorial/Regression/KernelRegression/KernelRegression.htm

Improve this answer
$\endgroup$
0
0
$\begingroup$

You might try using Excel2SVM if you want to organize your data in an excel format. http://www.bioinformatics.org/Excel2SVM/ could be helpful

Improve this answer
$\endgroup$
0
$\begingroup$

You can find a tutorial here, it uses Excel (no macros) and explains everything in an intuitive way (beware: most parts are behind a paywall, but the price is reasonable):

http://people.revoledu.com/kardi/tutorial/SVM/index.html

Improve this answer
$\endgroup$
1
  • $\begingroup$ Why the downvote?!? $\endgroup$
    – vonjd
    Commented May 8, 2017 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.