Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."
1
vote
0answers
7 views
How can invariances be taken advantage of when selecting a kernel?
Some answers to other questions on kernel selection suggest using invariances known about the problem domain, and choosing a kernel that preserves such invariances.
Can you give an explicit example ...
1
vote
0answers
24 views
SVM with quadratic loss
I've seen some statement where I got the impression that SVM with a quadratic loss is no more than having a kernel matrix where a multiple of the unit matrix is subtracted from the kernel. It was ...
1
vote
0answers
19 views
Does it make sense to minimize negative log likelihood on SVC probability outputs
I'd like to run a grid search cross-validation on the probability outputs of the SVC classifier. In particular I'd like to minimize the negative log likelihood. Is this a reasonable thing to do? I'm ...
0
votes
1answer
39 views
How does SVM prediction work?
As far as I understand, the training phase usually uses the dual optimization formulation where we can implicitly calculate the weight vector which defines the discriminant function.
How about the ...
1
vote
1answer
22 views
How do I identify which parameters are correlated to binary output?
I'm trying to program an SVM in Python to categorize proteins as "Go" or "No-Go".
I have a list of about 30 proteins, each with ~ 100 columns of structure-related parameters and 1 column of "True" or ...
1
vote
2answers
39 views
How to interpret the model parameters of libsvm via MATLAB interface?
I used the MATLAB interface of libsvm for doing binary classification of 997-dimensional training data. I am trying to understand how the resulting model is used to compute the predicted output (which ...
1
vote
1answer
36 views
Checking autocorrelation in non-parametric methods
I would like to forecast short-term electric load by using Artificial Neural Network and Support Vector Regression. However, there's one question that sticks in my mind. In such forecasting with ...
0
votes
1answer
52 views
Kernel selection intuition
Which problems are best solved using which kernels and why?
Can you give a simple toy problem that isn't linearly separable in input space but is linearly separable in feature space, using an RBF ...
2
votes
2answers
67 views
Why can the margin of SVM be approximated by 1?
The separate function of SVM is :
$wx+b=0$
The function distance of support vector to the separate plane is :
$|r| = wx_i+b$
And we can normalize the $w$, then the distance can be write as :
...
2
votes
0answers
37 views
SVM classifier (with soft-margin) implementation in R, gamma value and quadprog
I'm trying to implement a Support Vector Machine classifier in R and I have to solve the optimization problem using the quadprog R package which solves problems of the form :
$$min_b \frac{1}{2} ...
0
votes
1answer
49 views
regularized logistic regression and support vector machine
L2 regularized logistic regression differs with L2 regularized support vector machine with their loss function. Are there more deep differences for these two models? I tried several data sets, and ...
0
votes
0answers
11 views
nu-based Support Vector Machine (SVM): limit on nu for 1-class SVM?
For a nu-based SVM, there are limits on acceptable nu (soft margin) parameters (p. 126 of Appl. Stochastic Models Bus. Ind., 2005; 21:111–136).
Are there similar limits in the case of a 1-class SVM?
1
vote
1answer
34 views
How to understand effect of RBF SVM
How can I understand what the RBF Kernel in SVM does? I mean I understand the maths, but is there a way to get a feeling when this kernel will be useful?
Would results from kNN be related to SVM/RBF ...
0
votes
1answer
59 views
The Lagrange multipliers of SVM
Actually the solve the SVM is to solve the following Lagrangian Equation:
If we don't use kernel function, $\langle x^{(i)},x^{(j)}\rangle$ is just the vector vector inner product. The ...
1
vote
0answers
39 views
Which Regression methods are suitable for binary valued features and continuous output?
I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
1
vote
1answer
25 views
Bias term in support vector machine
In SVM, there is a bias term. But looks to me there are very few discussions on the physical meanings of this term. Why should we have that? How does this term affect the model?
0
votes
0answers
45 views
How can one setup a linear support vector machine in excel?
Through the last year I have been working with support vector machines for a binary text classification task. Having used software as R and Rapidminer I have not spend much time on understanding what ...
0
votes
0answers
8 views
Are support vectors from the same class always equally geometrically important?
Are support vectors from the same class always given equal importance / Lagrange-multiplier weight?
The paper called "Duality and Geometry in SVM Classifiers" ...
1
vote
1answer
80 views
When does Naive Bayes perform better than SVM?
In a small text classification problem I was looking at, Naive Bayes has been exhibiting a performance similar to or greater than an SVM and I was very confused.
I was wondering what factors decide ...
0
votes
0answers
13 views
Placement of positive and negative support vectors with respect to the origin
In all of the graphics I have seen on the hyperplanes of support vector machines, the positive class (+1s) is away from the origin, while the negative class (-1s) is closer to it. Is this always true?
...
0
votes
0answers
21 views
Pegasos algorithm parameters estimation
For learning pourpose I'm testing my own implementation of the Pegasos algorithm and I'm getting a quite high error rate.
My dataset contains 20k examples and I'm using 17,5k of them for the training ...
1
vote
1answer
30 views
Evaluating features and similarity measures
I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am
designing some features and similarity measures which I might use for a later ...
2
votes
0answers
28 views
Differences between SVR with a linear kernel and linear least squares
I've been working on a toy problem of predicting reviews a product will get in the future. I found that SVR with a linear kernel worked better than doing a linear least squares regression on the data ...
0
votes
0answers
33 views
Predicting with Relevance Vector Machines
I am trying out this Matlab toolbox for Relevance Vector Machines by Tipping: http://www.miketipping.com/sparsebayes.htm
This has an implementation of Relevance Vector Machines, and generates pretty ...
1
vote
3answers
61 views
Order of Support Vectors, and how to reduce them
I am working in an extremely memory constrained environment, and the number of support vectors my Matlab design is generating is just not something that scales. That led me to move to finding a way to ...
2
votes
1answer
55 views
What exactly is the equation for SVM classification for new example?
I understand that in the case of Logistic Regression, we simply multiply our weights with Input example for classification. But what exactly is the equation that we calculate in the case of SVM to ...
1
vote
1answer
49 views
Is this kernel proper?
I think it is proper as it follows the rule that K(x1,x2) = f(x1)f(x2)
Also they are both a function of only one of the points in the kernel respectively. Finally the product of two valid kernels ...
1
vote
0answers
23 views
Weighted SVM and Up-sampling
I'm performing classification with the libSVM package in R and am wondering about the correct procedure for weighting or up-sampling. I have a data set that is 19,396 observations of which only 81 are ...
2
votes
3answers
197 views
Why is svm not so good as decision tree on the same data?
I am new to machine learning and try to use scikit-learn(sklearn) to deal with a classification problem. Both DecisionTree and SVM can train a classifier for this problem.
I use ...
0
votes
0answers
41 views
Image classifier in python for few samples
I have 150 pictures that represent archeological signs and 5 categories to which they belong. These pictures have features like circularity, roughness and elongation that are expressed as continuous ...
1
vote
2answers
61 views
Highly unbalanced test data set and balanced training data in classification
I have a training set with about 3000 positive instances and 3000 negative instances. But my test data set is pretty much un-balanced. The positive set only has 50 instances and negative has 1500 ...
0
votes
1answer
38 views
Linear SVM and Random oversampling
Considering class imbalance, why does random oversampling, in general, improve the performance of a linear SVM?
Is it because the number of support vectors for the minority class are increased as a ...
0
votes
2answers
48 views
In SVM, what are the labels and how do you get them from the data?
I'm working on a school project and have decided to use SVM for stock market prediction. I have a 1000x5 matrix of stock quotes containg data for open, close, high, low, volume data.
From what I ...
0
votes
0answers
44 views
Energy estimation through machine learning
Greedings to everybody.
I have the dataset which you can find here, containing many different characteristics of different houses, including their types of heating, or the number of adults and ...
0
votes
1answer
17 views
in nonlinear binary classification problems, which is the optimal dimension for make it lineary separable?
My question pertains to linear separability with hyperplanes in a support vector machine.
Is posible to determinate the optimal dimension in which i have to transform a training data set for make it ...
1
vote
0answers
17 views
confusion related to the dual of svm
I have a confusion related to the dual of svm
In the main objective function I have
Now to solve the dual of this objective function, I will minimize with respect to the primal variables first to ...
0
votes
0answers
26 views
Confusion related to L2 and L1 SVM
I have this confusion related to L1 and L2 svm. I was reading this paper
I am attaching the screenshot and the part I didn't understand
The part that I didn't understand how it was derived
I ...
0
votes
0answers
25 views
coordinated dual descent method and sequential minimal optimization
Libsvm uses the sequential minimal optimization as its main solver while Liblinear uses coordinated dual descent method. What are the major differences between these two methods? Looks like both of ...
1
vote
0answers
54 views
Increasing the value of C in SVM (LibSVM) is not changing the accuracy at all
I am trying to learn SVM Classifier using some amount of training data.
and then I am predicting for another set (independent from training data)
I tried random C values from 0.000001 to 50000000 and ...
2
votes
2answers
81 views
ML with fastest classification speed
I have a data classification problem and I'm wondering what is the best machine learning approach to use for the particular constraints of my problem.
My constraints are as follows:
- the data ...
7
votes
1answer
271 views
Occam's razor obsolete?
I saw Vapnik's books about statistical learning... I read the first few chapters.
Anyway what surprised me the most was that he thought that the Occam's razor was obsolete.
I thought it was related ...
1
vote
1answer
56 views
SVM model selection for datasets with sharp corners
I'm working with an artificially generated dataset that is separated by many sharp corners. As an example, imagine an H-shape in a 3D (or higher-dimensional) space. Points within the H are positive, ...
4
votes
2answers
81 views
Support Vector machine : a simple question
I think that a formulation of SVM for points x with label y is :
$$
\begin{align}
\arg\min_{\substack{u,w,b}} \frac{1}{2} \cdot |w|^2 + C \cdot \sum_{i} u_i \\
s.t.\ \ y_i\cdot (w \cdot x_i + b) ...
1
vote
0answers
19 views
Performance worse with new observations
I come from the computer science area but am new to machine learning / stats, so this question may be fundamental and easy.
I have time-series data (biological data), and, without getting into the ...
1
vote
1answer
68 views
SVM confidence according to distance from hyperline
For a probabilistic multi-class classifier we can get probabilities of membership of a new point $x$ to each class $y_i$; in case of 3 classes suppose that we get $P(y_a|x) > P(y_b|x) > ...
0
votes
2answers
74 views
Normalizing SVM predications to [0,1]
I have trained an linear SVM which takes a pair of objects, computes features and is expected to learn a semantic similarity function between objects(we can say that it predicts whether the two ...
2
votes
0answers
27 views
Why are the Lagrange multipliers sparse?
I've read that for the Maximal Margin Classifier SVM, after solving the dual problem, most of the lagrange multipliers turn out to be zeros. Only the ones corresponding to the support vectors turn out ...
1
vote
1answer
109 views
Number of kernel evaluations in SVM training
What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training?
I am asking this question because I need to ...
0
votes
0answers
41 views
CV acc mismatch the prediction
Setting the Context :
My project is in C++, I'm using OpenCV svm here
I used the function train_auto for the CV, however, I implemented my own cross-validation base on this Matlab example here (I ...
2
votes
0answers
61 views
Probability output from support vector machine (svm) with soft margin
Based on my very simple understanding of SVMs, it seems like a probabilistic output would be a very useful thing to have. Soft margin seems to part of the way toward accounting for noisy data, but ...
