Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
113
votes
5answers
63k views

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
13
votes
1answer
8k views

How to know if a learning curve from SVM model suffers from bias or variance?

I created this learning curve and I want to know if my SVM model suffers from bias or variance? How can I conclude that from this graph?
35
votes
3answers
18k views

How to prove that the radial basis function is a kernel?

How to prove that the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ is a kernel? As far as I understand, in order to prove this we have to prove either of the following: For ...
151
votes
7answers
238k views

What is the influence of C in SVMs with linear kernel?

I am currently using an SVM with a linear kernel to classify my data. There is no error on the training set. I tried several values for the parameter $C$ ($10^{-5}, \dots, 10^2$). This did not ...
79
votes
1answer
7k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
20
votes
3answers
29k views

What algorithms need feature scaling, beside from SVM?

I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
28
votes
3answers
17k views

Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
100
votes
4answers
74k views

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?
38
votes
4answers
10k views

How can SVM 'find' an infinite feature space where linear separation is always possible?

What is the intuition behind the fact that an SVM with a Gaussian Kernel has infinite dimensional feature space?
46
votes
2answers
67k views

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
16
votes
5answers
13k views

Kernel SVM: I want an intuitive understanding of mapping to a higher-dimensional feature space, and how this makes linear separation possible

I am trying to understand the intuition behind kernel SVM's. Now, I understand how linear SVM's work, whereby a decision line is made which splits the data as best it can. I also understand the ...
48
votes
5answers
87k views

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. (I'm using scikit-learn): ...
47
votes
4answers
65k views

Comparing SVM and logistic regression

Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the ...
40
votes
3answers
50k views

SVM, Overfitting, curse of dimensionality

My dataset is small (120 samples), however the number of features are large varies from (1000-200,000). Although I'm doing feature selection to pick a subset of features, it might still overfit. My ...
5
votes
1answer
5k views

How to calculate decision boundary from support vectors?

I want to obtain decision boundary of SVM using OpenCV 2.4.11, but it seems that it's not returning it explicitly, but only support vectors. How we can calculate decision boundary from support ...
125
votes
4answers
76k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
56
votes
4answers
36k views

Why bother with the dual problem when fitting SVM?

Given the data points $x_1, \ldots, x_n \in \mathbb{R}^d$ and labels $y_1, \ldots, y_n \in \left \{-1, 1 \right\}$, the hard margin SVM primal problem is $$ \text{minimize}_{w, w_0} \quad \frac{1}{2} ...
16
votes
3answers
21k views

SVM for unbalanced data

I want to attempt to use Support Vector Machines (SVMs) on my dataset. Before I attempt the problem though, I was warned that SVMs dont perform well on extremely unbalanced data. In my case, I can ...
4
votes
1answer
6k views

SVM: Why does the number of support vectors decrease when C is increased?

I am learning how to use libsvm through sklearn.svm in python. I read here about what happens and why when you change the C value as part of your model. My intuition from what I've learned would be ...
69
votes
4answers
39k views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
33
votes
2answers
43k views

Which search range for determining SVM optimal C and gamma parameters?

I am using SVM for classification and I am trying to determine the optimal parameters for linear and RBF kernels. For the linear kernel I use cross-validated parameter selection to determine C and for ...
5
votes
1answer
2k views

Cross-validation techniques for time series data

What is an appropriate cross-validation technique for time series data? I have a daily 4 years time series data and fitting a SVM model by MATLAB R2015b: ...
21
votes
5answers
24k views

How to recode categorical variable into numerical variable when using SVM or Neural Network

To use SVM or Neural Network it needs to transform (encode) categorical variables into numeric variables, the normal method in this case is to use 0-1 binary values with the k-th categorical value ...
18
votes
2answers
17k views

Best way to perform multiclass SVM

I know that the SVM is a binary classifier. I would like to extend it to multi-class SVM. Which is the best, and maybe the easiest, way to perform it? code: in MATLAB ...
2
votes
1answer
1k views

Generalized RBF Kernels

There is the notion of Generalized RBF Kernels, for example in "Towards Optimal Bag-of-Features for Object Categorization and Semantic Video Retrieval" from Jiang (1) or in formula (2.72) in http://...
6
votes
1answer
757 views

What do “real values” refer to in supervised classification?

I'm using supervised classification algorithms from mlpy to classify things into two groups for a question-answering system. I don't really know how these algorithms work, but they seem to be doing ...
30
votes
1answer
50k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
8
votes
3answers
4k views

Choosing a classification performance metric for model selection, feature selection, and publication

I have a small, unbalanced data set (70 positive, 30 negative), and I have been playing around with model selection for SVM parameters using BAC (balanced accuracy) and AUC (area under the curve). I ...
13
votes
1answer
11k views

Can support vector machine be used in large data?

With the limited knowledge I have on SVM, it is good for a short and fat data matrix $X$, (lots of features, and not too many instances), but not for big data. I understand one reason is the Kernel ...
9
votes
2answers
10k views

When using SVMs, why do I need to scale the features?

According to the documentation of the StandardScaler object in scikit-learn: For instance many elements used in the objective function of a learning algorithm (such as the RBF kernel of Support ...
7
votes
3answers
2k views

Is a lower training accuracy possible in overfitting (one class SVM)

I am using the heart_scale data from LibSVM. The original data includes 13 features, but I only used 2 of them in order to plot the distributions in a figure. Instead of training the binary classifier,...
15
votes
3answers
497 views

What does the “machine” in “support vector machine” and “restricted Boltzmann machine” mean?

Why are they called "machines"? Is there an origin to the word "machine" used in this context? (Like the name "linear programming" can be confusing but we know why it is called "programming.")
1
vote
1answer
3k views

Proof of sum of kernels of concatenated vector

I am reading the Pattern Recognition and Machine Learning book by Bishops. I have some difficulties proving the (6.21) equation. Can someone help me? N.B: If ...
56
votes
5answers
52k views

Neural networks vs support vector machines: are the second definitely superior?

Many authors of papers I read affirm SVMs is superior technique to face their regression/classification problem, aware that they couldn't get similar results through NNs. Often the comparison states ...
19
votes
2answers
17k views

Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
20
votes
1answer
43k views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
18
votes
1answer
33k views

How to understand effect of RBF SVM

How can I understand what the RBF Kernel in SVM does? I mean I understand the maths, but is there a way to get a feeling when this kernel will be useful? Would results from kNN be related to SVM/RBF ...
8
votes
1answer
16k views

Non-linear SVM classification with RBF kernel

I'm implementing a non-linear SVM classifier with RBF kernel. I was told that the only difference from a normal SVM was that I had to simply replace the dot product with a kernel function: $$ K(x_i,...
22
votes
3answers
7k views

Is Gradient Descent possible for kernelized SVMs (if so, why do people use Quadratic Programming)?

Why do people use Quadratic Programming techniques (such as SMO) when dealing with kernelized SVMs? What is wrong with Gradient Descent? Is it impossible to use with kernels or is it just too slow (...
13
votes
2answers
894 views

KKT in a nutshell graphically

Objective Confirm if the understanding of KKT is correct or not. Seek for further explanation and confirmations on KKT. Background Trying to understand KKT conditions, especially the complementary ...
10
votes
1answer
4k views

Given a set of points in two dimensional space, how can one design decision function for SVM?

Can someone explain me how one goes about designing a SVM decision function? Or point me to resource that discusses a concrete example. EDIT For the below example, I can see that the equation $X_2 = ...
6
votes
3answers
9k views

Which performance measure to use when using SVM: MSE or MAE?

It is a common practice to measure an SVM model's performance by calculating its MSE (Mean Square Error). Why not use Mean Absolute Error (averaging errors' absolute values instead of squared values)?
9
votes
1answer
13k views

Difference between the types of SVM

I am new to support vector machines. Short explanation The svm function from the e1071 package in R offers various options: ...
7
votes
1answer
14k views

What does the cost (C) parameter mean in SVM?

I am trying to fit a SVM to my data. My dataset contains 3 classes and I am performing 10 fold cross validation (in LibSVM): ...
3
votes
1answer
9k views

About SVM cost and gamma parameters tuning

I am using R and e1071 package to tune a C-classification SVM. My question is: regardless of the kernel type (linear, ...
15
votes
2answers
31k views

Why scaling is important for the linear SVM classification?

When performing the linear SVM classification, it is often helpful to normalize the training data, for example by subtracting the mean and dividing by the standard deviation, and afterwards scale ...
13
votes
3answers
4k views

What is a kernel and what sets it apart from other functions

There seem to be many machine learning algorithms that rely on kernel functions. SVMs and NNs to name but two. So what is the definition of a kernel function and what are the requirements for it to be ...
9
votes
1answer
7k views

Best way to handle unbalanced multiclass dataset with SVM

I'm trying to build a prediction model with SVMs on fairly unbalanced data. My labels/output have three classes, positive, neutral and negative. I would say the positive example makes about 10 - 20% ...
9
votes
1answer
1k views

Why would scaling features decrease SVM performance?

I have used scaling on features of a model which contains 40 features (all columns are numbers) and a binary output variable. This is the Kaggle contest here I've scaled the features assuming it ...
6
votes
3answers
10k views

LibSVM cost weights for unbalanced data doesn't work

I have a dataset where the number of negative labeled values is 163 times the number of positive labeled values. That is, I have an unbalanced data set. I tried: ...

1
2 3 4 5