Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
149
votes
7answers
234k views

What is the influence of C in SVMs with linear kernel?

I am currently using an SVM with a linear kernel to classify my data. There is no error on the training set. I tried several values for the parameter $C$ ($10^{-5}, \dots, 10^2$). This did not ...
119
votes
4answers
74k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
113
votes
5answers
63k views

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
100
votes
4answers
73k views

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?
79
votes
1answer
7k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
69
votes
4answers
39k views

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive ...
56
votes
4answers
35k views

Why bother with the dual problem when fitting SVM?

Given the data points $x_1, \ldots, x_n \in \mathbb{R}^d$ and labels $y_1, \ldots, y_n \in \left \{-1, 1 \right\}$, the hard margin SVM primal problem is $$ \text{minimize}_{w, w_0} \quad \frac{1}{2} ...
55
votes
5answers
52k views

Neural networks vs support vector machines: are the second definitely superior?

Many authors of papers I read affirm SVMs is superior technique to face their regression/classification problem, aware that they couldn't get similar results through NNs. Often the comparison states ...
52
votes
6answers
26k views

What are alternatives of Gradient Descent?

Gradient Descent has a problem of getting stuck in Local Minima. We need to run gradient descent exponential times in order to find global minima. Can anybody tell me about any alternatives of ...
49
votes
3answers
21k views

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by ...
48
votes
5answers
85k views

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. (I'm using scikit-learn): ...
46
votes
2answers
67k views

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
46
votes
4answers
64k views

Comparing SVM and logistic regression

Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the ...
39
votes
3answers
49k views

SVM, Overfitting, curse of dimensionality

My dataset is small (120 samples), however the number of features are large varies from (1000-200,000). Although I'm doing feature selection to pick a subset of features, it might still overfit. My ...
38
votes
4answers
10k views

How can SVM 'find' an infinite feature space where linear separation is always possible?

What is the intuition behind the fact that an SVM with a Gaussian Kernel has infinite dimensional feature space?
37
votes
3answers
40k views

What is meant by 'weak learner'?

Can anyone tell me what is meant by the phrase 'weak learner'? Is it supposed to be a weak hypothesis? I am confused about the relationship between a weak learner and a weak classifier. Are both the ...
35
votes
3answers
17k views

How to prove that the radial basis function is a kernel?

How to prove that the radial basis function $k(x, y) = \exp(-\frac{||x-y||^2)}{2\sigma^2})$ is a kernel? As far as I understand, in order to prove this we have to prove either of the following: For ...
34
votes
5answers
20k views

Can SVM do stream learning one example at a time?

I have a streaming data set, examples are available one at a time. I would need to do multi class classification on them. As soon as I fed a training example to the learning process, I have to discard ...
33
votes
3answers
4k views

Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

I have seen people have put a lot of efforts on SVM and Kernels, and they look pretty interesting as a starter in Machine Learning. But if we expect that almost-always we could find outperforming ...
33
votes
3answers
19k views

Kernel logistic regression vs SVM

As is known to all, SVM can use kernel method to project data points in higher spaces so that points can be separated by a linear space. But we can also use logistic regression to choose this ...
33
votes
2answers
43k views

Which search range for determining SVM optimal C and gamma parameters?

I am using SVM for classification and I am trying to determine the optimal parameters for linear and RBF kernels. For the linear kernel I use cross-validated parameter selection to determine C and for ...
33
votes
3answers
23k views

Variable importance from SVM

How to obtain a variable (attribute) importance using SVM?
31
votes
2answers
75k views

libsvm data format [closed]

I'm using the libsvm (http://www.csie.ntu.edu.tw/~cjlin/libsvm/) tool for support vector classification. However, I'm confused about the format of the input data. From the README: The format of ...
31
votes
3answers
26k views

Difference between a SVM and a perceptron

I am a bit confused with the difference between an SVM and a perceptron. Let me try to summarize my understanding here, and please feel free to correct where I am wrong and fill in what I have missed. ...
30
votes
1answer
49k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
30
votes
3answers
8k views

How well does R scale to text classification tasks? [closed]

I am trying to get upto speed with R. I eventually want to use R libraries for doing text classification. I was just wondering what people's experiences are with regard to R's scalability when it ...
30
votes
4answers
66k views

How to determine the optimal threshold for a classifier and generate ROC curve?

Let say we have a SVM classifier, how do we generate ROC curve? (Like theoretically) (because we are generate TPR and FPR with each of the threshold). And how do we determine the optimal threshold for ...
29
votes
2answers
34k views

Is it essential to do normalization for SVM and Random Forest?

My features' every dimension has different range of value. I want to know if it is essential to normalize this dataset.
28
votes
2answers
3k views

What is the statistical model behind the SVM algorithm?

I have learned that, when dealing with data using model-based approach, the first step is modeling data procedure as a statistical model. Then the next step is developing efficient/fast inference/...
27
votes
3answers
16k views

Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
27
votes
4answers
55k views

The difference of kernels in SVM?

Can someone please tell me the difference between the kernels in SVM: Linear Polynomial Gaussian (RBF) Sigmoid Because as we know that kernel is used to mapped our input space into high ...
26
votes
2answers
5k views

Support vector machines and regression

There's already been an excellent discussion on how support vector machines handle classification, but I'm very confused about how support vector machines generalize to regression. Anyone care to ...
25
votes
10answers
13k views

Why not just dump the neural networks and deep learning? [closed]

Fundamental problem with deep learning and neural networks in general. The solutions that fit training data are infinite. We don't have precise mathematical equation that is satisfied by only a ...
25
votes
2answers
25k views

What is the loss function of hard margin SVM?

People says soft margin SVM use hinge loss function: $\max(0,1-y_i(w^\intercal x_i+b))$. However, the actual objective function that soft margin SVM tries to minimize is $$ \frac{1}{2}\|w\|^2+C\sum_i\...
25
votes
2answers
20k views

How does support vector regression work intuitively?

All the examples of SVMs are related to classification. I don't understand how an SVM for regression (support vector regressor) could be used in regression. From my understanding, A SVM maximizes the ...
23
votes
4answers
3k views

Why are optimization algorithms defined in terms of other optimization problems?

I am doing some research on optimization techniques for machine learning, but I am surprised to find large numbers of optimization algorithms are defined in terms of other optimization problems. I ...
23
votes
4answers
13k views

How to know whether the data is linearly separable?

The data has many features (e.g. 100) and the number of instances is like 100,000. The data is sparse. I want to fit the data using logistic regression or svm. How do I know whether features are ...
23
votes
3answers
21k views

Support vector regression for multivariate time series prediction

Has anyone attempted time series prediction using support vector regression? I understand support vector machines and partially understand support vector regression, but I don't understand how they ...
23
votes
0answers
1k views

How does a Relevance Vector Machine (RVM) work?

Relevance Vector Machines (RVMs) are really interesting models when contrasted with the highly geometrical (and popular) SVMs. In the light of a question like How does a Support Vector Machine (SVM) ...
22
votes
4answers
8k views

Is it a good idea to use CNN to classify 1D signal?

I am working on the sleep stage classification. I read some research articles about this topic many of them used SVM or ensemble method. Is it a good idea to use convolutional neural network to ...
22
votes
3answers
7k views

Is Gradient Descent possible for kernelized SVMs (if so, why do people use Quadratic Programming)?

Why do people use Quadratic Programming techniques (such as SMO) when dealing with kernelized SVMs? What is wrong with Gradient Descent? Is it impossible to use with kernels or is it just too slow (...
21
votes
5answers
24k views

How to recode categorical variable into numerical variable when using SVM or Neural Network

To use SVM or Neural Network it needs to transform (encode) categorical variables into numeric variables, the normal method in this case is to use 0-1 binary values with the k-th categorical value ...
21
votes
1answer
22k views

libsvm “reaching max number of iterations” warning and cross-validation

I'm using libsvm in C-SVC mode with a polynomial kernel of degree 2 and I'm required to train multiple SVMs. Each training set has 10 features and 5000 vectors. During training, I am getting this ...
20
votes
1answer
42k views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
20
votes
1answer
29k views

How to deal with an SVM with categorical attributes

I have a space of 35 dimensions (attributes). My analytic problem is a simple classification one. Out of 35 dimensions, more than 25 are categorical and each attribute takes more than 50+ types of ...
19
votes
3answers
28k views

What algorithms need feature scaling, beside from SVM?

I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
19
votes
1answer
70k views

Advantages and disadvantages of SVM

Can anyone explain to me advantages and disadvantages of classification SVM that distinguishes it from other classifiers?
19
votes
2answers
17k views

Computing the decision boundary of a linear SVM model

Given the support vectors of a linear SVM, how can I compute the equation of the decision boundary?
19
votes
3answers
9k views

Semi-supervised learning, active learning and deep learning for classification

Final edit with all resources updated: For a project, I am applying machine learning algorithms for classification. Challenge: Quite limited labeled data and much more unlabeled data. Goals: Apply ...
18
votes
1answer
33k views

How to understand effect of RBF SVM

How can I understand what the RBF Kernel in SVM does? I mean I understand the maths, but is there a way to get a feeling when this kernel will be useful? Would results from kNN be related to SVM/RBF ...

1
2 3 4 5
41