# Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

1,971 questions
Filter by
Sorted by
Tagged with
135 views

### SVM ensemble with logistic regression

It is possible to average several logistic models in a ensemble using the estimated probabilities of the models. Does it make sense to calculate an ensemble based on the raw SVM score, i.e. the ...
9 views

### SVM: Getting number of support vectors number and relationship between C and alpha in Python sklearn SGDClassifier

I am using sklearn.SGDClassifier to train my SVM model with loss='hinge'. My questions are: Is there a way to get support vectors number by having this SGD model? I found this online but it is not ...
14 views

### Organizing Data for hourly and daily predictions

Let's suppose I'm using SVM (Regression) to predict variable y and I have multiple input variables (x_i) which are data from sensors at intervals of 10 minutes. From an operational point of view, I ...
11 views

### Relation between Bayesian Linear Regression (Fixed base: Gaussian RBF) and SVM RBF?

I am trying to get my head around Bayesian Linear Regression. I am looking at a Gaussian radial basis function, which I assume acts as our prior. I have the following diagram: My current ...
9 views

### What is the decision function used in sklearn's SVC SVM

When I am using sklearn.svm.SVC I have set the decision function to 'ovr', but am struggling to find the exact decision function equation at the moment. Can anyone help me out? Thanks in advance.
20 views

### Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
7 views

### Why do Support Vector Data Description and One Class Support Vector Machine produce the same results?

Quoating from Chapter 5 of Kernel Methods in Computer Vision by Christoph H. Lampert 'A quick geometric check shows that if all data points have the same feature space norm and can be separated ...
8 views

### Precision and recall for SVM from Confusion matrix is different from Precision-Recall graph

Coming from Stackoverflow- So, I am creating a SVM model for a highly imbalanced data set and trying to create to calculate F, Pression and recall from the confusion matrix of the model. Confusion ...
9 views

### How to inplement SVOREX and SVORIM?

I'm currently working on an ordinal regression problem, and this paper seems to suggest that the best model to use (at least according to their study) is the SVOREX and SVORIM (Ordinal support vector ...
31 views

### Can non-linearly separable data always be made linearly separable?

A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
11 views

### How to convert SVM values into corresponding class type?

I am doing SVM on R, and when I do: ...
14 views

### Kernelized perceptron algorithm weights update

I'm asked to find the maximum margin decision surface separating positive from negative samples by inspection. The positive examples are (1,1) and (-1,-1), the negative ones are (1,-1) and (-1,1). The ...
363 views

### Finding optimal hyperplane

I have a set of vectors $\{V_i\}$ in $n$-dimensional space. There is a number corresponded to each vector $\alpha_i = f(V_i)$ ($\alpha_i$ can be negative). I want to find a hyperplane which would ...
573 views

### How does the shape of a decision boundary in relate between the original and kernel feature space?

I'm trying to get my head around the mathematics and implementation of SVM and hopefully gain some intuition into how kernels work and perhaps being able to, with a bit more confidence, define my own ...
53 views

### Why does the supporting vector satisfy $y_i(\mathbf{w}^T\mathbf{x_i}+b) = 1$ instead of $> 1$ or $= 2$

The SVM is about solving the constrained optimization such that $$\min_{\mathbf{w}} \dfrac{1}{2} \mathbf{w}^T\mathbf{w}$$ subject to $$y_i(\mathbf{w}^T\mathbf{x_i}+b)\geq{1}, i=1, 2, ...,n$$ ...
339 views

### Does SVM get biased towards majority class in case of imbalanced class proportion?

After reading many posts, I thought of asking: Why should a SVM be biased towards majority class like other classifiers, since an SVM never used the whole data of the training data set—it only uses ...
1k views

### Shapes of ROC curves for different classifiers ('steppy' for SVM and smooth for k-NN)

When constructing the ROC curve for various classifiers I've noticed that their actual shapes tend to be very different for models such as logistic regression or SVM compared to k-NN. For instance, in ...
25 views

### Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
31 views

### Optimization equivalence

Can someone help me with the step by step demonstration of the following equivalence used in SVM: $$maximize: m = \frac{1} {\|w\|} \equiv minimize: m =\frac{1} {2}\|w\|^2$$ I would be most grateful ...
18 views

### Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
2k views

### Training an SVM classifier with non-negative weight constraint

I have a problem, where I need to learn a classifier (such as SVM) such that all the learned weights to be non-negative due a constraint on the classifier function. I found out that "SVM Struct" is ...
23 views

### Kernel selection for one-class SVM learning

Has anyone seen compelling research on kernel selection for one-class SVM learning? I've not tracked this work in some time and am wondering if there's new work I've missed, particularly from the ...
320 views

### How to increase a particular terms's weightage?

I am doing Text classification using LibSVM in Rapid Miner. I am using TFIDF values for processing documents. I need to Increase weightage of some terms in the documents(for eg. words in BOLD and ...
15 views

### Dimensions of feature transfrom $\phi(x)$ for a kernel Support Vector Machines

Given a kernel function $K(x, x') = \langle \phi(x), \phi(x') \rangle$, how can we figure out the dimensions of the feature transform $\phi(x)$? For example, for $K(x, x') = (1+x^Tx')^M$
87 views

### How should I construct a binary classifier for thousands of positive data and millions of unlabeled data?

So far, I have stumbled upon many advices and papers on PU Learning and Unary classification. The simplest answer has been one-class SVM (https://stackoverflow.com/questions/25700724/binary-semi-...
14 views

### Using Support Vector Regression with Presence Absence Data - treating Y as a continuous variable

I am building species distribution models with boosted regression trees and support vector machines using a large number of Presence-(Pseudo)absence data (> 10.000 plots) Since my goal is not to ...
17 views

### Comparing margin width of SVM with different kernels as a performance metric

Assume we have applied SVM with different kernels to a problem, Alongside the performance metrics like accuracy, precision, etc, can we compare the margin size to decide which kernel is the best?
114 views

### How can I standarize/normalize my categorical, factorized features in outliers detection problem?

I'm working on anomaly detection in CTU-13 dataset. Records are labeled and there are a few categorical features with many categories (for example one of the features "State" has over 250 possible, ...
37 views

### High AUC, low f1, SVM threshold for an unbalanced problem

I have a very unbalanced binary classification problem (positive class: 0.2%). I need to evaluate it using f1 of the positive class. Now, I'm doing some baselines using an SVM. What I get is a ...
150 views

### nonseparable case of classification problem (SVM)

I am learning Soft Margin Classification (SVM) right now. In cases when the classes are non-separable by the usual hyperplane with a margin $M >0$, we modify the constraints and say that it is ok ...
24 views

### How to calculate Bias and Variance for SVM and Random Forest Model

I'm working on a classification problem (predicting three classes) and I'm comparing SVM against Random Forest in R. For evaluation and comparison I want to calculate the bias and variance of the ...
25 views

### Does there a non-linear SVM exist?

SVM is a linear classifier. But some articles talk about non-linear SVM that is quite contradictory. A "non-linear SVM" can perform non-linear classification over a dataset that is not linearly ...
137 views

### Normalization for pattern classification?

I'm working off my first independent project for some pattern classification. I'm utilizing some datasets from UCI machine learning, but am not sure on how to start with data normalization. The data ...
15k views

### Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
14 views

### Decrease hyparam 'C' in SVM classifier

In a hypothetical case where I have a small dataset and I break it into train/test. Then I tune the hyperparams doing k-fold on the train set and choose the 'C' hyperparameter that maximizes my AUC on ...
2k views

### How probabilities are calculated for SVM model?

I would like to know, how probabilities are calculated in support vector machine. I have used Iris data set and here is my decision values for three "SupportVectorMachine" (please find the PMML below ...
2k views

### SVM kernel parameter and tunning parameter

In the svm function, you can apply three cases to the kernel parameter. "Linear," "radial," and "polynomia." And I try to derive the optimal svm result by ...
15 views

### Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC (SVM scikit-learn) by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best ...
484 views

### Corresponding RKHS of Common Kernels

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian ...
42 views

### How can I tell if I am overtraining my support vector machine?

I am trying to train a Support Vector Machine (SVM) classifier to classify various items into 5 categories. I have trained two SVM classifiers, however, I am concerned that the accuracies and F1 ...
6 views

### Predicting post translational modifications(PTM) sites using SVM

I have dataset that contains 6394 samples with length of 27. Number of positive samples 3991 and negative samples 2403. I have used the sequences as features for positive-negative classes ...
68k views

### How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
20 views

### Effect of Gamma and C on distant points in SVM

(I am aware of the following, already answered questions: this,and this,as well as others, and IMHO they are not related) I am trying to precisely understand the behavior of SVM's with regards to the ...
21k views

### Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by ...
147 views

### Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss setup ...
4k views

### Best way to train one-class SVM

Let`s say I have training data which contains 10 classes and have build a classifier using this data. When applying this classifier in real life it may encounter examples not belong to the classes ...
175 views

### (Linear regression) Can I train and validate at the same time using the following approach?

In a lot of material I found online, training and validation seems to be an iterative process For example, the regularized regression problem $E = \|Xw - t\|_2^2 + \lambda \|w\|^2_2$ $X$ is data ...
27 views

### Stable model or overfitting?

I have a dataset of 150 patients (2:1 ratio of classes) and 78 features. I performed backwards elimination using logistic regression feature importance to end up with 13 features (SVC classifier). I ...