Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
2
votes
0answers
135 views

SVM ensemble with logistic regression

It is possible to average several logistic models in a ensemble using the estimated probabilities of the models. Does it make sense to calculate an ensemble based on the raw SVM score, i.e. the ...
0
votes
0answers
9 views

SVM: Getting number of support vectors number and relationship between C and alpha in Python sklearn SGDClassifier

I am using sklearn.SGDClassifier to train my SVM model with loss='hinge'. My questions are: Is there a way to get support vectors number by having this SGD model? I found this online but it is not ...
0
votes
0answers
14 views

Organizing Data for hourly and daily predictions

Let's suppose I'm using SVM (Regression) to predict variable y and I have multiple input variables (x_i) which are data from sensors at intervals of 10 minutes. From an operational point of view, I ...
0
votes
0answers
11 views

Relation between Bayesian Linear Regression (Fixed base: Gaussian RBF) and SVM RBF?

I am trying to get my head around Bayesian Linear Regression. I am looking at a Gaussian radial basis function, which I assume acts as our prior. I have the following diagram: My current ...
1
vote
0answers
9 views

What is the decision function used in sklearn's SVC SVM

When I am using sklearn.svm.SVC I have set the decision function to 'ovr', but am struggling to find the exact decision function equation at the moment. Can anyone help me out? Thanks in advance.
0
votes
1answer
20 views

Cannot use SVM with RBF Kernel

I'm new in R. I have an original dataset with 25771 variables and 118 samples. I already performed feature selection and split the dataset into 70 30 so i have 82 samples in my training data and 36 in ...
0
votes
0answers
7 views

Why do Support Vector Data Description and One Class Support Vector Machine produce the same results?

Quoating from Chapter 5 of Kernel Methods in Computer Vision by Christoph H. Lampert 'A quick geometric check shows that if all data points have the same feature space norm and can be separated ...
0
votes
0answers
8 views

Precision and recall for SVM from Confusion matrix is different from Precision-Recall graph

Coming from Stackoverflow- So, I am creating a SVM model for a highly imbalanced data set and trying to create to calculate F, Pression and recall from the confusion matrix of the model. Confusion ...
0
votes
0answers
9 views

How to inplement SVOREX and SVORIM?

I'm currently working on an ordinal regression problem, and this paper seems to suggest that the best model to use (at least according to their study) is the SVOREX and SVORIM (Ordinal support vector ...
2
votes
1answer
31 views

Can non-linearly separable data always be made linearly separable?

A data set that is linearly separable is a precondition for algorithms like the perceptron to converge. It's well-known that we can project low-dimensional data to a higher dimension using kernel ...
0
votes
1answer
11 views
1
vote
1answer
14 views

Kernelized perceptron algorithm weights update

I'm asked to find the maximum margin decision surface separating positive from negative samples by inspection. The positive examples are (1,1) and (-1,-1), the negative ones are (1,-1) and (-1,1). The ...
3
votes
1answer
363 views

Finding optimal hyperplane

I have a set of vectors $\{V_i\}$ in $n$-dimensional space. There is a number corresponded to each vector $\alpha_i = f(V_i)$ ($\alpha_i$ can be negative). I want to find a hyperplane which would ...
0
votes
1answer
573 views

How does the shape of a decision boundary in relate between the original and kernel feature space?

I'm trying to get my head around the mathematics and implementation of SVM and hopefully gain some intuition into how kernels work and perhaps being able to, with a bit more confidence, define my own ...
1
vote
2answers
53 views

Why does the supporting vector satisfy $y_i(\mathbf{w}^T\mathbf{x_i}+b) = 1$ instead of $> 1$ or $= 2$

The SVM is about solving the constrained optimization such that $$\min_{\mathbf{w}} \dfrac{1}{2} \mathbf{w}^T\mathbf{w}$$ subject to $$y_i(\mathbf{w}^T\mathbf{x_i}+b)\geq{1}, i=1, 2, ...,n$$ ...
1
vote
1answer
339 views

Does SVM get biased towards majority class in case of imbalanced class proportion?

After reading many posts, I thought of asking: Why should a SVM be biased towards majority class like other classifiers, since an SVM never used the whole data of the training data set—it only uses ...
5
votes
1answer
1k views

Shapes of ROC curves for different classifiers ('steppy' for SVM and smooth for k-NN)

When constructing the ROC curve for various classifiers I've noticed that their actual shapes tend to be very different for models such as logistic regression or SVM compared to k-NN. For instance, in ...
2
votes
1answer
25 views

Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
2
votes
2answers
31 views

Optimization equivalence

Can someone help me with the step by step demonstration of the following equivalence used in SVM: $$maximize: m = \frac{1} {\|w\|} \equiv minimize: m =\frac{1} {2}\|w\|^2 $$ I would be most grateful ...
0
votes
0answers
18 views

Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
4
votes
4answers
2k views

Training an SVM classifier with non-negative weight constraint

I have a problem, where I need to learn a classifier (such as SVM) such that all the learned weights to be non-negative due a constraint on the classifier function. I found out that "SVM Struct" is ...
0
votes
1answer
23 views

Kernel selection for one-class SVM learning

Has anyone seen compelling research on kernel selection for one-class SVM learning? I've not tracked this work in some time and am wondering if there's new work I've missed, particularly from the ...
0
votes
1answer
320 views

How to increase a particular terms's weightage?

I am doing Text classification using LibSVM in Rapid Miner. I am using TFIDF values for processing documents. I need to Increase weightage of some terms in the documents(for eg. words in BOLD and ...
1
vote
0answers
15 views

Dimensions of feature transfrom $\phi(x)$ for a kernel Support Vector Machines

Given a kernel function $K(x, x') = \langle \phi(x), \phi(x') \rangle$, how can we figure out the dimensions of the feature transform $\phi(x)$? For example, for $K(x, x') = (1+x^Tx')^M$
1
vote
0answers
87 views

How should I construct a binary classifier for thousands of positive data and millions of unlabeled data?

So far, I have stumbled upon many advices and papers on PU Learning and Unary classification. The simplest answer has been one-class SVM (https://stackoverflow.com/questions/25700724/binary-semi-...
0
votes
0answers
14 views

Using Support Vector Regression with Presence Absence Data - treating Y as a continuous variable

I am building species distribution models with boosted regression trees and support vector machines using a large number of Presence-(Pseudo)absence data (> 10.000 plots) Since my goal is not to ...
0
votes
1answer
17 views

Comparing margin width of SVM with different kernels as a performance metric

Assume we have applied SVM with different kernels to a problem, Alongside the performance metrics like accuracy, precision, etc, can we compare the margin size to decide which kernel is the best?
1
vote
1answer
114 views

How can I standarize/normalize my categorical, factorized features in outliers detection problem?

I'm working on anomaly detection in CTU-13 dataset. Records are labeled and there are a few categorical features with many categories (for example one of the features "State" has over 250 possible, ...
0
votes
1answer
37 views

High AUC, low f1, SVM threshold for an unbalanced problem

I have a very unbalanced binary classification problem (positive class: 0.2%). I need to evaluate it using f1 of the positive class. Now, I'm doing some baselines using an SVM. What I get is a ...
2
votes
1answer
150 views

nonseparable case of classification problem (SVM)

I am learning Soft Margin Classification (SVM) right now. In cases when the classes are non-separable by the usual hyperplane with a margin $M >0$, we modify the constraints and say that it is ok ...
1
vote
0answers
24 views

How to calculate Bias and Variance for SVM and Random Forest Model

I'm working on a classification problem (predicting three classes) and I'm comparing SVM against Random Forest in R. For evaluation and comparison I want to calculate the bias and variance of the ...
1
vote
2answers
25 views

Does there a non-linear SVM exist?

SVM is a linear classifier. But some articles talk about non-linear SVM that is quite contradictory. A "non-linear SVM" can perform non-linear classification over a dataset that is not linearly ...
0
votes
1answer
137 views

Normalization for pattern classification?

I'm working off my first independent project for some pattern classification. I'm utilizing some datasets from UCI machine learning, but am not sure on how to start with data normalization. The data ...
25
votes
3answers
15k views

Feature map for the Gaussian kernel

In SVM, the Gaussian kernel is defined as: $$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$. I do not know the explicit equation of $\phi$. I ...
1
vote
0answers
14 views

Decrease hyparam 'C' in SVM classifier

In a hypothetical case where I have a small dataset and I break it into train/test. Then I tune the hyperparams doing k-fold on the train set and choose the 'C' hyperparameter that maximizes my AUC on ...
5
votes
1answer
2k views

How probabilities are calculated for SVM model?

I would like to know, how probabilities are calculated in support vector machine. I have used Iris data set and here is my decision values for three "SupportVectorMachine" (please find the PMML below ...
0
votes
1answer
2k views

SVM kernel parameter and tunning parameter

In the svm function, you can apply three cases to the kernel parameter. "Linear," "radial," and "polynomia." And I try to derive the optimal svm result by ...
0
votes
0answers
15 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC (SVM scikit-learn) by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best ...
3
votes
2answers
484 views

Corresponding RKHS of Common Kernels

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian ...
0
votes
0answers
42 views

How can I tell if I am overtraining my support vector machine?

I am trying to train a Support Vector Machine (SVM) classifier to classify various items into 5 categories. I have trained two SVM classifiers, however, I am concerned that the accuracies and F1 ...
0
votes
0answers
6 views

Predicting post translational modifications(PTM) sites using SVM

I have dataset that contains 6394 samples with length of 27. Number of positive samples 3991 and negative samples 2403. I have used the sequences as features for positive-negative classes ...
111
votes
4answers
68k views

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is ...
1
vote
0answers
20 views

Effect of Gamma and C on distant points in SVM

(I am aware of the following, already answered questions: this,and this,as well as others, and IMHO they are not related) I am trying to precisely understand the behavior of SVM's with regards to the ...
48
votes
3answers
21k views

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by ...
7
votes
1answer
147 views

Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss setup ...
2
votes
1answer
4k views

Best way to train one-class SVM

Let`s say I have training data which contains 10 classes and have build a classifier using this data. When applying this classifier in real life it may encounter examples not belong to the classes ...
2
votes
2answers
175 views

(Linear regression) Can I train and validate at the same time using the following approach?

In a lot of material I found online, training and validation seems to be an iterative process For example, the regularized regression problem $E = \|Xw - t\|_2^2 + \lambda \|w\|^2_2$ $X$ is data ...
0
votes
1answer
27 views

Stable model or overfitting?

I have a dataset of 150 patients (2:1 ratio of classes) and 78 features. I performed backwards elimination using logistic regression feature importance to end up with 13 features (SVC classifier). I ...
1
vote
1answer
64 views

Is there an ExtraTreesClassifier-like classifier that has decision boundary function like SVM?

I'm using sklearn and I tested many models and those two worked best: Linear SVM and the ExtraTreesClassifier as binary classifiers. The ExtraTreesClassifier outperforms the Linear SVM in terms of ...
0
votes
0answers
20 views

Working of Dual Perceptron Algorithm

I was going for the theory and maths behind the online perceptron algorithm and it is very easy to under stand it intuitively that on a positive mistake, you just add the ...

1 2 3 4 5 40