Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
2
votes
1answer
37 views

Why the weight vector is a linear combination of the inputs and the outputs in the Perceptron

I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane. $w=\sum\limits_{i\in I}^{} y_i x_i$ For reference, I'm studying from the ...
1
vote
0answers
24 views

Dimensions of feature transfrom $\phi(x)$ for a kernel Support Vector Machines

Given a kernel function $K(x, x') = \langle \phi(x), \phi(x') \rangle$, how can we figure out the dimensions of the feature transform $\phi(x)$? For example, for $K(x, x') = (1+x^Tx')^M$
0
votes
0answers
21 views

Using Support Vector Regression with Presence Absence Data - treating Y as a continuous variable

I am building species distribution models with boosted regression trees and support vector machines using a large number of Presence-(Pseudo)absence data (> 10.000 plots) Since my goal is not to ...
0
votes
1answer
28 views

Kernel selection for one-class SVM learning

Has anyone seen compelling research on kernel selection for one-class SVM learning? I've not tracked this work in some time and am wondering if there's new work I've missed, particularly from the ...
0
votes
1answer
44 views

High AUC, low f1, SVM threshold for an unbalanced problem

I have a very unbalanced binary classification problem (positive class: 0.2%). I need to evaluate it using f1 of the positive class. Now, I'm doing some baselines using an SVM. What I get is a ...
1
vote
0answers
54 views

How to calculate Bias and Variance for SVM and Random Forest Model

I'm working on a classification problem (predicting three classes) and I'm comparing SVM against Random Forest in R. For evaluation and comparison I want to calculate the bias and variance of the ...
1
vote
2answers
31 views

Does there a non-linear SVM exist?

SVM is a linear classifier. But some articles talk about non-linear SVM that is quite contradictory. A "non-linear SVM" can perform non-linear classification over a dataset that is not linearly ...
0
votes
1answer
36 views

Comparing margin width of SVM with different kernels as a performance metric

Assume we have applied SVM with different kernels to a problem, Alongside the performance metrics like accuracy, precision, etc, can we compare the margin size to decide which kernel is the best?
1
vote
0answers
17 views

Decrease hyparam 'C' in SVM classifier

In a hypothetical case where I have a small dataset and I break it into train/test. Then I tune the hyperparams doing k-fold on the train set and choose the 'C' hyperparameter that maximizes my AUC on ...
0
votes
0answers
37 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC (SVM scikit-learn) by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best ...
0
votes
0answers
15 views

Predicting post translational modifications(PTM) sites using SVM

I have dataset that contains 6394 samples with length of 27. Number of positive samples 3991 and negative samples 2403. I have used the sequences as features for positive-negative classes ...
0
votes
0answers
75 views

How can I tell if I am overtraining my support vector machine?

I am trying to train a Support Vector Machine (SVM) classifier to classify various items into 5 categories. I have trained two SVM classifiers, however, I am concerned that the accuracies and F1 ...
1
vote
0answers
41 views

Effect of Gamma and C on distant points in SVM

(I am aware of the following, already answered questions: this,and this,as well as others, and IMHO they are not related) I am trying to precisely understand the behavior of SVM's with regards to the ...
2
votes
2answers
294 views

(Linear regression) Can I train and validate at the same time using the following approach?

In a lot of material I found online, training and validation seems to be an iterative process For example, the regularized regression problem $E = \|Xw - t\|_2^2 + \lambda \|w\|^2_2$ $X$ is data ...
0
votes
1answer
34 views

Stable model or overfitting?

I have a dataset of 150 patients (2:1 ratio of classes) and 78 features. I performed backwards elimination using logistic regression feature importance to end up with 13 features (SVC classifier). I ...
0
votes
0answers
56 views

Working of Dual Perceptron Algorithm

I was going for the theory and maths behind the online perceptron algorithm and it is very easy to under stand it intuitively that on a positive mistake, you just add the ...
1
vote
1answer
89 views

Is there an ExtraTreesClassifier-like classifier that has decision boundary function like SVM?

I'm using sklearn and I tested many models and those two worked best: Linear SVM and the ExtraTreesClassifier as binary classifiers. The ExtraTreesClassifier outperforms the Linear SVM in terms of ...
0
votes
0answers
21 views

NB or linear SVM for text Classification

I'm trying to compare the time complexity of the NB and the linear SVM when using them for text classification but can't find out a response to what to use considering that the number of features is ...
2
votes
1answer
70 views

can support vector regression be used in large data?

My aim is to predict a continous variable with a support vector regression. However, my dataset is large. My question is if it is possible to use SVR in this case.
1
vote
0answers
24 views

Question concerning SVMs in machine learning course CS229 by Andrew Ng

On page 12 in https://see.stanford.edu/materials/aimlcs229/cs229-notes3.pdf, the author uses the claim that the gradient of the lagrangian with respect to the non-constraint variables is zero. Why is ...
7
votes
1answer
167 views

Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss setup ...
1
vote
1answer
62 views

Regarding 0-1 loss and hinge loss functions of SVM

I would like to ask about SVM. I want to compare between 0-1 loss and hinge loss functions. My question is how to compare between them?!. Should we construct different SVM models, which each for ...
1
vote
0answers
44 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed RFE + stratified cross validation to reduce the features to 15 and I get an AUC of 0.9 with Logistic ...
0
votes
1answer
20 views

Identifying Feature Importance in Text

I am trying to perform feature interpretability on a text corpus but I am becoming quite confused as to how I identify the importance of particular features (words). I have done substantial research ...
0
votes
0answers
10 views

Question about support vectors for the general C-SVM problem

I understand that the support vectors must lie on the margin, within the margins, or lie on the wrong side of the margin (i.e, a point correctly classified outside the margin can't be an support ...
0
votes
0answers
23 views

Share price difference in SVM model

I have an SVM model which takes as input the difference (subtraction) of two values ​​(max and min) to solve a binary classification problem. These two values ​​are calculated as the minimum and ...
0
votes
0answers
17 views

When tuning an SVM what are reasonable bounds for C and gamma when performing a gridsearch?

I am trying to tune the hyperparamters of an RBF-kernel SVM by utilizing a gridsearch strategy. I found different sources stating different ranges (2^-15, ... 2^15 or 10^-3,...10^3) all they have in ...
0
votes
0answers
18 views

Effect of Bimodal distibution and mitigation while performing Classification

I am trying to solve a classification problem where one of my numerical attributes age is BiModal in nature. Will it cause any ...
0
votes
0answers
46 views

how to deal with time series data for svm multi-class classification problem

I am more used to data sets that looks like the breast cancer data set and iris flower data set, and am very unfamiliar with time series data sets. The problem is to classify signals according to the ...
0
votes
1answer
33 views

Is it possible to vary the slack of a Support Vector Machine (SVM), such that there is more slack on one side of the decision boundary than the other?

Take for example a case where you need to train a model to classify one scenario over another, where a false negative is much more costly than a false positive. Example: Credit Card Fraud Detection ...
1
vote
0answers
43 views

Can someone explain the prediction equation for support vector regression?

I am trying to implement my own SVR code from scratch to understand the theory in depth. Referring to Equation (19) in Smola, A Tutorial in support vector regression. Predictions are found by. Am i ...
1
vote
1answer
53 views

Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM)

I want to understand the criteria of selection of ML algorithms i.e what are the guidelines on which algorithm to be selected in which case ? The reasons I know are : Logistic regression to be ...
3
votes
3answers
249 views

SVM vs Logistic Regression [duplicate]

What are the pros and cons of logistic regression and SVM (support vector machines)?
1
vote
0answers
18 views

How to find initial feasible point for the dual problem of SVMs (Support Vector Machines)?

Problem Is there a systematic way of finding an initial feasible point for the dual problem of an SVM? Context I'm writing an implementation of Support Vector Machines for Binary Classification in ...
1
vote
1answer
63 views

About 10-fold cross validation train/test split

So, I want to do 10 fold CV. After I googled it, all of the websites I've found told me that to do the split, take 1 fold as test and the rest as train. But my professor told me another way. She told ...
0
votes
1answer
28 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
0
votes
0answers
42 views

Binary classification Task : Least Squares kernel regression(squared loss) Vs SVM (hinge loss)

In binary classification, the solution function, in order to fit the training data, it just needs to acquire values that have the same polarity as the desired values, rather than accurately acquiring ...
1
vote
0answers
37 views

Uniqueness of svm solution

How does one show that if the solution of the primal linear SVM is unique, then there exists a support vector whose corresponding slack variable is equal to $0$? I tried showing this through proof by ...
0
votes
0answers
8 views

Number of Kernels in LSSVM

I'm studying LSSVM algorithm at the moment, and find one nuance very strange in most all papers I've read so far. The input/desired output are usually described as a set of {Xi, Yi}, where i = 1 .. N,...
0
votes
0answers
43 views

How do I create a classification model to predict observations with different feature vector lengths?

Say I have a dataset containing hourly records of the vital signs of people trying to survive in the wild, their environmental conditions, and a label of whether the person survives. I would like to ...
1
vote
2answers
67 views

Feature scaling dramatically improves performance

I am working with "Forest Coverage Type" Kaggle dataset (https://www.kaggle.com/c/forest-cover-type-prediction/data) and have applied support vector machine classification to predict forest coverage ...
1
vote
1answer
93 views

What are w and b parameters in SVM?

I've read almost every article on the web, every question regarding SVM here, but I still don't get how to calculate w and b, how did they appear in formula, what is weight and what is bias: $$\vec{w}...
8
votes
1answer
347 views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
0answers
20 views

Choice of the learning algorithm for Recursive feature elimination

I have a dataset that I divide into 80% for training+test and 20% for validation I have been using Recursive feature elimination for feature selection with SVM on the 80% partition...
2
votes
2answers
45 views

Rain overflow modeling: Categorical variables or separated models?

I'm working on a project where I have to predict rain overflow due to rain for 5 sewer locations. I have a file which tells me if there is a rain overflow (=1) at a given date for a given sewer or no (...
0
votes
0answers
25 views

How does QSVM algorithm differs from SVM?

Where does the working of QSVM differs from classical SVM ? And how it is fast ? During which steps of the algorithm do they differ ?
0
votes
0answers
19 views

Maximizing payoff when a 'dont care' state is present

Consider the following: We have a training dataset ($y_i$, $\mathbf{x}_i$) $i \in [1,n]$ where $y_i \in \{-1, 1\}$. We can build any model (logistic / SVM / anything else) to predict $y_i$ given $\...
0
votes
0answers
58 views

ROC curve under diagonal?

I trained an SVM to classify images based on some extracted features (using the ISIC dataset). The resulting ROC curve produced by sklearn looks like this: I have don't quite understand the line for ...
1
vote
0answers
14 views

Consistency on SVM results

I have a conversation in my office regarding the results on SVM classification. As far as I understand it SVM does not contain any random initialization that could produce a different result on ...
-1
votes
1answer
127 views

Why am I getting accuracy of 100 percent using SVM

I am working on Credit card data set for fraud detection. When I apply SVM for it, I am getting the accuracy as 100 %. What might be going wrong here? Here is the code ...

1 2
3
4 5
41