Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
1
vote
0answers
19 views

Question concerning SVMs in machine learning course CS229 by Andrew Ng

On page 12 in https://see.stanford.edu/materials/aimlcs229/cs229-notes3.pdf, the author uses the claim that the gradient of the lagrangian with respect to the non-constraint variables is zero. Why is ...
3
votes
0answers
58 views
+50

Support Vector Machine with Perceptron Loss

Typical support vector classifier uses the following optimization procedure: $$\min ||w||^2 + C\sum_{i=1}^N \zeta_i$$ $$y_i(w^Tx_i+b) \geq 1 - \zeta_i$$ $$\zeta_i \geq 0$$ This hinge loss setup ...
1
vote
1answer
15 views

Regarding 0-1 loss and hinge loss functions of SVM

I would like to ask about SVM. I want to compare between 0-1 loss and hinge loss functions. My question is how to compare between them?!. Should we construct different SVM models, which each for ...
1
vote
0answers
38 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed RFE + stratified cross validation to reduce the features to 15 and I get an AUC of 0.9 with Logistic ...
0
votes
0answers
14 views

Identifying Feature Importance in Text

I am trying to perform feature interpretability on a text corpus but I am becoming quite confused as to how I identify the importance of particular features (words). I have done substantial research ...
0
votes
0answers
10 views

Question about support vectors for the general C-SVM problem

I understand that the support vectors must lie on the margin, within the margins, or lie on the wrong side of the margin (i.e, a point correctly classified outside the margin can't be an support ...
0
votes
0answers
22 views

Share price difference in SVM model

I have an SVM model which takes as input the difference (subtraction) of two values ​​(max and min) to solve a binary classification problem. These two values ​​are calculated as the minimum and ...
0
votes
0answers
10 views

When tuning an SVM what are reasonable bounds for C and gamma when performing a gridsearch?

I am trying to tune the hyperparamters of an RBF-kernel SVM by utilizing a gridsearch strategy. I found different sources stating different ranges (2^-15, ... 2^15 or 10^-3,...10^3) all they have in ...
0
votes
0answers
6 views

Effect of Bimodal distibution and mitigation while performing Classification

I am trying to solve a classification problem where one of my numerical attributes age is BiModal in nature. Will it cause any ...
0
votes
0answers
31 views

how to deal with time series data for svm multi-class classification problem

I am more used to data sets that looks like the breast cancer data set and iris flower data set, and am very unfamiliar with time series data sets. The problem is to classify signals according to the ...
0
votes
1answer
25 views

Is it possible to vary the slack of a Support Vector Machine (SVM), such that there is more slack on one side of the decision boundary than the other?

Take for example a case where you need to train a model to classify one scenario over another, where a false negative is much more costly than a false positive. Example: Credit Card Fraud Detection ...
1
vote
0answers
32 views

Can someone explain the prediction equation for support vector regression?

I am trying to implement my own SVR code from scratch to understand the theory in depth. Referring to Equation (19) in Smola, A Tutorial in support vector regression. Predictions are found by. Am i ...
1
vote
1answer
47 views

Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM)

I want to understand the criteria of selection of ML algorithms i.e what are the guidelines on which algorithm to be selected in which case ? The reasons I know are : Logistic regression to be ...
3
votes
3answers
87 views

SVM vs Logistic Regression [duplicate]

What are the pros and cons of logistic regression and SVM (support vector machines)?
1
vote
0answers
15 views

How to find initial feasible point for the dual problem of SVMs (Support Vector Machines)?

Problem Is there a systematic way of finding an initial feasible point for the dual problem of an SVM? Context I'm writing an implementation of Support Vector Machines for Binary Classification in ...
1
vote
1answer
51 views

About 10-fold cross validation train/test split

So, I want to do 10 fold CV. After I googled it, all of the websites I've found told me that to do the split, take 1 fold as test and the rest as train. But my professor told me another way. She told ...
0
votes
1answer
20 views

Adaptive Gamma in RBF Kernel

The RBF Kernel is defined by $K(x,y)=\exp(-\gamma ||x-y||^2)$ Wouldnt it be better to find a suited gamma value for each dimension? $K(x,y)=\exp(-\sum_i \gamma_i * (x_i-y_i)^2 )$ This would add ...
0
votes
0answers
13 views

Binary classification Task : Least Squares kernel regression(squared loss) Vs SVM (hinge loss)

In binary classification, the solution function, in order to fit the training data, it just needs to acquire values that have the same polarity as the desired values, rather than accurately acquiring ...
1
vote
0answers
18 views

Uniqueness of svm solution

How does one show that if the solution of the primal linear SVM is unique, then there exists a support vector whose corresponding slack variable is equal to $0$? I tried showing this through proof by ...
0
votes
0answers
7 views

Number of Kernels in LSSVM

I'm studying LSSVM algorithm at the moment, and find one nuance very strange in most all papers I've read so far. The input/desired output are usually described as a set of {Xi, Yi}, where i = 1 .. N,...
0
votes
0answers
40 views

How do I create a classification model to predict observations with different feature vector lengths?

Say I have a dataset containing hourly records of the vital signs of people trying to survive in the wild, their environmental conditions, and a label of whether the person survives. I would like to ...
1
vote
2answers
55 views

Feature scaling dramatically improves performance

I am working with "Forest Coverage Type" Kaggle dataset (https://www.kaggle.com/c/forest-cover-type-prediction/data) and have applied support vector machine classification to predict forest coverage ...
0
votes
1answer
45 views

What are w and b parameters in SVM?

I've read almost every article on the web, every question regarding SVM here, but I still don't get how to calculate w and b, how did they appear in formula, what is weight and what is bias: $$\vec{w}...
7
votes
1answer
134 views

Why are random Fourier features efficient?

I am trying to understand Random Features for Large-Scale Kernel Machines. In particular, I don't follow the following logic: kernel methods can be viewed as optimizing the coefficients in a weighted ...
1
vote
0answers
14 views

Choice of the learning algorithm for Recursive feature elimination

I have a dataset that I divide into 80% for training+test and 20% for validation I have been using Recursive feature elimination for feature selection with SVM on the 80% partition...
2
votes
2answers
42 views

Rain overflow modeling: Categorical variables or separated models?

I'm working on a project where I have to predict rain overflow due to rain for 5 sewer locations. I have a file which tells me if there is a rain overflow (=1) at a given date for a given sewer or no (...
0
votes
0answers
17 views

How does QSVM algorithm differs from SVM?

Where does the working of QSVM differs from classical SVM ? And how it is fast ? During which steps of the algorithm do they differ ?
0
votes
0answers
19 views

Maximizing payoff when a 'dont care' state is present

Consider the following: We have a training dataset ($y_i$, $\mathbf{x}_i$) $i \in [1,n]$ where $y_i \in \{-1, 1\}$. We can build any model (logistic / SVM / anything else) to predict $y_i$ given $\...
0
votes
0answers
32 views

ROC curve under diagonal?

I trained an SVM to classify images based on some extracted features (using the ISIC dataset). The resulting ROC curve produced by sklearn looks like this: I have don't quite understand the line for ...
1
vote
0answers
11 views

Consistency on SVM results

I have a conversation in my office regarding the results on SVM classification. As far as I understand it SVM does not contain any random initialization that could produce a different result on ...
-1
votes
1answer
81 views

Why am I getting accuracy of 100 percent using SVM

I am working on Credit card data set for fraud detection. When I apply SVM for it, I am getting the accuracy as 100 %. What might be going wrong here? Here is the code ...
0
votes
0answers
13 views

Preprocessing during a Kfold cross validation

I have noticed from various sites online that preprocessing and feature selection when dealing with Kfold Cross Validation suggest that the preprocessing and feature selection should be done on the ...
0
votes
0answers
17 views

Tuning SVM when extracting features from DCT?

What will be the parameters for "SVM" classifier when extracting features from Discrete Cosine Transform (DCT) and classifiying features using SVM considering having 200+ classes?. What should be a "...
13
votes
1answer
233 views

What is the support vector machine?

What IS the support vector machine? Can someone clarify my confusion? Possible answers: The SVM is the problem: given data $(x_n, y_n), n = 1, \ldots, N$ $$\min_{w, b}\frac{1}{2}||w||^2$$ $$\text{ ...
0
votes
0answers
8 views

What statistical classifiers can use unlabelled data to enhance their performance similar to a transductive support vector machine?

I was wondering if statistical machine learning methods like tree based methods, ANNs, logistic regression can make use of unlabelled data to enhance their performance similar to the way a ...
0
votes
0answers
23 views

What is the role of C hyperparamter in SVM? [duplicate]

Why increasing C hyperparamter the margin gets smaller?
1
vote
1answer
29 views

Why does the formulation of the SVM problem has the bias (something we try to optimize) as a part of the constraint?

The common formulation of the SVM problem is $$\min_{\theta, \theta_0}\frac{1}{2}||\theta||^2$$ $$\text{ subject to: } y^{(t)}(\theta \cdot x^{(t)} + \theta_0) \geq 1, \ t=1,...,n,$$ However, it ...
1
vote
1answer
24 views

Is SVM RBF applied to both classes?

Lets say i have following 1D data (position on x), color is target class and I need a classifier which classifies green from red: I decided to use SVM. Data is clearly not linearly separable, so i ...
1
vote
0answers
39 views

Gaussian RBF vs KNN explanation

I was studying SVM ML alghorythm and I was wondering about solution for non-linear cases. As I understand it for know, SVM tries to find hyperplane or object in defined n-dimensional space, which ...
1
vote
0answers
13 views

Plot Actual vs Predicted SVM Regression

I am building an SVM regression model using caret package, however, I am not sure what is the best approach to plot predicted vs actual values. I have the code below. You can reproduce the output by ...
0
votes
0answers
17 views

Compute Bias Term in Support Vector Regression (SVR)

I'm trying to figure out how the bias term in SVR is computed. I've already optimized the alphas, so I have a Nx1 vector for a_i and a_i*. Furthermore, I have the predictor matrix X which is Nx2 and a ...
0
votes
0answers
10 views

Does reducing dimensionality of data makes it less linearly separable?

I recently read about kernel trick in SVM that says that mapping data to higher dimensions makes it more linearly separable but can we conversely say that "mapping ...
0
votes
0answers
13 views

does the resolution of the vectors influence the result?

I have setup a model for prediction of free parking places. Thi model based on SVR (support vector regression) and sklearn libs. My feature vectors are: time as float (e.g. 8.00, 8.25, 8.5, 8.75, 9.0....
1
vote
0answers
15 views

Machine learning model giving different probability on different scenarios [closed]

I am working on Iris dataset classification model. I am using SVM model and getting the probability of each class using predict_proba function. I tried the same example on different mediums ...
0
votes
0answers
23 views

Separating Hyperpanes - The Elements of Statistical Learning

The authors say that the green line depicts a hyperplane or affine set $L$ defined by the equation $f(x) = \beta_0+\beta^Tx = 0$; since we are in $R^2$ this is a line. One property is: For any two ...
0
votes
0answers
9 views

Question about one example in Burges' SVM tutorial

On page 127 of Burges' SVM tutorial paper [1]. He listed an example to validate the bound for true error rate. However, it seems to me that the left side of equation (8) $\frac{m}{4l}\leq \ln(2l/m)+...
0
votes
1answer
46 views

Is the kernel trick unnecessary for for non-linear SVM?

I am just learning about Mercer Kernels, and a question came up. Since using Mercer's theorem, we know that a positive definite kernel matrix can be represented by an inner production of the input ...
1
vote
0answers
54 views

feature importance for SVM with a nonlinear kernel

Using sklearn, I did SVR using rbf kernel. Though I got good results, problem is I don't know how to get the important feature that the algorithm used. Also coef_ ...
2
votes
0answers
33 views

In semi-supervies learning, is “low density separation” the same thing as “pseudo-labelling”?

I'm looking into the different methods of semi-supervised learning. In the wikipedia page, one of the methods described is called "low-density separation", where we attempt to minimize this loss ...
0
votes
1answer
33 views

Object classification

I'm currently working on a "Where's Waldo" project as part of my coursework, where I have to find 3 different characters in any given image - Waldo, Wenda, and Wizard. I'm trying to convert this ...