Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Vectorised Implementation of SVM Gradient

I am trying to implement the SVM loss and gradient. The loss is given as $$L(w) = \sum_{i=1}^N max\{1-y_iw^tx_i, 0 \} + \lambda ||w^2||_2^2$$ I believe that for the loss, this is a good implementation;...
0
votes
1answer
30 views

Why is One Class SVM predicting that half my dataset consists of outliers?

I am currently working on a dataset with 14 continuous features, a categorical target over five classes, and 90,000 samples. My current goal is to explore outliers in the dataset, and to that end I ...
0
votes
0answers
23 views

Plot of Decision Boundaries intuition for different SVMs

I have a class imbalanced dataset and I used two different SVMs for binary classification. One plain SVM and one class weighted(i overweighted the positive class). Below are the decision regions I ...
0
votes
0answers
16 views

How can I write dual problem for the given little dataset and kernel function? Then how can I write kernel matrix(inner product)? [closed]

Actually I know implementation of this problem on code, however I cannot solve it through mathematically, could anyone help me to solve this little problem? I need to get sense about dual problem ...
1
vote
1answer
25 views

Why minimizing the ||w|| penalty term in SVM definition makes the overfitting less probable?

I understand overfitting and why we want our classifier to be reasonably simple. If we introduce more complexity to the predictor, we are risking that we fit it too closely to data, thus overfit, and ...
0
votes
0answers
45 views

Representer Theorem for Support Vector Regression

I would like to know what is the expression of the predictor function in terms of the Representer Theorem in the case of Support Vector Regression. For example, in the SVM binary classification case, ...
3
votes
1answer
61 views

In this example, which of these vectors are support vectors?

The hyperplane of hard margin SVM with $\phi$ kernel is calculated as following that input space using $\phi$ to map to higher dimension space. $$f(\phi(x))=4\phi_1(x)+9\phi_2(x)+4\phi_3(x)$$ $$ \phi(...
2
votes
1answer
63 views

misclassified data in SVM

In soft margin SVM with Gaussian kernel, How will the decision boundary change, if $ \sigma $ is increased? I means what will be happened in this case. i.e: $(1)$ Dose the number of misclassified ...
0
votes
0answers
53 views

On Support Vector Regression, given the $\vec{\alpha}$ and $\vec{\alpha}^*$, how $b$ (the bias term of the predictor expression) is computed?

It is not clear to me how the bias term $b$, given $\vec{\alpha}$ and $\vec{\alpha}^*$, is computed on Support Vector Regression, after each iteration of an optimizer.
1
vote
0answers
37 views

Why do people say linear SVM is a parametrized model and kernel SVM is not?

I am confused by what people mean when they say that linear SVM is a parametrized model and kernel SVM is not. Aren't both methods trying to compute a decision boundary using some optimization program?...
0
votes
0answers
22 views

is scaling in SVM always necessary?

I've already read some posts here about SVM scaling and why it is so important. Now I'm wondering if scaling in the ksvm function is always necessary if my data are ...
1
vote
0answers
28 views

How does the python port of libsvm's predict_proba work?

I've followed through the original libsvm code on it's [github][1]. I'm not concerned about the theoretical backfground of how the probability estimates are derived. All that I care about is how to ...
2
votes
1answer
135 views

How does the support vector machine constraint imply that sample selection bias will not systematically affect the output of the optimisation?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.4. Support vector machines, the author says the following: 3.4. ...
2
votes
1answer
42 views

What will happen if the variance of Gaussian kernel in Gaussian kernel SVM decreases?

I prepare my self for ML exam ad I saw this question. My note tells me that the option (3) is correct, but maybe there is a trick because all of sentences look like true. (it means all of them seem ...
1
vote
1answer
23 views

How did $y_i$ go missing in the final equation?

I am following this video on Support Vector Machines. Could someone explain to me how $y_i$ go missing in the final equation? Much appreciated.
1
vote
3answers
34 views

Train an SVM with only a single example per class?

Suppose I am doing multi-class classification (for example on MNIST), but I only give a single labeled example of each class. So like the training set has only a single 0, a single 1, a single 2, and ...
2
votes
0answers
59 views

Why SVM with gamma='scale' for RBF kernel works so well?

The intuitive explanation for the gamma parameter of the RBF kernel in SVMs is the following: Intuitively, the gamma parameter ...
2
votes
0answers
10 views

Is there any relationship between an SVM classifier performance and the distribution of the dataset?

Do the accuracy and performance of an SVM classifier have a relationship with the distribution of a dataset? For instance if the dataset is distributed uniformly, how does it affect the performance of ...
1
vote
1answer
31 views

Numerical example on Support Vector Machines

I was watching this numerical problem on SVM link here. At 4.42 he wrote this equation: where $\tilde{s_1}, \tilde{s_2}, \tilde{s_3}$ are support vectors. But from the support vector we see that: $$ ...
0
votes
0answers
26 views

High training and testing accuracy when training a binary classification model

I have a dataset where I split the train/test set to 66%/33%. I noticed my training accuracy is very high (in the 99s) regardless of which classification model algorithm. I also noticed while making ...
0
votes
1answer
15 views

Non-negative Constraints in Soft-Margin SVM Lagrange Equation

I was reading the A Tutorial on Support Vector Machines for Pattern Recognition as a supplemental for my Intro to ML class and I wasn't sure why $ a_i \geq 0 $ and $ \lambda_i \geq 0 $ cannot be ...
1
vote
1answer
131 views

SVM loss function

I am going through Bishop's book and especially SVM. I am trying to understand the logic behind minimizing the specific loss $argmax_{\mathbf{w}} \frac{1}{2}||\mathbf{w}||^{2}$. On page 327, in 7.3 we ...
0
votes
0answers
44 views

How to manually calculate predictions of kernlabs SVM

I am trying to manually replicate the predictions of kernlabs SVM (polynomial & radial kernel) using caret. Here is the code to fit the model: ...
0
votes
2answers
42 views

SVM Model: What's a healthy number of support vectors?

For a SVM model what is a healthy number of support vectors? or more precisely what's a good ratio of number of support vectors to the total number of training samples, 10%, 20%, 30%, 50% ... 80%? Is ...
0
votes
0answers
14 views

Why minimize radius in support vector clustering

I have recently started studying machine learning on my own. I am reading support vector machines and then support vector clustering. https://papers.nips.cc/paper/2000/file/...
0
votes
0answers
23 views

Are radial basis kernels able to model interactions between predictors?

I have been doing research using Support Vector Regression for some time, especially using radial basis kernel, for predicting a response variable from a set of numeric predictors. As a consequence of ...
1
vote
1answer
23 views

SVM: Would we care about the functional margin if maximizing only with geometric margin were convex?

I am reading Andrew Ng's SVM notes (https://see.stanford.edu/materials/aimlcs229/cs229-notes3.pdf) and am lacking the intuition for why we need the functional margin. As far as I understand we need it ...
7
votes
2answers
376 views

How support vectors is calculated on SVM example?

Question: If we map input data with following $\phi$ function to higher dimension via Hard Margin SVM then support vectors are $a$ and $b$. How we can find support vectors of this example, i.e: ...
2
votes
3answers
47 views

What's the speed bottleneck in sklearn.svm.SVC.predict?

I'm working with some high resolution images of specimens in test tubes and I found that using an SVC to classify each pixel by HSV value helps me to a great job at segmenting out just the specimen ...
0
votes
0answers
8 views

AUC plot from a MLSeq::classify object

I have generated a classify object using the MLSeq::classify function. I wonder how I can visualise this using a ROC or AUC curve with sensitivity and specificity on the axis. ...
0
votes
0answers
21 views

How to find 95% CI of a matrix of classification data?

I am running some support vector machine (SVM) analysis. I can run the analysis and even plot the obtained hyperplane, with methods similar to what is reported here. Essentially, I create a large ...
1
vote
0answers
60 views

How to differentiate the hinge loss?

I'm asked to differentiate the following hinge loss term. $$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$ where $$ L\left( w^{j'};\...
1
vote
0answers
22 views

SVM: decrease/increase of geometric margin after feature transformation

Consider two labeled points $(x = 1, y = 1)$ and $(x = 3, y = -1)$. Is the resulting geometric margin we attain in the feature space using feature vectors $\phi(x)=[x, x^2]^T$ greater then the ...
0
votes
0answers
22 views

Interpreting a precision recall curve

I am working on my ms thesis about predicting skin cancer I have these PR curves 1 - They start from 1.0 but they don't end at 1.0, is this a problem? 2 - SVM falls from 1.0 to 0.0 immediately and ...
0
votes
0answers
16 views

Which SVM kernel (or classifier) to use when there is a structured covariance among the features?

I am trying to use SVM for multi-class classification. The input features are assumed to be generated from a multi-variate Gaussian distribution. Each class corresponds to a different set of mean ...
1
vote
1answer
53 views

Minimization of the loss function in soft-margin SVM

According to Wikipedia, the goal of the soft-margin SVM is to minize the hinge loss function: $$\left[\frac{1}{n} \sum_{i=1}^{n} \max \left(0,1-y_{i}\left(\vec{w} \cdot \vec{x}_{i}-b\right)\right)\...
0
votes
0answers
12 views

What is the way to compare SVM output?

How can I rigorously compare the results from SVM? I have a feature matrix that contains ~1000 features and the label is either 1 or 0. The features can be grouped into 4 categories, let's say they ...
0
votes
0answers
48 views

How to use the kernel trick on a XOR-like dataset

Let's say that I have the following data: I want to find a transformation of this dataset that will make it linearly separable. My thought was to bring the data around the origin and then multiply $...
0
votes
1answer
38 views

Need help understanding how C hyperparameter influences $w $ in regularized SVM

I am following Andrew Ng's lecture notes on SVM from CS229. What I am having trouble understanding is the new objective function. $min_{γ,w,b} \frac{1}{2} ||w||^2 + C\sum_{i = 1}^{m}ξ_i$ From what I ...
1
vote
1answer
32 views

How do we come up with the SVM Kernel giving $n+d\choose d$ feature space?

I was going through the CS229 notes on SVM and Kernel tricks and I came across the following line. More generally the kernel $K(x,z)=(xTz+c)^d$ corresponds to a feature mapping to an $n+d\choose d$ ...
0
votes
0answers
12 views

What's the best strategy to fill NAs for a predictor in supervised learning e.g. SVM?

What's the best strategy to fill NAs for a predictor in supervised learning e.g. SVM? I have monthly data for all other predictors since 1963 and for one predictor I have data since 1990 only. So I ...
0
votes
0answers
35 views

Training/test splits (Monte Carlo sensitivity analysis) or Cross-validation

I am using SVM in Matlab (fitcsvm function) to train a classifier for a problem with two classes. Further, I have three features, e.g. A1, A2 and A3, available for each observation composing my full ...
1
vote
1answer
12 views

SVM classification metrics are all 1 although there are mistakes in classification

Here is a fitted LinearSVC model showing the learned separating hyperplane for my training samples: And when I use ...
0
votes
0answers
39 views

How to determine equation of hyperplane for SVM?

Assume we have only two features in our training dataset that is already classified into class C1 and class C2. The transposes of the feature vectors are given below for each class: C1: [2 6], [1 1], [...
0
votes
0answers
19 views

SVM predicts always the same class

I have a dataset with tf-idf values and their corresponding classes and I am trying to do predictions using SVM. The problem is that all the results that it produces have the same class. Most related ...
1
vote
1answer
33 views

Testing for causality with Support Vector Machines

Can a support vector machine (SVM) be used to test for causality between 2 or more variables? I know that the original purpose for SVM is classification. I also know that there is a variation of the ...
0
votes
1answer
60 views

Hard-margin SVM and logistic regression for non-linearly separable data

Hard-margin SVM doesn't seem to work on non-linearly separable data. It seems to only work if your data is linearly separable. What happens if you try to use hard-margin SVM? Does the algorithm blow-...
0
votes
1answer
28 views

How can I extract the correct hyper-plane from sklearn.svm's LinearSVC

I'm not certain I understand how sklearn's Linear SVC works. I had assumed that it would find an optimal hyper-plane to divide one class from another. I tried to recover the separating hyper-plane ...
2
votes
1answer
32 views

SVR with combination of kernels

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of ...

1
2 3 4 5
42