Questions tagged [svm]
Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."
2,282
questions
0
votes
1
answer
25
views
Maxima in the dual of hard margin and soft margin SVMs?
The dual problem for hard margin SVM is:
\begin{align*}
&\max_{\alpha} \left( \sum_{i=1}^{N} \alpha_i - \frac{1}{2} \sum_{i=1}^{N} \sum_{j=1}^{N} \alpha_i \alpha_j y^{(i)} y^{(j)} \langle x^{(i)}, ...
0
votes
0
answers
30
views
SVM kernels corresponding to different types of distance measures
This answer to Data normalization for RBF kernel points out that RBF kernel implies Eucledean distance. Are there kernels corresponding to other popular distance/dissimilarity measures, such as Bray-...
- 3,512
2
votes
0
answers
7
views
How to prove that 2d support vectors are enough for Hard Margin Linear SVM?
As the question states, how can I prove mathematically that 2d support vectors are enough to always be able to formulate the Maximum Margin Hyperplane in d dimensions?
- 21
2
votes
1
answer
40
views
In case of no correlation, can a model make predictions above the expected values?
For simplicity's sake, let's suppose a binary classification problem, with a perfect 50% of probability for each of the classes, and a SkLearn's SVC model.
Let's ...
0
votes
0
answers
23
views
Why use Kernel trick if soft margin SVM works for non-linearly separable data? [duplicate]
Most articles and textbooks say that soft margin SVM is used as the data is messy/not linearly separable. We introduce slack variables to make the data linearly separable. Kernels are used when the ...
- 1,245
0
votes
1
answer
18
views
The distance from the hyperplane to the points
In Support Vector Machines, the distance from the hyperplane to each class of nearest points should have the same length.
Is this correct?
0
votes
1
answer
50
views
Performing a classification if having categorial labels and a distance matrix
I encountered a multi-class classification problem and I wonder which model would work the best in my scenario. I have around 50,000 vectors (each of size 200) with corresponding categorical labels ...
0
votes
0
answers
10
views
Kernel + Mutliple SVM's + Platt Scaling = 1 layer neural network?
I have built my own Support Vector Machine by using quadratic programming and I'm using Kernel PCA with SVM. The output is tanh e.g Platt scaling.
When I combinde ...
- 349
0
votes
0
answers
5
views
Which method should be used to determine the class ID of multiple SVM models?
I'm using Support Vector Machine(SVM) with image classification.
Each SVM model results a linear model
$$y = wx + b$$
Where $w$ and $b$ is the SVM parameters.
If I have multiple SVM models, I will get ...
- 349
1
vote
0
answers
15
views
Can multiple Support Vector Machine models achieve the same accuracy as one deep neural network?
Assume that we have one deep neural network with an input for images. That deep neural network can classify images.
On the other side, we have multiple Support Vector Machines that classify parts of ...
- 349
4
votes
2
answers
106
views
Scikitlearn: Why are hyperplane coefficients not available if kernel is not linear
I am interested in learning the math behind support vector machines. So far, I understand that SVMs attempt to find hyperplanes that maximize the margin distance between support vectors associated ...
- 41
1
vote
0
answers
75
views
Geometric intuition of kernel trick
I would like to understand better the geometry underlying the Kernel trick with the Gaussian Kernel. In particular my question is:
How the Kernel trick can be interpreted geometrically, in particular ...
- 840
0
votes
1
answer
58
views
Determine 'w' and 'b' in hard margin SVM
I have been asked the following question related to SVM (Hard Margin) in the exam, and I failed to answer it. Can anyone help me find the solution?
Consider the dataset M:
\begin{align*}
& \left(\...
0
votes
1
answer
20
views
Result after applying kernel trick
I understand when the data is not linearly separable, it has to transformed into higher dimensional space, to make it linearly separable. Applying kernel trick can perform it without even computing ...
1
vote
0
answers
50
views
How to solve alphas (or the dual equation) after getting Lagrangian dual of SVM
I'm trying to learn SVM by myself, and I'm stuck after getting the dual of SVM. I understand getting the dual after the primal. But, I am stuck here. Please help.
We assume that the hard margin case ...
- 11
3
votes
1
answer
169
views
Can SVM and Decision Trees be seen as instances of neural networks?
We already know that neural networks with specific choices of activation function as well as connections can generalize large amount of ML models.
My question is: neural network also generalize SVM ...
- 1,373
1
vote
0
answers
46
views
Achieve 99% accuracy with svm
I was going through this paper: https://www.mdpi.com/2227-7080/9/3/52, it compares different data scaling methods on different algorithms to see how performance is impacted.
using this database: https:...
- 23
0
votes
0
answers
17
views
SVM with too many observations, any library or solution?
In R Studio I have a database with 77,000 rows and 50 columns, I divide the observations into train and test, and the train table is left with 59,000 observations, I am making the SVM model, I am now ...
- 13
1
vote
0
answers
10
views
Is it necessary to choose predictors for svm or I use all my variables?
I have transformed my categorical variables to dummies and I have used the lasso method to decide which variables I choose to do the logistic regression, my question is: for the svm model do I need to ...
- 13
0
votes
1
answer
54
views
would One class svm train on only normal data or normal-outlier data both
I was going through this scikit-learn link and I noticed, OneClassSVM is trained on normal and outlier both. Specifically, they are adding the outliers in the following line:
...
- 101
0
votes
0
answers
11
views
SVM and logistic regression: steps and selection of predictors
I need help, I am a statistics student and I am doing my final degree project and I am a bit lost, the steps I have taken to make the svm and logistic regression models have been:
univariate analysis,...
- 13
2
votes
1
answer
72
views
Question about Platt Scaling in sklearn's implementation of SVM
From sklearn's documentation here:
The decision_function method of SVC and NuSVC gives per-class scores for each sample (or a single score per sample in the binary case). When the constructor option ...
- 1,045
0
votes
0
answers
14
views
Why isn't weights normalization required in SVM?
As I understand :
Distance from $x_0$ to the hyperplane $a$: $ \rho(x_0, a) = \frac{|<w, x_0> + b|}{||w||}$
We require $ min\ |<w, x_0> + b| = 1$, so $min \frac{|<w, x_0> + b|}{||w||...
- 1
0
votes
0
answers
23
views
What means 'scale' hyperparameter in SVM Polinomial (svmPoly method in caret)?
I have searched in many places, and can't find what the hyperparameter 'scale' means in SVM polynomial in caret (method = svmPoly)
Caould you tell me, please?
- 219
0
votes
0
answers
9
views
Short linear basis expansion for support vector regression
In support vector regression (SVR), I know that many different linear basis expansions for the predictors can be used. I am interested in a basis expansion that is as algebraically short as possible ...
- 153
1
vote
0
answers
21
views
Doubt in Linear SVM solving with mathematical equations
I am following the paper https://axon.cs.byu.edu/Dan/678/miscellaneous/SVM.example.pdf and trying to solve the SVM on paper using mathematical equations.
When I try to solve them, I get different set ...
- 11
3
votes
1
answer
84
views
Finding support vectors by hand for SVM
The problem I am trying to solve goes like this:
Consider the points $(1, 1)$, $(2, 2)$ and $(3, 3)$, of class $1$, and the points $(4, 5)$, $(5, 7)$, $(6, 5)$ and $(7, 7)$, of class $-1$. Find the ...
- 133
0
votes
0
answers
33
views
Is it ok to normalize data using minmaxscalar on dependent variable?
I'm trying to make a sales prediction using the column X = item_amount and y = item_price_total, I'm confused whether it's okay to normalize data on the dependent variable using minmaxscalar?
With the ...
- 1
4
votes
1
answer
197
views
What is exact code used for SVM in sklearn library? What is its criterion to stop training?
I am learning SVM. I have 2 questions in mind. First, what criteria use SVM to stop training? when I search for SVM tutorials, I found some. But, in all of them, training is done for example for 1000 ...
- 67
1
vote
0
answers
20
views
Why is there only a box constraint on alpha and not on mu when solving the dual problem of soft linear SVM?
I am currently learning about the linear SVM in the non-separable case. In the dual representation, we introduce the Lagrange multipliers μk and αk (see also this source: https://...
1
vote
0
answers
16
views
Are solutions to the Lagrangian multipliers ($\alpha_i$) in a hard-margin SVM unique?
An intermediate step in the derivation of the hard-margin SVM's dual form is as follows:
I also know that $a_i$ for all points not on the margin boundary is 0, which makes sense; they must be zeroed ...
1
vote
0
answers
17
views
Is it possible for no points to lie on the "gutters" in a hard-margin SVM?
I am taught that the hard-margin SVM is defined as the optimization problem:
$\min_{w, b} \frac{1}{2}||w||^2_2, \text{ s.t. } y_i(\langle x_i, w \rangle + b) \ge 1$
Furthermore, the two planes $\...
0
votes
0
answers
5
views
Where can I find documentation or projects related to Clustered Z-score Least Square Support Vector Method?
I have read about CZLSSVM (Clustered Z-score Least Square Support Vector Method) being better than kNN for missing value imputation but could not find projects on this method. Do you guys have ...
0
votes
0
answers
19
views
How to calculate the optimal weights and bias in SVM (by hand)
I've been trying to solve the following exercise:
-> Consider a dataset with two points in 1D: (x1 = 0, y1 = −1) and (x2 = √2, y2 = 1).
Consider also the mapping to 3D φ(x) = [1, √2x, x2].
a) Find ...
- 377
0
votes
1
answer
32
views
Does C parameter in Linear SVM guarantee the margin does not increase?
From my understanding a large value of the C parameter in SVM decreases the number of misclassified instances and narrows the margin. My question is - considering that SVM with linear kernel is used, ...
1
vote
1
answer
45
views
Upper bound on classification performance
Given a set of 128x128 images from three classes, I obtained an accuracy of 50% with a SVM on the flattened images (16384 'features').
Is this an upper bound on the performance of a SVM using any ...
- 111
0
votes
0
answers
14
views
Cleaning Data Before SVM
I want to classify diabetic retinopathy grades using SVM. I have 32 extracted features, and those features won't all be used in classification stage. Before entering feature selection, I want to clean ...
- 21
1
vote
1
answer
21
views
SVM margin equations
The equation of a margin in SVM is $\frac{2}{\lVert \mathbf{w} \rVert}$, which I completely understand.
It is also rewritten as $\frac{1}{2} \lVert \mathbf{w} \rVert^2$ for the sake of mathematical ...
- 13
0
votes
0
answers
19
views
AUC for Multi-Label Classification using SVM
I am tackling a multi-label classification problem and I want to choose a SVM model maximising the AUC. I am not sure if AUC can be used in this case and if yes it is sufficient just to change the ...
0
votes
0
answers
7
views
Small number of weights in the SVM separator function
The separator function in SVM is:
$f(\mathbf{x})=\sum\limits_{\mathbf{x_i} \in Support}\alpha_i\times K(\mathbf{x}, \mathbf{x_i})-b$
Depending on the kernel, this may correspond to adding any number ...
- 133
0
votes
1
answer
13
views
How do you find the dual form of the class weighted soft-margin SVM?
I am familiar with the dual form of the soft margin SVM when there is only one parameter $C,$ but I cannot find the dual form of the class-weighted soft margin SVM which has the following objective ...
- 1
0
votes
0
answers
6
views
VC dimension of an SVM with Radial basis function as kernel [duplicate]
How is the VC dimension of an SVM with Radial basis function as kernel bounded although it is projected to an infinite dimension?
0
votes
1
answer
25
views
Multilabel Classification Task using SVM
I want to classify diabetic retinopathy grades (normal, mild, moderate, severe, PDR) using SVM.
But the problem is i don't know which type of svm should i use, because i extract three lession features ...
- 21
0
votes
0
answers
12
views
Will Support Vector Machine always return linear weights and bias?
This code create the linear bondary between two classes.
...
- 349
1
vote
1
answer
113
views
An OCR model that can be easily improved on the user side
I am building OCR software, for this purpose I trained a model on many types of fonts, the model is SVM but it is not principled.
Now I want the users of the ...
- 71
2
votes
0
answers
32
views
Repeating Target Values in output [closed]
I have created a multiclass predictive model, however, the target values are repeating. Why one of the target value is repeating and how can it removed?
The code for this is as follows:
...
- 21
0
votes
0
answers
12
views
Inconsistencies with OneClassSVM model training
In the literature, for a binary classification problem, I have come across examples where a One-ClassSVM model is trained using the data for only one of the two training labels and sometimes using ...
- 123
0
votes
0
answers
43
views
Time Complexity of using the kernel trick on polynomial basis expansion
I have read that:
If we have two feature vectors ${x = (x_1,x_2,…,x_D)}$ and $y=(y_1,y_2,…,y_D)$ and we do a degree d polynomial basis expansion to get $f(x)$ and $f(y)$, then to calculate the inner ...
- 1
1
vote
0
answers
34
views
How does quadratic programming solve the Support Vector Machine problem?
I have just been reading that Quadratic Programming can be used to solve the Support Vector Machine optimization.
My solver can minimize this typ of problem
$$\text{J}_{min} = \frac{1}{2}x^TQx + c^Tx$$...
- 349
1
vote
0
answers
12
views
Stardardization for Random Forest, SVM and Logistic Regression
I have a classification project and I want to compare three models: Random Forest, SVM and Logistic Regression.
Random Forest are tree based algorithms wheras, SVM is a distance based model and LR is ...
- 11