Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

The derivation of the RBF kernel to the inner product form, and what does this notation $\sum _{n_{1}+n_{2}+\dots +n_{k}=j}$ mean?

I have two questions regarding the following derivation: What does this notation $\sum _{n_{1}+n_{2}+\dots +n_{k}=j}$ mean in the following equation? How is it derived from step 3 to step 4, and from ...
user avatar
-1 votes
0 answers
5 views

SVM decision boundary and resolving process with given simple value (1,1), (2,2) by hand [duplicate]

I'm beginner of SVM and have difficulty understanding SVM mechanism. Anybody help me explaning svm process with simple value (1,1), (2,2) and decision boundary?
user avatar
1 vote
0 answers
14 views

Why am I getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression?

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
17 views

Why is the model getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression? [closed]

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
0 votes
0 answers
8 views

SVM working with simple value [duplicate]

I'm beginner of SVM and have difficulty understanding SVM mechanism. Anybody help me explaning svm process with simple value (1,1), (2,2) and decision boundary?
user avatar
0 votes
0 answers
7 views

Fuzzy membership - Kernel SVM in R

I'm trying to perform a binary classification task using SVM with radial basis kernel in R and I want to assign fuzzy memberships to the datapoints. Already existing function in R packages such as ...
user avatar
  • 1
1 vote
1 answer
26 views

Support vector machine, complementary slackness and marginal hyperplane

One of the complementary slackness conditions for a support vector machine states that $$\alpha_i ( y_i (\langle w, x_i \rangle + b ) -1 ) = 0,$$ where $\alpha_i$ is the lagrange variable. One can ...
user avatar
0 votes
0 answers
15 views

What kernel to use for image classification from pre-trained CNN feature extractor

Suppose I have a pre-trained CNN feature extractor and I connect those to a soft margin SVM, what is the recommended kernel to use to replace $x_n^Tx$ in SVM? My dataset comprises of pictures of ...
user avatar
  • 287
0 votes
0 answers
15 views

Why does SVM always find separating hyperplane in p>>N and avoid overfitting?

I have gene expression data with ~20,000 features and nearly 600 samples, with 5 classes of cancer. I used grid search with 5 fold CV to find the optimal kernel and regularization for SVM, and I found ...
user avatar
10 votes
1 answer
795 views

Averaging SVM and GLM results: sensible or stupid?

I have taken two different approaches to calculate probability: using a GLM and an SVM. They are giving slightly different results (which is understandable, they are completely different approaches). ...
user avatar
0 votes
0 answers
36 views

One-class SVM with single dimension and polynomial kernel

Context: I'm studying anomaly detection without prior experience in machine learning, although I'm a senior web developer. This article talks about the kernel trick and gives this example with single ...
user avatar
0 votes
0 answers
8 views

Would logistic regression/support vector-machine with l-2 regularization and early stopping regularization cause underfitting?

Would early stopping regularization combined with l-2 regularization or in logistic regression/support vector machine cause underfitting? Does a kernel-trick affect what combination of regularization ...
user avatar
  • 224
0 votes
0 answers
32 views

How variable alpha changes SGDRegressor behavior for outlier?

I am using SGDRegressor with a constant learning rate and default loss function. I am curious to know how changing the alpha parameter in the function from 0.0001 to 100 will change regressor behavior....
user avatar
1 vote
0 answers
22 views

Confusion matrix error [closed]

I have been trying to do support vector machine classification in R and confusion matrix has been showing the error even after changing into factors confusionMatrix(as.factor(yTestPred), as.factor(...
user avatar
  • 11
0 votes
1 answer
31 views

Selection of informative examples of majority class for undersample using SVM

I have this idea in mind, but I am not sure how to implement it. Suppose I have an imbalanced data that I want to down-sample instances of majority class, such that it becomes equal in size to the ...
user avatar
  • 231
0 votes
0 answers
15 views

Trying to predict the next ignition off for a vehicle

Overview: I have a dataset that captures 6 months of data for 1000 cars. Each car is represented by it's unique identifier. The data captures the exact timestamp when the car was turned ON and turned ...
user avatar
  • 1
2 votes
0 answers
17 views

Is my use of Early Stopping appropriate in my model selection process?

My data is split into 80% train, 10% validation and 10% test. My dataset has three versions: i) Base dataset, ii) SMOTE adjusted version 1 and iii) SMOTE Adjusted version 2. I have run a 4 different ...
user avatar
  • 21
1 vote
0 answers
17 views

What kind of semi-supervised learning method should be used for a low quality data set?

Consider a binary classification problem, there are $1000$ samples in the data set, of which $500$ positive and negative samples each. Positive samples have the label $1$ and negative samples have the ...
user avatar
0 votes
1 answer
39 views

Why does linear SVM perform better than deep learning techniques on my problem?

I am doing a multiclass text classification problem. My data consists of tweets. I tried many variations of deep learning models (ex: LSTM, GRU and pre-trained word embeddings) and I also tried linear ...
user avatar
  • 1,240
0 votes
0 answers
30 views

SVM: How to derive the KKT condition with soft margin term is quadratic?

I was reading the Introdunction to Data Mining (2013) when I came across this in section 5.5: $$ f(\textbf{w})=\frac{||\textbf{w}||^2}{2}+C\left(\sum^N_{i=1}\xi_i \right)^k $$ I found similar ...
user avatar
4 votes
3 answers
682 views

Can correlation matrix be used as features in machine learning classification

Can I use correlation between the training data as features, and if possible how will I test the test data with the model coefficients I will try to explain more If the training data are ...
user avatar
6 votes
2 answers
192 views

Is there a Good Illustrative Example where the Hinge Loss (SVM) Gives a Higher Accuracy than the Logistic Loss

Vladimir Vapnik wrote: “When solving a problem of interest, do not solve a more general problem as an intermediate step. Try to get the answer that you really need but not a more general one.” ...
user avatar
0 votes
0 answers
23 views

Primal Problem of SVM

The primal problem of SVM is denoted as below. $$min_{w,b}\left(\phi \:\left(w\right)\right)=min_{w,b}\left(\frac{1}{2}w^Tw\right)$$ Subject to $$y_n\left(w^Tx_n+b\right)\ge 1,\:n=1,2,3,...,l$$ And If ...
user avatar
0 votes
0 answers
12 views

What's the effect of Heteroscedasticity in predictor variables for SVM classification?

It's usually a good practice as part of the modeling stage to apply transformations to all predictor variables so that they are stationary or in other words, that the statistical properties such as ...
user avatar
  • 875
4 votes
1 answer
359 views

How can I mathematically represent the one-hot encoding?

If we have 5 classes and 3 inputs, let's say [C1, C2, C3, C4, C5] and [X1, X2, X3] then, If ...
user avatar
1 vote
0 answers
48 views

How can I make sense of alpha values obtained from scikit-learn OC-SVM?

I am building a ML model that uses the OC-SVM for anomaly detection. For our cost function we require the alphas obtained from the OC-SVM. We use the OC-SVM of scikit-learn, which I assume is based on ...
user avatar
1 vote
1 answer
27 views

Learning-Agnostic Evaluation of SVM Models

I am at a point where I want to evaluate existing SVM models. For this task I assume I have: SVM model (to make it easier let's say it's a scikit-learn one) Training Dataset that was used to learn (1)...
user avatar
  • 11
1 vote
0 answers
87 views

Square Root of kernel function validity [closed]

If k1(x,z) and k2(x,z) are valid kernels, then is k(x,z) a valid kernel, where k(x,z) = sqrt(k1(x,z)k2(x,z)) Prove using mercers theorem
user avatar
  • 19
0 votes
0 answers
69 views

Proving validity of kernels

If k1(x,z) and k2(x,z) are valid kernels, then is k(x,z) a valid kernel, where k(x,z) = a1k1(x,z) - a2k2(x,z) (where a1, a2 > 0 are real numbers) I don't think this is true, but I am having trouble ...
user avatar
  • 19
1 vote
1 answer
26 views

SVC: why is at least one sample always exactly lying on the support vectors?

I don't really understand, why at least one sample is always exactly lying on the support vector(s). When assuming, that the dataset is exactly linearly seprable and considering the cost function of ...
user avatar
1 vote
0 answers
18 views

Predicting negative class rather than target in SVM [closed]

I am trying to classify a target group from controls using a SVM. I am predicting probabilities, and noticed that when predicting the target class, the SVM performance was horrible (AUC ~0.2). This ...
user avatar
  • 11
0 votes
0 answers
18 views

Effect of apparent correlation due to clustered data on performance of binary classifier

I am exploring possible features for a binary classifier, probably using a SVM, and have encountered an issue with correlation of features. I have chosen a number of features that may be of interest ...
user avatar
0 votes
0 answers
12 views

Significance Test for Linear Support Vector Machines [duplicate]

As we know, after applying a linear regression model to a continuous data set we carry out a significance test to each parameter βi associated with the predictor variable xi to check whether the ...
user avatar
0 votes
0 answers
45 views

Why are radial basis functions so different from classic inner product?

I was studying SVM with kernel tricks and it seems that the kernel is a modified dot product. A simple kernel would be $K(x,y) = <x,y>^2$. I understand how this is a modification of the dot ...
user avatar
  • 1
1 vote
0 answers
30 views

Is using SVC for detection problems is the right choice?

I have an NLP problem(fake news detection) and I used the code below, can I use svc for classification? and is my cross-validation accurate? ...
user avatar
0 votes
0 answers
8 views

Dummy coding with overlapping categories

Spotify has a structured dataset which contains song tracks with its associated audio features such as energy, speechiness (numeric data), whether there is explicit language in the lyrics (boolean), ...
user avatar
  • 1
0 votes
0 answers
23 views

ValueError: Requesting n-fold cross-validation but provided less than n examples for at least one class

I have been training a text classifier to then later use to predict characters of a TV show. So far, my code looks like: ...
user avatar
  • 1
0 votes
1 answer
28 views

Why is Scikit's Support Vector Classifier returning support vectors with decision scores outside [-1,1]? Is this a mistake?

I'm currently playing around with support vector machines in Scikit Learn and I've come across some unusual behaviour. For a basic simulated dataset, I've trained an SVC estimator (with linear kernel),...
user avatar
0 votes
0 answers
23 views

Why can the hyperdimensional plane be discribed as $\textbf{w} \cdot \textbf{x} - b$ for support vector machines

So given the picture and the related definitions from this answer: How does the equation $\textbf{w} \cdot \textbf{x}^{(i)} - b = -1$ hold for several vectors $\textbf{x}^{(i)}$ when $\textbf{w} \...
user avatar
0 votes
1 answer
25 views

Why do we need to multiply linear equation with class in the SVM?

At 2:29 in the video there is an equation; $$ y_i (w^T x_i - b) \ge 1 $$ Could anyone please explain why do we need to multiply the class which is $y_i$ with the linear equation $w^T x_i - b$? N.B: I ...
user avatar
  • 625
2 votes
0 answers
125 views

Disadvantages of cross entropy loss comparing to SVM loss [closed]

What are some disadvantages and limitations of the cross entropy loss, especially compared with SVM loss/hinge loss? I am just looking for a general idea of when would one use SVM loss over cross ...
user avatar
1 vote
0 answers
90 views

Code with explanation on using the OpenCV for image classification using the SVM with python from Scratch

Could anyone share links and resources on Image classification using SVM(Support Vector Machine) from Scratch? Also, there should be the use of the OpenCV library. I got one link, where I am trying to ...
user avatar
  • 625
0 votes
0 answers
14 views

How to proceed with the following dual optimization question

So, here is an image of the question: We are being taught binary classification currently, but I am not sure how to even use this in my objective /cost function. Even if I did, how would I take ...
user avatar
1 vote
2 answers
81 views

How to perform data augmentation with traditional machine learning algorithms?

I am currently working on a multi-class image classification project, in which I have to use traditional machine learning and feature extraction methods (no convolutional neural networks). I know data ...
user avatar
  • 11
4 votes
1 answer
25 views

How the SVM algorithm works with one label in range?

I have a dataset where the features are configurations (numeric values) that describe the situation and the label (only one) is the ranking of the situation (natural value between $[1,5]$). If label ...
user avatar
  • 161
0 votes
0 answers
15 views

Why do we multiply class labels y with linear function (w^t . x + b) in SVM or Logistic regression?

In SVM or Logistic Regression, If we want to see whether a point is properly classified or not in such case we multiply label with linear model which is, $$ y * (w^{t} * x + b) < 1 $$ For point x ...
user avatar
  • 625
2 votes
2 answers
443 views

What is a sigmoid function and what does it give as output?

I know the equation of the sigmoid function and use it in logistic regression, SVM, etc. $$ S(x) = \frac{1}{1 + e^{-x}} $$ In the case of the sigmoid function, What is the exact input and output of ...
user avatar
  • 625
4 votes
1 answer
99 views

Why Are Neural Networks Considered "Expensive" to Train?

Recently, I was looking at the optimization functions required in training Kernel Based Methods compared to Neural Networks. 1) Kernel Methods: For instance, I was looking at the optimization in ...
user avatar
  • 5,650
0 votes
0 answers
12 views

How do you obtain extra-dimensional values using the kernel trick in SVMs?

To preface, I've been reading about SVMs and the kernel trick for the better part of an hour, and I think I understand what it is trying to accomplish fairly well, but what I don't understand is the ...
user avatar
  • 1
1 vote
0 answers
52 views

Why the constraint always holds in soft margin SVM?

In the soft margin SVM, the loss minimization function is given as - Subject to $y_i(w^Tx_i + b) \geq 1 - \varepsilon_i$ and $\varepsilon_i \geq 0$ The 2nd constraint will always be true for any ...
user avatar

1
2 3 4 5
45