Skip to main content

Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

How do I perform a permutation test on a machine learning model to obtain a p-value for its performance?

this is kinda of the same question of this previous post. But since there's no reply, and I'm having a hard time to find some answers, I'd like to ask it again. I'm training a regression model (SVM ...
artvmac's user avatar
  • 43
0 votes
0 answers
30 views

How to handle Data Normalization in case that a Logarithmic scale is required?

Let's say we wished to build a Regressor (e.g. a Support Vector Regressor) to predict the price of an asset, within a given time span from now on. However, what if the historical data we have ...
Juan Flautista De Torrepacheco's user avatar
0 votes
0 answers
12 views

Derivation of dual formulation of support vector regression

I'm trying to derive the dual formulation of epsilon-insensitive support vector regression. I think my derivation is correct, but I can't match it up to a result for the dual that I've seen given in ...
oweydd's user avatar
  • 225
0 votes
0 answers
7 views

I want to plot the decision boundaries of an SVM model with more than 2 variables

I understand that that is impossible to visualize, so I went in and PCA-transformed the variables. The problem is that I still need more than 2 principal components to get "good" ...
maglorismyspiritanimal's user avatar
0 votes
0 answers
28 views

Applying PCA Before Training Multiple SVM Binary Classifiers To Reduce Data

I am working on a project which has a goal to determine if a new sample is part of Class A or Class A'. I need multiple of those classifiers. I will have an SVM to classify between: ClassA - ClassA' ...
guitardenver's user avatar
1 vote
1 answer
115 views

Non-linear kernel for classifying data points corresponding to two concentric circles [closed]

Have seen article, while doing self-study, on Non-linearly seperable problems, here. The images as given there are here, and here. It deals a common text-book problem, where the data points are in two ...
jiten's user avatar
  • 113
0 votes
1 answer
21 views

SVM Kernel to compare histograms as input vectors

In lecture 7 of CS229 by Andrew Ng he mentions at the very end a specific Kernel that allows an SVM to "classify" how similar two histograms are, such as the demographics of 2 countries. He ...
yyyLLL's user avatar
  • 33
4 votes
5 answers
134 views

Is it valid to exhaustively test all possible combinations of features to find the best combination?

I have about 1000 labelled observations from about 50 subjects responding physiologically under different situations and am trying to classify the situation (usually into three classes of roughly ...
user1596274's user avatar
2 votes
1 answer
68 views

Is my understanding/approach to nested cross-validation, final model tuning correct?

I am training a SVM on limited training data with unbalanced classes. Here are the things that I want to do: 1.) I want to make a statement of the generalizability ...
curious's user avatar
  • 115
0 votes
0 answers
10 views

How is ROC AUC calculated for a Support Vector Machine?

My understanding is that a support vector machine (SVM) finds a hyperplane that separates two classes from each other. During training, there can be some amount of error allowed so that some classes ...
inquisitive_hamster's user avatar
0 votes
0 answers
40 views

Should I interprete data as noise or not

I am tackling a classification problem with 3 classes. Here is what those classes look like on the Two first principal axes. I fine-tuned a SVM model and the best performance achievable was 50%. By ...
Yann's user avatar
  • 43
0 votes
1 answer
24 views

How does fitting data work in SVM using the Kernel Trick?

In SVM, I understand how to fit some data after transforming it into a higher dimension. (ex: $(X_1, X_2) \to (X_1, X_2, X_1^2, X_2^2, X_1X_2)$, which is a 2 dimension to 5 dimension transformation). ...
Random user33's user avatar
0 votes
0 answers
33 views

About the hinge loss and slack variables

I'll be denoting the $ith$ training example, target label and slack variable as $\mathbf{\vec x}^{(i)}$, $y^{(i)}$ and $\xi_i$ respectively. Hinge Loss : The hinge loss function in the context of ...
Sagnik Taraphdar's user avatar
0 votes
0 answers
41 views

How is the SVM optimization objective derived from the hinge loss function?

The hinge loss function, in the context of SVMs, is given as: $$ \mathcal{L}(\mathbf{\vec w}, b\,; \mathbf{\vec x}^{(i)}, y ^{(i)}) = \max(0, 1-y ^{(i)}(\mathbf{\vec w}\cdot \mathbf{\vec x}^{(i)} + b))...
Sagnik Taraphdar's user avatar
3 votes
1 answer
39 views

How to determine one-class SVM's $r$ parameter after obtaining $\alpha$ from QP programming solver?

I'm reading about one-class SVM in wiki here: One-class SVM. One-class SVM attempts to learn $r$ and $c$ to fit a hypersphere to the dataset. The formula for assigning labels is: $$sign(r^2 - ||\phi(x)...
MathematicsBeginner's user avatar
0 votes
0 answers
24 views

Scholkopf single class linear SVM equation: why ρ substracted to 1/2 ||w||² is the same as maximizing the distance

In the one class linear SVM, the equation is : $\min_{w, \rho} \frac{1}{2} \|w\|^2- \rho + C\sum_{i=1}^{n} \xi_i$ subject to: $\begin{align*} & w \cdot x_i \geq \rho - \xi_i, \\ & \xi_i \geq 0,...
Arnaud Feldmann's user avatar
0 votes
1 answer
44 views

Learning Curve to Know Underfitting or Overfitting

I want to know if the model I am using tends to be overfitting or underfitting. I am using SVM and Random Forest algorithms. How to figure it out?
Anna's user avatar
  • 3
0 votes
0 answers
8 views

Introducing bias via combining probability outputs from multiple models

I am working on a classification task, where I am trying to estimate the probability that a patient may not die. I did use a Survival Analysis approach at first, but the results seemed unintuitive and ...
vjgu's user avatar
  • 23
0 votes
0 answers
15 views

Can I find the explicit feature map that generates exponent of a kernel?

Let's say I have a kernel $K$, and another kernel of the form : $$ K' = e^K $$ now I know how to prove K' is a kernel, I can do it using taylor expansion of $e^x$ around $0$, but let's say if I want ...
aroma's user avatar
  • 123
1 vote
0 answers
36 views

Support Vector Classifiers for Overlapping Classes

I am currently studying support vector classifiers (SVC), more specifically, the solution to the Lagrangian (Wolfe) dual function with the help of the book "The Elements of Statistical Learning&...
Kobi's user avatar
  • 11
0 votes
0 answers
12 views

Does L2-SVM involve always using the squared hinge loss?

Im trying to understand the math behind the L1 and L2 SVM but Im kind of confused at this point : in the following picture we see that the regularization term is squared for the L2-SVM. Does that mean ...
Imene Charabi's user avatar
2 votes
2 answers
72 views

How is the Representer theorem used in the derivation of the SVM dual form?

This is the primal form of the SVM hypothesis : $$ h _{\mathbf{\vec w}, b}(\mathbf{\vec x}^{(i)}) = \mathbf{\vec w}\cdot \mathbf{\vec x}^{(i)} + b $$ The Representer theorem as formulated here ...
Sagnik Taraphdar's user avatar
3 votes
1 answer
105 views

Why is the regularization term multiplied by the error term in the cost function of SVM?

The cost function of the Optimal Margin Classifier(non-kernelized SVM) is given as : $$ J(\mathbf{\vec w}, b) = \frac{1}{2}\|\mathbf{\vec w}\|_{2}^{2} + C \sum_{i=1}^{n}\max(0, 1-y ^{(i)}(\mathbf{\vec ...
Sagnik Taraphdar's user avatar
3 votes
1 answer
171 views

Scenario where minimizing 0-1 loss is different than minimizing hinge loss

Suppose we're using linear predictors. I'm trying to conceptually understand how minimizing hinge loss and 0-1 loss aren't necessarily the same. For instance I was told that one can choose a set of ...
redbull_nowings's user avatar
0 votes
0 answers
45 views

How to use random kitchen sinks for $\sigma \neq 1$?

The RBF kernel is given by $$ k(x,y) = \exp\left(-\frac{\| x - y \|_2^2}{2 \sigma^2}\right) $$ where $\sigma$ is the length-scale parameter. I want to use the random kitchen sinks method to create a ...
user336650's user avatar
0 votes
0 answers
12 views

Linear SVM vs Decision Stumps for AdaBoost

I have heard that AdaBoost can use a linear SVM as a weak classificer. I wonder why Decision Stumps is often used with AdaBoost? Booth are binary classifiers. In my opinion, linear SVM seems to be a ...
euraad's user avatar
  • 425
3 votes
2 answers
158 views

Support Vector machine - Hingeloss

What does it mean that 'The SVM hinge loss estimates the mode of the posterior class probabilities'(Elements of statistical Learning p.427). The decision function f(x) assigns to the positive class(+...
J.doe's user avatar
  • 359
1 vote
1 answer
40 views

Availability of Linear Grouping Algorithms to Linearly Cluster Datasets

I have been trying to cluster a scatter plot that has a triangular graph, ideally the proper clustering plot should have a linear form, as shown below: I tried using Spectral Clustering: and ...
NOT-A-CS-GUY's user avatar
0 votes
0 answers
25 views

How to use RFE for RF and SVM

Considering I have a big data (lots of OTUs and clinical), which I will be using to input into RF and SVM for prediction (classification), will it make sense to perform RFE as a feature selection step?...
Tori's user avatar
  • 11
1 vote
0 answers
27 views

Feature selection before ML (RF and SVM)

I am new to machine learning and have to work with big data (lots of OTUs along with clinical) which I will input into 2 different machine learning models (RF and SVM) that will be used for prediction ...
Tori's user avatar
  • 11
2 votes
1 answer
30 views

Interpreting the formula for Riemannian metric tensor

In Improving support vector machine classifiers by modifying kernel functions, the authors defined Riemannian metric tensor for a kernel as follows: $$ \begin{align} g(\vec{x}) &= \text{det}|g_{ij}...
Omar Shehab's user avatar
1 vote
2 answers
1k views

Support Vector Regression vs. Linear Regression

I am new to ML and I am learning the different algorithms one can use to perform regression. Keep in mind that I have a strong mathematical background, but I am new in the ML field. So I understand ...
kubo's user avatar
  • 145
0 votes
1 answer
32 views

An extremely simple classification problem leads to intractable SVM program

In the popular textbook Mathematics for Machine Learning, creating a SVM requires solving: $\text{min}_{w,b} \dfrac{1}{2}\|w\|^2$ subject to $y_n (w^T x_n + b) \geq 1$, for all $n = 1, \ldots, N$ Ok, ...
Fraïssé's user avatar
  • 1,580
2 votes
0 answers
111 views

Textbook Recommendation other than ESL [duplicate]

My current background is as follows: (core subjects only) Math : Linear Algebra, Analysis, (half of) Measure TheoryStats : Mathematical Statistics, Regression Analysis, Multivariate Analysis "...
jason 1's user avatar
  • 311
1 vote
0 answers
29 views

Convexitiy of multi-class hinge loss

The empirical risk of a multi-class hinge-loss is given by $$L(\Theta,(x,y) = \max_{j \neq y} \Big[1+ \sum_{i=1}^{d} x_i(\Theta_{ij} - \Theta_{iy}) \Big]_{+} $$ where $x \in \mathbb{R}^{d}$ is a ...
Oskar's user avatar
  • 255
2 votes
0 answers
27 views

Implement Nesterov's acceleration for SVM

I am trying to implement Nestrov's acceleration gradient descent for SVM. The objective function I need to minimize is $$\frac{1}{2}\lVert Au-Bv\rVert_2^2$$ with constraints $\sum_{i}u_i=\sum_{j}v_j=1$...
struggleinmath's user avatar
0 votes
0 answers
12 views

Classify text by topics using SVM: Derive the upper bound for the norm of the weight vector

In the section on SVM in one book I'm reading, the authors wrote: Consider the problem of learning to classify a short text document according to its topic, say, whether the document is about sports ...
Tran Khanh's user avatar
0 votes
0 answers
26 views

Sklearn feature selection performs strangely with 2 groups (and with SVC)

Previously I've successfully performed support vector classification with recursive feature elimination in R using the e1071 package, but I'm now hoping to move over to SciKit Learn given that Python ...
Benjamin Taylor's user avatar
1 vote
0 answers
26 views

How to forecast changepoints from Gas Concentration Data?

So I'm trying to predict when gas concentrations change from sensor conductivity readings over a day. The gases randomly change concentrations around every 80-120 seconds and are kept constant between ...
Jawi Doen's user avatar
0 votes
0 answers
50 views

Is $\ell_1$ regularization not compatible with SVM?

In the notes of Andrew Ng's CS229 Machine Learning course, it is mentioned: The $\ell_2$ norm regularization is much more commonly used with kernel methods because $\ell_1$ regularization is ...
Katatonic's user avatar
2 votes
2 answers
297 views

What method should be used if the clusters contains different classes?

Assume that you having $N$ clusters. Each cluster have multiple classes. So we know the class ID for every major clusters, but not the class ID for the data points inside the major clusters. Each ...
euraad's user avatar
  • 425
1 vote
1 answer
197 views

Lasso for feature selection in classification models

I want to perform classification of breast cancer cases by using models like SVM or Random Forest. When I was browsing the web I saw that one could use Lasso for feature selection, and then applied ...
Layla's user avatar
  • 621
0 votes
0 answers
10 views

Which method should be used if I want to find relations between two variables

Assume that you have a matrix $X \in \Re^{M x N}$ that have $M$ rows and $N$ columns. The rows $M$ can vary in length, but the $N$ columns remains constant. Each row is labeled with a class ID. The ...
euraad's user avatar
  • 425
0 votes
0 answers
21 views

What is the value that the "b" term should have before the optimization problem starts?

I have been beating my head against this wall: There is this optimization problem for SVM (photo taken from Andrew Ng's lecture notes) What I did not get is how, by using a quadratic solver for ...
Flavius Miron's user avatar
0 votes
1 answer
45 views

Maxima in the dual of hard margin and soft margin SVMs?

The dual problem for hard margin SVM is: \begin{align*} &\max_{\alpha} \left( \sum_{i=1}^{N} \alpha_i - \frac{1}{2} \sum_{i=1}^{N} \sum_{j=1}^{N} \alpha_i \alpha_j y^{(i)} y^{(j)} \langle x^{(i)}, ...
something something's user avatar
1 vote
0 answers
41 views

SVM kernels corresponding to different types of distance measures

This answer to Data normalization for RBF kernel points out that RBF kernel implies Eucledean distance. Are there kernels corresponding to other popular distance/dissimilarity measures, such as Bray-...
Roger V.'s user avatar
  • 4,421
2 votes
0 answers
10 views

How to prove that 2d support vectors are enough for Hard Margin Linear SVM?

As the question states, how can I prove mathematically that 2d support vectors are enough to always be able to formulate the Maximum Margin Hyperplane in d dimensions?
heloworld's user avatar
4 votes
1 answer
55 views

In case of no correlation, can a model make predictions above the expected values?

For simplicity's sake, let's suppose a binary classification problem, with a perfect 50% of probability for each of the classes, and a SkLearn's SVC model. Let's ...
Juan Flautista De Torrepacheco's user avatar
0 votes
1 answer
35 views

The distance from the hyperplane to the points

In Support Vector Machines, the distance from the hyperplane to each class of nearest points should have the same length. Is this correct?
user395520's user avatar
0 votes
1 answer
81 views

Performing a classification if having categorial labels and a distance matrix

I encountered a multi-class classification problem and I wonder which model would work the best in my scenario. I have around 50,000 vectors (each of size 200) with corresponding categorical labels ...
Denis Marcinkov's user avatar

1
2 3 4 5
46