Tag Info

What is the influence of C in SVMs with linear kernel?

In a SVM you are searching for two things: a hyperplane with the largest minimum margin, and a hyperplane that correctly separates as many instances as possible. The problem is that you will not ...

How to intuitively explain what a kernel is?

Kernel is a way of computing the dot product of two vectors $\mathbf x$ and $\mathbf y$ in some (possibly very high dimensional) feature space, which is why kernel functions are sometimes called "...
• 8,407

How does a Support Vector Machine (SVM) work?

Ryan Zotti's answer explains the motivation behind the maximization of the decision boundaries, carlosdc's answer gives some similarities and differences with respect to other classifiers. I'll give ...
• 43.5k

How to intuitively explain what a kernel is?

A visual example to help intuition Consider the following dataset where the yellow and blue points are clearly not linearly separable in two dimensions. If we could find a higher dimensional space ...
Accepted

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

What is an SVM, anyway? I think the answer for most purposes is “the solution to the following optimization problem”:  \begin{split} \operatorname*{arg\,min}_{f \in \mathcal H} \frac{1}{n} \sum_{i=1}...
• 22.5k

What makes the Gaussian kernel so magical for PCA, and also in general?

I think the key to the magic is smoothness. My long answer which follows is simply to explain about this smoothness. It may or may not be an answer you expect. Short answer: Given a positive ...
• 2,043

Why not just dump the neural networks and deep learning?

Not being able to know what solution generalizes best is an issue, but it shouldn't deter us from otherwise using a good solution. Humans themselves often do not known what generalizes best (consider, ...

How to intuitively explain what a kernel is?

A very simple and intuitive way of thinking about kernels (at least for SVMs) is a similarity function. Given two objects, the kernel outputs some similarity score. The objects can be anything ...
• 2,365

How can SVM 'find' an infinite feature space where linear separation is always possible?

This answer explains the following: Why perfect separation is always possible with distinct points and a Gaussian kernel (of sufficiently small bandwidth) How this separation may be interpreted as ...
• 10.2k
Accepted

What algorithms need feature scaling, beside from SVM?

In general, algorithms that exploit distances or similarities (e.g. in the form of scalar product) between data samples, such as k-NN and SVM, are sensitive to feature transformations. Graphical-model ...
• 546

What algorithms need feature scaling, beside from SVM?

Here is a list I found on http://www.dataschool.io/comparing-supervised-learning-algorithms/ indicating which classifier needs feature scaling: Full table: In k-means clustering you also need to ...
• 43.5k
Accepted

Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

Here is one theoretical and two practical reasons why someone might rationally prefer a non-DNN approach. The No Free Lunch Theorem from Wolpert and Macready says We have dubbed the associated ...
• 19.5k
Accepted

How to know whether the data is linearly separable?

There are several methods to find whether the data is linearly separable, some of them are highlighted in this paper (1). With assumption of two classes in the dataset, following are few methods to ...
• 631

Are neural networks better than SVMs?

Short answer: On small data sets, SVM might be preferred. Long answer: Historically, neural networks are older than SVMs and SVMs were initially developed as a method of efficiently training the ...
• 6,725
Accepted

Logarithmic loss minimization leads to well-behaved probabilistic outputs. Hinge loss leads to some (not guaranteed) sparsity on the dual, but it doesn't help at probability estimation. Instead, it ...
• 16.1k

How to prove that the radial basis function is a kernel?

I'll add a third method, just for variety: building up the kernel from a sequence of general steps known to create pd kernels. Let $\mathcal X$ denote the domain of the kernels below and $\varphi$ the ...
• 22.5k

Comparing SVM and logistic regression

Image signifies the difference between SVM and Logistic Regression and where to use which method this picture comes from the coursera course : "machine learning" by Andrew NG. It can be ...
• 423
Accepted

What is the statistical model behind the SVM algorithm?

You can often write a model that corresponds to a loss function (here I'm going to talk about SVM regression rather than SVM-classification; it's particularly simple) For example, in a linear model, ...
• 264k

Is it a good idea to use CNN to classify 1D signal?

I guess that by 1D signal you mean time-series data, where you assume temporal dependence between the values. In such cases convolutional neural networks (CNN) are one of the possible approaches. The ...
• 120k

Are neural networks better than SVMs?

You may have heard of the "no free lunch theorem" in machine learning. For each model, there are pros and cons for specific data and use case. So. NN is not better than SVM and I can give ...
• 33.8k
Accepted

Why are optimization algorithms defined in terms of other optimization problems?

You are looking at top level algorithm flow charts. Some of the individual steps in the flow chart may merit their own detailed flow charts. However, in published papers having an emphasis on ...
• 12.7k

Is there any supervised-learning problem that (deep) neural networks obviously couldn't outperform any other methods?

Somewhere on this playlist of lectures by Geoff Hinton (from his Coursera course on neural networks), there's a segment where he talks about two classes of problems: Problems where noise is the key ...
• 4,737
Accepted

Does Dimensionality curse effect some models more than others?

In general, the curse of dimensionality makes the problem of searching through a space much more difficult, and effects the majority of algorithms that "learn" through partitioning their vector space. ...
• 1,023
Accepted

What does the cost (C) parameter mean in SVM?

I'm trying to give a simple and easy-to-understand answer. A complete answer would likely need to cover everything from the purpose behind SVMs to the finer details of loss and support vectors. If you ...
• 3,731

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. A good way to understand how the weights are calculated and how to interpret them in the case of linear SVM is to perform ...
Accepted

How to determine the optimal threshold for a classifier and generate ROC curve?

Use the SVM classifier to classify a set of annotated examples, and "one point" on the ROC space based on one prediction of the examples can be identified. Suppose the number of examples is ...
• 818

PCA on train and test datasets: should I run one PCA on train+test or two separate on train and on test?

(1) is incorrect, because if you run PCA on the two sets separately, you will end up with two different spaces. You cannot train a classifier in one space, and apply it to a different space. (2) is ...
• 421

How does a Support Vector Machine (SVM) work?

The technique is predicated upon drawing a decision boundary line leaving as ample a margin to the first positive and negative examples as possible: As in the illustration above, if we select an ...
• 24.5k

What does average of word2vec vector mean?

You can think of it in terms of physical analogy. You can take a flat surface, like a table, and arrange 30 balls on it. Then you can cut legs from the table and replace it with a single leg. In order ...
• 7,314
Accepted

Difference between ep-SVR and nu-SVR (and least squares SVR)

In $\nu$-SVR, the parameter $\nu$ is used to determine the proportion of the number of support vectors you desire to keep in your solution with respect to the total number of samples in the dataset. ...
• 481

Only top scored, non community-wiki answers of a minimum length are eligible