Marc Claesen
  • Member for 8 years, 8 months
  • Last seen more than a month ago
Why the sudden fascination with tensors?
94 votes

Tensors often offer more natural representations of data, e.g., consider video, which consists of obviously correlated images over time. You can turn this into a matrix, but it's just not natural or ...

View answer
Linear kernel and non-linear kernel for support vector machine?
Accepted answer
76 votes

Usually, the decision is whether to use linear or an RBF (aka Gaussian) kernel. There are two main factors to consider: Solving the optimisation problem for a linear kernel is much faster, see e.g. ...

View answer
Why downsample?
Accepted answer
46 votes

Most classification models in fact don't yield a binary decision, but rather a continuous decision value (for instance, logistic regression models output a probability, SVMs output a signed distance ...

View answer
Why use gradient descent with neural networks?
Accepted answer
36 votes

Because we can't. The optimization surface $S(\mathbf{w})$ as a function of the weights $\mathbf{w}$ is nonlinear and no closed form solution exists for $\frac{d S(\mathbf{w})}{d\mathbf{w}}=0$. ...

View answer
Area under the ROC curve or area under the PR curve for imbalanced data?
Accepted answer
33 votes

The question is quite vague so I am going to assume you want to choose an appropriate performance measure to compare different models. For a good overview of the key differences between ROC and PR ...

View answer
What is the loss function of hard margin SVM?
Accepted answer
31 votes

The hinge loss term $\sum_i\max(0,1-y_i(\mathbf{w}^\intercal \mathbf{x}_i+b))$ in soft margin SVM penalizes misclassifications. In hard margin SVM there are, by definition, no misclassifications. ...

View answer
Why is "statistically significant" not enough?
27 votes

Just to add to the existing answers (which are great, by the way). It is important to be aware that statistical significance is a function of sample size. When you get more and more data, you can ...

View answer
Advantages of Particle Swarm Optimization over Bayesian Optimization for hyperparameter tuning?
Accepted answer
26 votes

As the lead developer of Optunity I'll add my two cents. We have done extensive benchmarks comparing Optunity with the most popular Bayesian solvers (e.g., hyperopt, SMAC, bayesopt) on real-world ...

View answer
Feature map for the Gaussian kernel
24 votes

You can obtain the explicit equation of $\phi$ for the Gaussian kernel via the Tailor series expansion of $e^x$. For notational simplicity, assume $x\in \mathbb{R}^1$: $$\phi(x) = e^{-x^2/2\sigma^2} \...

View answer
What are the theoretical guarantees of bagging
22 votes

The main use-case for bagging is reducing variance of low-biased models by bunching them together. This was studied empirically in the landmark paper "An Empirical Comparison of Voting Classification ...

View answer
How to predict outcome with only positive cases as training?
Accepted answer
20 votes

This is called learning from positive and unlabeled data, or PU learning for short, and is an active niche of semi-supervised learning. Briefly, it is important to use the unlabeled data in the ...

View answer
What are the limitations of Kernel methods and when to use kernel methods?
Accepted answer
20 votes

Kernel methods can be used for supervised and unsupervised problems. Well-known examples are the support vector machine and kernel spectral clustering, respectively. Kernel methods provide a ...

View answer
Does Support Vector Machine handle imbalanced Dataset?
Accepted answer
19 votes

For imbalanced data sets we typically change the misclassification penalty per class. This is called class-weighted SVM, which minimizes the following: $$ \begin{align} \min_{\mathbf{w},b,\xi} &\...

View answer
Why is logistic regression called a machine learning algorithm?
18 votes

As others have mentioned already, there's no clear separation between statistics, machine learning, artificial intelligence and so on so take any definition with a grain of salt. Logistic regression ...

View answer
Why splitting the data into the training and testing set is not enough
17 votes

Even though you are training models exclusively on the training data, you are optimizing hyperparameters (e.g. $C$ for an SVM) based on the test set. As such, your estimate of performance can be ...

View answer
ROC curves for unbalanced datasets
Accepted answer
17 votes

ROC curves are insensitive to class balance. The straight line you obtain for a random classifier now is already the result of using different probabilities of yielding positive (0 brings you to (0, 0)...

View answer
When using SVMs, why do I need to scale the features?
Accepted answer
16 votes

All kernel methods are based on distance. The RBF kernel function is $\kappa(\mathbf{u},\mathbf{v}) = \exp(-\|\mathbf{u}-\mathbf{v}\|^2)$ (using $\gamma=1$ for simplicity). Given 3 feature vectors: $...

View answer
Are precision and recall supposed to be monotonic to classification threshold
Accepted answer
15 votes

This may be counterintuitive, but precision is not necessarily monotonically decreasing in terms of the classification threshold. On the other hand, recall is monotonically increasing. (I am assuming ...

View answer
Where can I read about gamma coefficient in SVM in scikit-learn?
Accepted answer
15 votes

The RBF kernel function is as follows, for two vectors $\mathbf{u}$ and $\mathbf{v}$: $$ \kappa(\mathbf{u},\mathbf{v}) = \exp(-\gamma \|\mathbf{u}-\mathbf{v}\|^2). $$ The hyperparameter $\gamma$ is ...

View answer
Class imbalance in Supervised Machine Learning
15 votes

This heavily depends on the learning method. Most general purpose approaches have one (or several) ways to deal with this. A common fix is to assign a higher misclassification penalty on the minority ...

View answer
A way to maintain classifier's recall while improving precision
Accepted answer
14 votes

Precision and recall are a tradeoff. Typically to increase precision for a given model implies lowering recall, though this depends on the precision-recall curve of your model, so you may get lucky. ...

View answer
Guideline to select the hyperparameters in Deep Learning
14 votes

A wide variety of methods exist. They can be largely partitioned in random/undirected search methods (like grid search or random search) and direct methods. Be aware, though, that they all require ...

View answer
Classifier with adjustable precision vs recall
Accepted answer
13 votes

Almost all of scikit-learn's classifiers can give decision values (via decision_function or predict_proba). Based on the decision values it is straightforward to compute precision-recall and/or ROC ...

View answer
Output of Scikit SVM in multiclass classification always gives same label
Accepted answer
13 votes

A likely cause is the fact you are not tuning your model. You need to find good values for $C$ and $\gamma$. In your case, the defaults turn out to be bad, which leads to trivial models that always ...

View answer
What is AUC of PR-curve?
Accepted answer
13 votes

Area under the ROC curve is equivalent to concordance (aka $c$-statistic) (not accuracy!). This can be interpreted as the probability that a random positive is assigned a higher score than a random ...

View answer
Open source libraries in science
13 votes

I don't consider this an R specific question. The real question is: can you trust other people's code? Or, taking the other perspective: do you think you can do better? (in the time you are willing/...

View answer
Machine learning classifiers
Accepted answer
12 votes

Rules of thumb can only get you so far, but scikit-learn's cheat sheet is quite helpful for basic guidance. Here's a blog post by the creator of said diagram.

View answer
How to form a Precision-Recall curve when I only have one value for P-R?
Accepted answer
11 votes

Generating a PR curve is similar to generating an ROC curve. To draw such plots you need a full ranking of the test set. To make this ranking, you need a classifier which outputs a decision value ...

View answer
Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?
10 votes

There is a saturation point. Increasing the size of your training set can't help you surpass the assumptions of your modeling method. For example, if you use a linear model to classify data that is ...

View answer
Linear combination of two kernel functions
Accepted answer
10 votes

A necessary and sufficient condition for a function $\kappa(\cdot,\cdot)$ to be expressible as an inner product in some feature space $\mathcal{F}$ is a weak form of Mercer's condition, namely that: $...

View answer
1
2 3 4 5
11