Tensors often offer more natural representations of data, e.g., consider video, which consists of obviously correlated images over time. You can turn this into a matrix, but it's just not natural or ...

Usually, the decision is whether to use linear or an RBF (aka Gaussian) kernel. There are two main factors to consider: Solving the optimisation problem for a linear kernel is much faster, see e.g. ...

Most classification models in fact don't yield a binary decision, but rather a continuous decision value (for instance, logistic regression models output a probability, SVMs output a signed distance ...

Because we can't. The optimization surface $S(\mathbf{w})$ as a function of the weights $\mathbf{w}$ is nonlinear and no closed form solution exists for $\frac{d S(\mathbf{w})}{d\mathbf{w}}=0$. ...

The question is quite vague so I am going to assume you want to choose an appropriate performance measure to compare different models. For a good overview of the key differences between ROC and PR ...

The hinge loss term $\sum_i\max(0,1-y_i(\mathbf{w}^\intercal \mathbf{x}_i+b))$ in soft margin SVM penalizes misclassifications. In hard margin SVM there are, by definition, no misclassifications. ...

Just to add to the existing answers (which are great, by the way). It is important to be aware that statistical significance is a function of sample size. When you get more and more data, you can ...

As the lead developer of Optunity I'll add my two cents. We have done extensive benchmarks comparing Optunity with the most popular Bayesian solvers (e.g., hyperopt, SMAC, bayesopt) on real-world ...

You can obtain the explicit equation of $\phi$ for the Gaussian kernel via the Tailor series expansion of $e^x$. For notational simplicity, assume $x\in \mathbb{R}^1$: $$\phi(x) = e^{-x^2/2\sigma^2} \... View answer 22 votes The main use-case for bagging is reducing variance of low-biased models by bunching them together. This was studied empirically in the landmark paper "An Empirical Comparison of Voting Classification ... View answer Accepted answer 20 votes This is called learning from positive and unlabeled data, or PU learning for short, and is an active niche of semi-supervised learning. Briefly, it is important to use the unlabeled data in the ... View answer Accepted answer 20 votes Kernel methods can be used for supervised and unsupervised problems. Well-known examples are the support vector machine and kernel spectral clustering, respectively. Kernel methods provide a ... View answer Accepted answer 19 votes For imbalanced data sets we typically change the misclassification penalty per class. This is called class-weighted SVM, which minimizes the following:$$ \begin{align} \min_{\mathbf{w},b,\xi} &\...

As others have mentioned already, there's no clear separation between statistics, machine learning, artificial intelligence and so on so take any definition with a grain of salt. Logistic regression ...

Even though you are training models exclusively on the training data, you are optimizing hyperparameters (e.g. $C$ for an SVM) based on the test set. As such, your estimate of performance can be ...

All kernel methods are based on distance. The RBF kernel function is $\kappa(\mathbf{u},\mathbf{v}) = \exp(-\|\mathbf{u}-\mathbf{v}\|^2)$ (using $\gamma=1$ for simplicity). Given 3 feature vectors: $... View answer Accepted answer 15 votes This may be counterintuitive, but precision is not necessarily monotonically decreasing in terms of the classification threshold. On the other hand, recall is monotonically increasing. (I am assuming ... View answer Accepted answer 15 votes The RBF kernel function is as follows, for two vectors$\mathbf{u}$and$\mathbf{v}$: $$\kappa(\mathbf{u},\mathbf{v}) = \exp(-\gamma \|\mathbf{u}-\mathbf{v}\|^2).$$ The hyperparameter$\gamma$is ... View answer 15 votes This heavily depends on the learning method. Most general purpose approaches have one (or several) ways to deal with this. A common fix is to assign a higher misclassification penalty on the minority ... View answer Accepted answer 14 votes Precision and recall are a tradeoff. Typically to increase precision for a given model implies lowering recall, though this depends on the precision-recall curve of your model, so you may get lucky. ... View answer 14 votes A wide variety of methods exist. They can be largely partitioned in random/undirected search methods (like grid search or random search) and direct methods. Be aware, though, that they all require ... View answer Accepted answer 13 votes Almost all of scikit-learn's classifiers can give decision values (via decision_function or predict_proba). Based on the decision values it is straightforward to compute precision-recall and/or ROC ... View answer Accepted answer 13 votes A likely cause is the fact you are not tuning your model. You need to find good values for$C$and$\gamma$. In your case, the defaults turn out to be bad, which leads to trivial models that always ... View answer Accepted answer 13 votes Area under the ROC curve is equivalent to concordance (aka$c$-statistic) (not accuracy!). This can be interpreted as the probability that a random positive is assigned a higher score than a random ... View answer 13 votes I don't consider this an R specific question. The real question is: can you trust other people's code? Or, taking the other perspective: do you think you can do better? (in the time you are willing/... View answer Accepted answer 12 votes Rules of thumb can only get you so far, but scikit-learn's cheat sheet is quite helpful for basic guidance. Here's a blog post by the creator of said diagram. View answer Accepted answer 11 votes Generating a PR curve is similar to generating an ROC curve. To draw such plots you need a full ranking of the test set. To make this ranking, you need a classifier which outputs a decision value ... View answer 10 votes There is a saturation point. Increasing the size of your training set can't help you surpass the assumptions of your modeling method. For example, if you use a linear model to classify data that is ... View answer Accepted answer 10 votes A necessary and sufficient condition for a function$\kappa(\cdot,\cdot)$to be expressible as an inner product in some feature space$\mathcal{F}$is a weak form of Mercer's condition, namely that:$...