New answers tagged

0

To add gunes' excellent answer, you may also use several scoring functions following: scoring = {'accuracy': make_scorer(accuracy_score), 'precision': make_scorer(precision_score, average = 'macro'), 'recall': make_scorer(recall_score, average = 'macro'), 'f1_macro': make_scorer(f1_score, average = 'macro', '...


0

Short answer: yes. Your choice of kernel function may very well be sensitive to feature scaling. Since the SVM generates a "linear" boundary in the kernel space, different scalings of features will affect how that classification occurs. Recall that SVMs classify points based on the value of $sign(\phi(w)^\top \phi(x) + b)$, where $\phi(\cdot)$ is the ...


1

In soft margin SVM, we start with loss finction $$ min \space\space \space \frac{1}{2}\lVert \beta \rVert ^2 + \gamma\sum_{i} \xi_i $$ $$ subject \space to \space\space\space y_i(\beta^Tx_i + \beta_0) \ge 1 - \xi_i \\\xi_i \ge 0 $$ The margin here also remains the unit distance from the decision boundary, but, we allow some points to cross this margin ...


1

I would advise against dropping features from a model unless there is a clear indication that they are irrelevant for our modelling task. That said the down-grading of performance is noticeable (72% -> 63%). Assuming that the data is properly normalised so the regularisation effect from $C$ is properly applied (i.e. the scaling is done using the parameters ...


0

I resolved this problem some time ago and thought I would present the answer for those who are looking. In short, the Sklearn SVM does not perform in a typical one-vs-rest implementation. Instead it uses a wrapper to develop a decision function that combines the decision functions of several one-vs-other classifiers to perform its classification. This ...


1

For reference, the optimization problem in hand is \begin{gather}\tag{1} \max_{\beta, \beta_0} M\\ \text{subject to }y_i(x_i^T\beta + \beta_0)\geq M\|\beta\|, \forall i. \end{gather} Assume the pair $(\widehat{\beta}, \widehat{\beta}_0)$ is a solution to this problem and let $\widehat{M}$ be the achieved objective value. Also assume that $(\beta^*, \beta_0^*...


0

As usεr11852 says, cross validation does make sense for optimizing linear SVMs. According to Hastie et al.: The Entire Regularization Path for the Support Vector Machine, Journal of Machine Learning Research 5 (2004) 1391–1415, inside the cross validation you don't need to compute one SVM per C value as it is possible to compute the complete path of SVMs ...


0

Yes, it makes full sense to use cross-validation to find optimal hyper-parameters values in the case of SVM with a linear kernel. If anything, the choosing the regularisation parameter $C$ in the Lagrange formulation is analogous to choosing the ridge regularisation parameter in ridge regression. Therefore it is necessary for our training data to be scaled ...


0

As described in here, libsvm uses 1 vs 1 classifiers for multi-class classification (it's not the only option). So, it uses $n(n-1)/2$ binary SVM classifiers internally. Here, that makes $45$ classifiers, which explains the number of bias terms. sv_coef contains $n-1$ columns. Number of rows is equal to number of SVs. The number of SVs belonging to each ...


0

Firstly, why are you only using one classifier? See this answer on data mining theorems and problems with using one classifier. You should be using linear regression first, possibly followed by logistic regression, Naive Bayes, k-nearest neighbor, linear discriminant analysis, etc. You should also be considering use of an ensemble of classifiers based on ...


0

I recommend you to use RBF kernel, followed by hyperparameter tuning of C and gamma. Linear kernel is not recommended when you have many labels.


1

For SVMs the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane. The closest positive and negative examples to the separating hyperplane are, $\arg\max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $\arg \min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$ These verify (...


Top 50 recent answers are included