# Tag Info

17

Let's start with warnings: All the preprocessing should be done using training set's fitted values: X_test = scaler.transform(X_test) X_test = normalizer.transform(X_test) degree is a hyperparameter for polynomial kernel and is ignored if the kernel is not poly: model_2 = SVC(kernel='poly', degree=2, gamma='auto', C=100) OR model_2 = SVC(kernel='rbf', ...

6

@gunes has a very good answer: degree is for poly, and rbf is controlled by gamma and C. In general, it is not surprising to see the default parameter does not work well. See RBF SVM parameters If you change your code model_2 = SVC(kernel='rbf', gamma=1000, C=100) You will see 100% on training but 56% on testing. The reason is As @gunes mentioned the pre-...

2

Yes, it all depends on the algorithm and the cost function. For example, logistic regression's output is between 0-1 and you can't calculate cross entropy loss with labels -1 and 1. Similar things happen in neural nets, you'd want to use tanh in the last layer (because it's output is between -1 and 1). But, you won't be able to use the cross entropy loss ...

1

Maybe here's a simple example to demystify high-dimensional space. Imagining a blood test that gives you the result of one parameter, say haemoglobin concentration. You can plot the results from many patients as points on a line and see if they form clusters. Now consider a blood test that gives two parameters, say haemoglobin and hematocrit. Now you can ...

1

You may think Hyperplane is a linear "decision boundary" on high dimensional space. We can start with 1D and add it up to build up the intuition: When D=1, an example of hyperplane can be x=0. So, the "decision boundary" is a point. And we can use this decision point, to classify any real number into 2 classes. When D=2, an example of ...

1

You are correct that the “number of support vectors” are the training points directly used to find your linear classification boundary. By decreasing the C variable, you are decreasing the amount of variance allowed in your classification boundary, as more support vector are used. By having more than 2 support vectors in many cases, the variance of that ...

1

You can run SVM and, if you wish, different feature selection mechanisms, quite easily with scikit-learn. Hint: Note that SVM is already regularized (margin is maximized = weight vector norm is minimized). So you can try first with vanilla linear SVM. Exactly because of its inherent regularization it is known to work well even when there are millions of ...

1

The distance metric is usually attributed to either LeCam or I. Vincze. The reason why people started calling it $\chi^2$ is that it can be seen as "symmetrized Pearson", see this excerpt from On Measures of Entropy and Information, Tech. Note 009 v0.7, http://threeplusone.com/info , Gavin E. Crooks,2018-09-22:

1

I think it originated from [1], it seems inspired by the $\chi^2$ distance as you said, but it does not have any theoretical motivation. The average between each feature is used as their expected value. Therefore, $$D(x, y) = \sum_i \frac{(x_i - \mu_i)^2 }{\mu_i}$$ where $$\mu_i = \frac{x_i + y_i}{2}$$ such that  D(x, y) = 2 \sum_i \frac{\left(x_i -...

Only top voted, non community-wiki answers of a minimum length are eligible