Leo
  • Member for 10 years, 9 months
  • Last seen more than 1 year ago
libsvm "reaching max number of iterations" warning and cross-validation
Accepted answer
20 votes

This warning means that the iterative routine used by LIBSVM to solve quadratic optimization problem in order to find the maximum margin hyperplane (i.e., parameters $w$ and $b$) separating your data ...

View answer
Transform data to a higher dimensional space
Accepted answer
11 votes

A general comment: if you want to make two classes of points linearly separable by mapping them into another space, that does not mean that your new space should necessarily have more dimensions than ...

View answer
Use of the Gamma parameter with support vector machines
8 votes

I would suggest the following theoretical guidance. When you are using Gaussian RBF kernel, your separating surface will be based on a combination of bell-shaped surfaces centered at each support ...

View answer
A priori selection of SVM class weights
7 votes

For SVM that minimizes objective function $$\frac{1}{2}||w||^2 + C_1 \sum_{\xi_i: y_i=-1}^{l}\xi_i + C_2 \sum_{\xi_i: y_i=1}^{l}\xi_i $$ you can choose constants $C_1$ and $C_2$ inversely proportional ...

View answer
How to prove there is no finite-dimensional feature space for Gaussian RBF kernel?
Accepted answer
2 votes

Assume that Gaussian RBF kernel $k(x, y)$ is defined on domain $X \times X$ where $X$ contains an infinite number of vectors. One can prove (Gaussian Kernels, Why are they full rank?) that for any set ...

View answer
Meaning of 0-weighted linear weights in SVM
2 votes

Have you normalized the data? If not, and you have features measured on very different scales, then your zero coefficients in the equation of the SVM hyperplane may not be zeros, but merely some small ...

View answer
Calculate probability for LibLinear classification results
2 votes

You can use a sigmoid function $f(d) = \frac{1}{1 + e^{-\alpha(d-\beta)}}$ to convert your SVM decision value $d = (w, x) + b$ into a number between 0 and 1 which can be treated as probability. You ...

View answer
These two variables are almost uncorrelated. What else can I say?
2 votes

So you don't want to add any other variables in your model (average age, level of education, ethnicity, etc.)? It may be useful to visualize these two variables on a geographical map. You may see ...

View answer
Polynomial kernel function
1 votes

If your question is whether it is possible to separate without errors a linearly separable set of points by using polynomial kernel $k(x, z) = (\langle x, z \rangle + 1)^d$, $d > 1$, then the ...

View answer
How to obtain decision boundaries from linear SVM in R?
1 votes

For data point $x$ your SVM calculates decision value $d$ in the following way: d <- sum(w * x) + b If $d > 0$ then label of $x$ is $+1$, else it's $-1$. You can also get labels or decision ...

View answer
How can you predict the likelihood of someone doing something given previous data?
1 votes

You can have a look at this "Mind Reading" game and at the details of its implementation. I think it is very relevant to your second question.

View answer
Beginner to prediction/statistics: Where do I start?
1 votes

"anomaly detection" -- This is usually called detection of outliers. You can find many references via googling. "can more aggressions predict more self injurious behaviors" -- You can try one of the ...

View answer
Improving accuracy of a binary classification when the target is unbalanced
1 votes

Regarding decision trees, I would suggest the following. Assume that you have 10 training examples from class $C_1$ and 90 training examples from class $C_2$. You can use an ensemble of $N$ decision ...

View answer
How do you transform a decision boundary in the angle kernel to the original space?
0 votes

Note that the image of $\phi$ is the unit circle $C$ centered at zero. So for any $y \in C$ $\phi^{-1}(y) = \{ay|a \in R, a \neq 0\}$, while for any $y \notin C$ $\phi^{-1}(y) = \emptyset$. Let's ...

View answer
Training an SVM classifier with non-negative weight constraint
0 votes

I don't have any references on this either, but maybe the following thoughts will be helpful. First, why would you need all $w_i$ to be non-negative? If you need a black box that classifies your data, ...

View answer
Can I perform an exhaustive search with cross-validation for feature selection?
0 votes

I think this is a valid procedure for feature selection which is no more prone to overfitting than other feature selection procedures. The problem with this procedure is that it has large ...

View answer