Hot answers tagged

11 votes
Accepted

Meaning of Epsilon in SVM regression

You have it backwards. Traditional $\epsilon$-SVR works with the epsilon-insensitive hinge loss. The value of $\epsilon$ defines a margin of tolerance where no penalty is given to errors. Remember ...
  • 16.1k
10 votes
Accepted

R caret Naive Bayes (untuned) results differ from klaR

The problem lies in the fact that you use a different specification in the models. In fit1 and fit2 you use the x and y combination, in fit3 the formula notation If you switch all models in the ...
  • 1,017
8 votes
Accepted

When SVM can do multi-class classification itself, why do people still use (one vs one) or (one Vs many) classification?

multi-class in any SVM package (including e1071) is either one vs one or one vs many. From the e1071 manual: For multiclass-classification with k levels, k>2, libsvm uses the ‘one-against-one’-...
  • 8,802
8 votes
Accepted

Difference between the types of SVM

Short answer You can select what to used based on your goal and what kind of data you have. If you have a classification problem, i.e., discrete label to predict, you can use ...
  • 33.8k
5 votes
Accepted

R e1071 tune plot does not give me the best gamma?

When you say "better results" and provide the different errors you got, is that evaluated in-sample or out-of-sample? SVM tuning will attempt to give you the best tuning parameters for out-of-sample ...
4 votes
Accepted

SVM in R (package e1071): predicting class using predict()

From the documentation you can read that see ?svm (or here): The probability model for classification fits a logistic distribution using maximum likelihood to ...
  • 120k
4 votes

Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

It certainly helps, but isn't absolutely essential. The choice of cross-validation splits introduces a source of (uninteresting) variability. Using the same set of splits removes this source of ...
  • 19.5k
4 votes

R caret Naive Bayes (untuned) results differ from klaR

From my knowledge, for the first two models, you shouldn't be giving the whole spam data frame as the training variables (the class labels are considered as a feature in this case). Instead, you ...
  • 216
4 votes
Accepted

Random Forest vs Support Vector Machine. Which is faster?

Recall how SVM works, it applies the kernel to each pair of the inputs, and this scales badly. SVM has time complexity of $O(dn^2)$ or $O(dn^3)$ for libsvm used by e1071. Random forest uses ...
  • 120k
3 votes

SVM always gives me (in average) below chances cross validation accuracy with random data

your question intrigued me as I use the svm() function from this package from time to time. At first, I thought that you did run only one experiment per sample size,...
  • 126
3 votes

Does SVM perform poorly with too few observations

It sounds like you're trying to apply the intuitions of a hard-margin SVM to the results of a soft-margin SVM and being led astray by the differences. I can tell you're using a soft-margin SVM ...
  • 2,828
3 votes
Accepted

Naive Bayes Classifier in R with class weights

caret uses the naive Bayes function from the klaR package. It sounds like you want to adjust the prior: ...
  • 5,880
3 votes
Accepted

Why does the linear SVM give a lot of support vectors?

Any linear SVM can be summarized as a single vector in input space: $$f(\mathbf{z}) = \sum_{i \in SV} y_i \alpha_i \mathbf{x}_i^T\mathbf{z} +b,$$ can be rewritten as: $$f(\mathbf{z}) = \mathbf{w}^T\...
  • 17.6k
3 votes

c-classification SVM vs nu-classification SVM in e1071 R

I once asked a question very similar to your: Difference between the types of SVM. Here is the relevant part of the answer. C-classification and nu-classification is for binary classification usage. ...
  • 5,003
2 votes

Methods to combine ( e1071 svm ) models in R to generate a more complete, accurate model

First, make sure you shuffle your data before you separate it in to the 50 partitions. If the original data is at all ordered, this can make a huge difference in the performance of your final model. ...
2 votes

e1071 svm queries regarding plot and tune

I'm not familiar with the plot function of e1071 but I can help with your second question. The purpose of the tune function is to find the optimal cost and gamma ...
  • 18.8k
2 votes

Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

In addition to @Matt Krause's answer: I'd approach the question from two different sides: One of the basic assumptions underlying cross validation is that the models built on the different splits ...
2 votes
Accepted

Do we have to fix splits before 10-folds cross validation if we want to compare different algorithms?

The results will be sensitive to the splits, so you should compare models on the same partitioning of the data. Compare these two approaches: Approach 1 will compare two models, but use the same CV ...
  • 81.2k
2 votes
Accepted

Any way of getting vector of probabilities for each response with Naive Bayes in R?

You just need to set type = "raw" in the predict object. ...
  • 16.1k
2 votes

Fastest SVM library in R for large data and Multiclass problem

"LiblineaR" can be used on large data sets. Alternatively, you may try H2O libraries.
2 votes

naiveBayes does not give expected probabilities

Computer does the calculations for you, computers do not operate on real numbers, but approximate them. The results would never be exact, and depending on how exactly the calculations were done and ...
  • 120k
2 votes
Accepted

R - SVM (radial) regression using tune() (e1071) - How to uncover influential features?

You can't really interpret SVM models unless a linear kernel is used (you have the RBF in your code above). The matrix multiplication is only for the linear case too (I believe). There is an ...
  • 5,880
2 votes

Manually finding the bias term in classification SVM in R

There's a tiny mistake in your loop. From e1071 documentation: coefs The corresponding coefficients times the training labels. It means ...
  • 121
2 votes
Accepted

Error and Dispersion meaning in tune.out for SVM Classifier

If you dig into the code of tune, you'll find that it calculates error for each of the surrogate models, and then aggregates these per-model error estimates into a ...
2 votes

R Naive Bayes and Laplace: Even turned off, works fine with unseen words in test data?

If you check the source code, predict.naiveBayes method from e1071 package simply ignores the new columns from test set (words that were not seen in training set) ...
  • 120k
2 votes
Accepted

Recover $\rho$ of $\nu$-SVM from e1071 package in R

We can recover this by noting that $0 \leq \alpha_i \leq C = \frac{1}{\rho n}$ (from that same section in the LIBSVM paper, substituing $n$ for $l$). Thus, $\rho = \frac{1}{n \max \alpha_i}$. Note ...
  • 1,379
1 vote

SVM always gives me (in average) below chances cross validation accuracy with random data

There may be some pessimistic bias, yes: If the splitting (accidentally) leads to one class being underrepresented in the training set (which will happen if the sampling is not stratified for the ...
1 vote

How to train an SVR model?

In your second example you are using only very small values for c in grid search: ((0.000001,0.00001,0.0001)) Low values of c in ...
  • 1,128
1 vote

naiveBayes does not give expected probabilities

Providing an answer to this old question in case someone else stumbles here late as I did. It seems like the e1071 package does not compute the posterior ...
1 vote

How to calculate decision boundary from nonlinear svm in R?

That depends. In the radial-basis function (RBF) case, it's generally impossible to obtain the weights. The kernel trick is applied as I outlined in this answer. Basically, we define new weights $\...
  • 16.1k

Only top scored, non community-wiki answers of a minimum length are eligible