Questions tagged [caret]
Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.
396
questions
0
votes
0answers
12 views
caret() and glmnet() give different coefficients
I have understood from this post (https://stackoverflow.com/questions/48653465/r-coefficients-from-glmnet-and-caret-are-different-for-the-same-lambda) that caret() and glmnet() may not use the same ...
1
vote
1answer
25 views
Predictions based on k-fold Cross Validation, which model is used (Caret)
I am sorry if there is an obvious or intuitive answer to this, which I missed.
We have tuned the hyperparameters of a RF using Grouped 10 - Fold CV (repeated 5 times), to obtain the values for mtry ...
0
votes
1answer
36 views
Grouped 7-fold Cross Validation in R
I am searching for a grouped 7-fold cross validation function. I couldn't find it in the caret package.
I got 70 subjects performing 7 trials (Outcome variable: categorical with 7 values) = 490 ...
0
votes
0answers
39 views
R-Caret, Regression, different number of PCA components for finalModel and resampling with PCA?
In my project I train models with the "Timeslice" method from the caret package but this question also fits in with other methods, such as cross-validation.
Imagine you have 2584 records and split ...
0
votes
0answers
7 views
Any project on iterative evaluation of dimensionality reduction and model selection strategies?
Caret and Scikit-learn offer great many alternatives for various steps in machine learning. Is there any project that aims at trying all(or most) available alternatives in these packages (or other ...
1
vote
1answer
15 views
Testing set accuracy by using cross validation using xgboost with caret
I am working on an xgboost model using caret. I'm using cross validation, but don't know if I'm understanding it correctly. As I understand, it creates multiple training and test sets. Does this mean ...
0
votes
0answers
25 views
Is the use of Nested Cross Validation and train- test CV necessary or an overkill?
I have been relatively obsessed lately in the proper way of selecting a model (including tuning hyper parameters) and then assessing model performance.
I have read various posts and the approach I ...
1
vote
1answer
22 views
How does caret resolve ties in the KNN classification? [closed]
I have a multi-class classification problem, in which I'm using caret package k nearest neighbour classifier, (4 classes), which means that an odd number for k won't prevent classification ties.
So ...
1
vote
0answers
24 views
How is this standard error obtained?
I am working through the exercises in Kuhn and Johnson's "Applied Predictive Modelling" and cannot reproduce one of their results in the exercises.
Looking at 4.3 we have
... find the number of ...
0
votes
0answers
27 views
Can we calculate Variable Importance in Projection (VIP) scores for PLS-DA in R-caret? Is it comparable with coefficients?
Can we calculate Variable Importance in Projection (VIP) scores for PLS-DA in caret (R)? Are VIP scores comparable with PLS-DA coefficients?
1
vote
0answers
26 views
How to select the most important features, categorical & numerical data
I need to find out which factors are relevant when predicting low birth weight. My model looks like this:
...
0
votes
0answers
17 views
Identifying important variables in a PLSDA model using caret in R: are coefficients standardized?
I am doing a PLSDA using the caret package in R. My objective is to predict a status of a cow (0 vs 1) using spectral data. I want to compare the coefficients to know which spectral points contribute ...
1
vote
0answers
18 views
Caret rfe varImp: scaled variable imprtance for rfe results [closed]
I want to plot the scaled variable importance of a rfe object (recursive feature elimination). With the following code I compute the rfe model and the variable importance:
...
0
votes
0answers
30 views
When to use which classification model?
This is something that continues to give me trouble.
Assuming I am working to extract a classification from a dataset and assuming I have the computing resources to do the necessary calculations (in ...
0
votes
0answers
17 views
How train function in caret choose lamda for elastic net
I'm a beginner in elastic net. I'm using following code for elastic net in R
...
0
votes
1answer
17 views
Correct calculation of repeated cross-validation classification metrics
We can obtain a resampled estimate of training set classification accuracy from caret::confusionMatrix.train(model)
e.g.,
...
1
vote
1answer
34 views
How do I avoid time leakage in my KNN model?
I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem.
Data -
...
1
vote
0answers
54 views
What is the basis for the default sigma value used by svmRadial in caret? [closed]
I am looking at the source code (I think) for the "svmRadial" function in the caret package. It looks like the default sigma values are calculated by first using the kernlab package's "sigest" ...
0
votes
0answers
11 views
svm model caret package
I am trying to reproduce the svm model example reported in the original caret paper by khun. But it runs forever and I don't get the output. not even after 2 days. What's wrong?
this is the paper:
...
0
votes
0answers
27 views
textclassification caret(svm): very low accuracy for testset because few words matching with trainingset
I am doing text analysis of tweets with caret ("classif.svm). I have manually classified 1500 documents (consisting of 1-5 tweets). When I tune the parameters it shows me an accuracy of about 80% and ...
0
votes
0answers
19 views
NA and missing values preprocessing within caret with cross-validation
I wanted to use caret package to train a model on some messy data (many NAs and missing values) but I cannot find a solution that would fit to my problem.
While NAs are part of numerical features I ...
1
vote
1answer
28 views
Conflict between predicated outcomes in logistic regression
I was using caret in R to use logistic regression to make prediction. I only have one predictor named OEI and the outcome variable is pass/fail. However, although I was able to perform that task and ...
1
vote
1answer
25 views
Construction of confusion matrix when cross-validating with k-NN in R
I've a dataset looking like this:
...
0
votes
1answer
22 views
How do I run cross validation on a decision tree in an uplift model?
I have this model from the uplift package,
...
3
votes
1answer
77 views
Caret: gradient vs gam boosting
What is the difference between a boosted additive model (e.g. caret model: gamboost) and a general stochastic gradient boosting model (caret model: gbm)?
A gradient boosting model is additive by ...
1
vote
1answer
43 views
What's the purpose of `tunegrid` in the caret package?
I am deciding the parameters for a random forest classification model. I am using the caret package and read, here, about the tuning grid. My understanding is that it helps determine the best values ...
0
votes
0answers
104 views
variable importance for multi-class classification using caret (penalized discriminant analysis")
I have data with three classes and have performed classification using caret (pda). My question is about interpreting predictors based on variable importance (varImp). In two class data, we get ...
0
votes
0answers
42 views
How does multicollinearity affect the feature selection process?
I have a classification problem with a modest number of records (approx. 10,000) and dimensions (30 dimensions, 25 are categoric and 5 are numeric). The response variable has two classes (T/F).
I'm ...
1
vote
0answers
95 views
LDA Fit using Caret does not give Standard Deviation
I am following the steps outlined in this tutorial.
I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy ...
0
votes
0answers
61 views
How do I diagnose collinearity with rfe() from the caret package?
I have 12,000 records and I"d like to predict a two-class outcome. I'm deciding which predictors to keep and I'm having trouble with two problems.
1- I get an error message because I have categories ...
3
votes
2answers
182 views
Can the alpha, lambda values of a glmnet object output determine whether ridge or Lasso?
Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On ...
1
vote
0answers
283 views
Random forest vs Random Forest tuned with caret
Looking for some help please. I have used the randomForest package extensively and have been happy with its performance. I am currently writing a journal paper investigating match outcomes in sport, ...
1
vote
0answers
93 views
Linear discriminant scores using caret
I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict.
The model is ...
...
0
votes
0answers
186 views
SMOTE and ROSE methods in CARET (R) do not work for balancing a multi-class dataset
SMOTE and ROSE methods in CARET (R) do not work for balancing a multi-class data set. Specifically, ROSE throws an error telling that it needs two levels. It means that it does not work for multi-...
0
votes
1answer
66 views
How to do backward subset selection on a random forest model for classification?
I'd like to identify the most predictive features for my classification model. I'm using this data.
Here is a sample.
This is my code. I'd like to use the predictors to predict loan status.
...
0
votes
0answers
65 views
Caret: Feature selection with Chi2 / f_classif
I try to classify texts which I have converted to term-document matrices before. I would like to perform feature selection to reduce the number of predictors. In Python, you can do this by means of ...
0
votes
1answer
83 views
SMOTE - What is the difference in sampling before or inside train() [closed]
I have an unbalanced dataset and would like to apply SMOTE to the training data. I can either do one of the following:
Inside trainControl() add ...
0
votes
1answer
63 views
Differences in calibration plots for machine learning models
I'm using machine learning methods in R for descriptive regression modelling of a small dataset. I have fit random forest (randomForest), unbiased random forest (cforest) and boosted regression trees (...
1
vote
0answers
30 views
How to approach memory issues with up/down-sampling problem across millions of rows in database that can't be loaded locally? (class imbalance)
I'm faced with fairly typical class imbalance problem across a dataset with nearly 9MM rows (hard drive failures) that's not stored locally (it's in Postgres table; downloading a .csv of it is not ...
0
votes
0answers
89 views
how to extract rules from final model made by caret
I have a made cross-validation (k=5) by caret package using C5.0 method.
I have 21 features and 7000 instances. The C5.0 trials default is 40.
The problem is C5.0 made > 1600 rules over 40 trials, ...
0
votes
0answers
40 views
k-fold optimization in C5.0 algorithm
How can I do k-fold cross validation for C5.0 algorithm, I know caret package has createFolds function but I think in k-fold process we must do kind of averaging from all models.
Because I didn't ...
0
votes
0answers
66 views
Using Caret is there a way to explicityly state what the true class is in binary classification?
I'm using caret + XGBoost to make binary predictions on a dataset. The data are very imbalanced so I'm using prAUC metric.
My results using 5 fold cross validation are a prAUC of 0.99+ which is not ...
0
votes
0answers
66 views
caret chooses non-optimal RMSE?
I run a linear regression via caret / glmnet method with "RMSE" as metric. In the final model, caret tells me which values of the tuning parameters alpha and lambda were selected to minimize RMSE.
If ...
0
votes
0answers
109 views
what does scale hyperparameter mean in svm polynomial kernel using kernlab in r
I'm trying to train my svm model with polynomial kernel. I'm using caret package with method ...
0
votes
0answers
69 views
How does Caret's PCA function work?
I've been reading through this question: PCA and k-fold cross-validation in caret package in R . In one of the answers, it was suggested to do PCA within the train function rather than before.
However,...
0
votes
1answer
771 views
How can I one-hot encode a variable that has only 2 levels? [closed]
I'm trying to do OHC in R to convert categorical into numerical data. However R's caret package requires one to use factors with greater than 2 levels. Any idea how to go around this? I've searched ...
2
votes
1answer
283 views
Parameter tuning with vs without nested cross-validation
Disclaimer: This question has been inspired by this one, which is a good question but has unfortunately not attracted an answer that actually answers OPs question.
Statistical models often times have ...
1
vote
2answers
477 views
dummy vs one-hot encoding - ML for prediction
I understand there is a lack of consensus in the difference (if any) between one-hot (k variables) and dummy (k - 1 variables) encoding from a k-level factor.
The caret package seems to auto-encode ...
0
votes
0answers
167 views
Repeated CV evaluation with confidence intervals in R caret?
it occurs to me that there is a part of model evaluation that I have not understood yet. The problem that I am working on now illustrates the point well I think.
I need to fit a model of >400 ...
0
votes
0answers
128 views
Difference between AUPRC in caret and PRROC
I'm working in a very unbalanced classification problem, and I'm using AUPRC as metric in caret. I'm getting very differents results for the test set in AUPRC from caret and in AUPRC from package ...
