Questions tagged [caret]
Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.
448
questions
0
votes
1answer
22 views
Cross-validation using Caret in R: Why are coefficients from FinalModel identical to those from lm()?
I think I must be missing some fundamental part of the logic of cross-validation, or machine learning in general.
Using the caret package in R, I ran a repeated k-...
2
votes
0answers
14 views
Which R-Squared formula is used in carets' k-fold-cross validation function
I am currently trying to implement a linear model and calculate some out-of-sample accuracy metrics while using k-fold-cross validation. I try to do this using R's caret package, so this question ...
0
votes
0answers
27 views
Interpret varimp() with Caret LDA model
I'm using caret's train() function for a binary classification outcome with an lda models, based on several features
...
1
vote
0answers
41 views
Drop unused parameters from model?
I have trained a model using caret's train function in R (LASSO logistic regression with glmnet, repeatedcv). The goal is both feature selection and getting the model. With the preferred lambda, LASSO ...
1
vote
0answers
19 views
KNN and Classification Factor
I'm studying the KNN algorithm and its application on R. If I correctly understood, KNN is a supervised algorithm able to classify an unlabeled item according to the predominant belonging class in the ...
1
vote
1answer
18 views
What really is 'glmnet' when used in caret in R for binary classification?
like lasso and ridge, elastic net can also be used for classification
by using the deviance instead of the residual sum of squares. This
essentially happens automatically in caret if the response ...
7
votes
2answers
139 views
How should two cross-validated logistic regression models be compared?
I'm using 100 times 10-fold repeated cross-validation to assess the ROC-AUC performance improvement of adding a biomarker to an existing model:
Model_A : pred1 + pred2
Model_B :pred1 + pred2 + pred3
I'...
1
vote
0answers
44 views
2
votes
1answer
39 views
Model stacking, Super Learner Algorithm
I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...
0
votes
1answer
21 views
Confusion Matrix for Multi-class Variables using cross-validation
my dataset includes multi-class variables (11 different variables). All of the columns are numeric except the last column which is the label ( Last Column Name = Movement, 11 different types, Type = ...
0
votes
0answers
46 views
Permutation Testing Machine Learning Model Performance With Caret in R
Using the caret package in R, classification machine learning model performance is usually assessed using the confusionmatrix() function, which uses a Binomial Test ...
0
votes
0answers
11 views
What can go wrong if we don't scale and center data in a predictive model? [duplicate]
I have started using R recently, along with the caret package. I note that there is an option to preprocess data so that it is scaled and centered.
I am interested in multiple linear regression. What ...
0
votes
0answers
38 views
Random Forest doesn't converge in caret
I'm using some synthetic data to try and understand the performance of different ML algorithms on my real data. I'm finding that RF is consistently overfitting the training data even though its ...
1
vote
1answer
27 views
What's random in Caret if the hold out is fixed for cross validation [closed]
I tried to fix the folds for cross validation in Caret. But there is still some randomness returned by the train function. See code below. Don't they should return the same?
...
2
votes
1answer
47 views
RFE: Pre-define a specificity threshold
I would like to use recursive feature elimination (implemented via caret in R) to perform feature selection for about 40 test results with 2 possible outcomes. Consequently, RFE either models by ...
1
vote
1answer
90 views
How do I calculate confidence intervals on an elastic net regression in R
I am performing an elastic net regression on my data n = 34, p = 46
I first built the model using the "caret" package with the cross validation method to set the optimal alpha and lambda ...
2
votes
0answers
61 views
Elastic net: how to tune for both lambda and alpha? Why is caret better than cv.glmnet?
I'm learning how to use R to fit Elastic Net models using the glmnet package. There appears to be a function, cv.glmnet that ...
0
votes
0answers
61 views
Using Cross-Validation and regsubset for logistic /multinomial.(Caret/Leaps)
I've been reading some interesting posts about caret, and the function from leaps regsubsets; and i was wondering if i could implement a regsubset on a CV-trainControl from caret.
https://rpubs.com/...
0
votes
0answers
25 views
Why is Rsquared smaller in whole sample than k-fold cross-validated?
I want to get a cross-validation $R^2$ for a linear regression model with one predictor to get a measure of how well my model predicts rather than explains data (in R with package caret). I assumed ...
0
votes
1answer
44 views
What does the featurePlot() function from the caret package actually do?
Just a quick example from Max Kuhn's book.
...
0
votes
0answers
23 views
Can I use caret RFE result for subsequent random forest with CV?
I've been googling and reading alot about my issue but couldn't find a clear answer.
In order to prevent data leakage, I use caret RFE for feature elimination:
...
1
vote
0answers
20 views
How to apply svm to a dataset with a numerical (not categorical) dependent variable?
Context: I have some band values from a sentinel - 2 derived .tiff
I now want to make a prediction regarding areas where i have no actual field data
i.e. make a carbon map of sorts
Libraries are:
<...
1
vote
0answers
47 views
is a final model different from cross validation?
It appears that I have completely misunderstood cross validation for several months so I want your help clarifying the idea using an example rather than only theory from some great SE questions. My ...
1
vote
2answers
33 views
How does repeated cross validation go about 'averaging' model coefficients?
No matter how much I google, I cannot find the answer to this simple question.
Say you do 10-fold, repeated (5x) CV logistic regression with elastic net regularization.
For alpha you try ...
1
vote
0answers
11 views
Weighted observations for linear model [closed]
I have dataset with sale price of products in years 2015-2020. I would like use it to predict future sale prices. I would like to use sale year as a weight of observation (eg. for 2020-100%, for 2019-...
1
vote
1answer
22 views
How can I implement 5 times repeated 10 fold cross validation using the randomForest package ( instead of caret)?
I would like to eventually use the PIMP-Algorithm (Permutation Variable Importance Measure) in order to get p values for the variables' importance. However, the formula
...
0
votes
1answer
22 views
Difference Between Built-In Cross Validation Functions and Using Caret
I was wondering if someone could provide some insight on the pros/cons of using built-in cross validation functions like cv.glmnet (https://www.rdocumentation.org/packages/glmnet/versions/3.0-2/topics/...
0
votes
0answers
29 views
How to group factor levels for stepwise regression using caret
Using the train() function from caret in R, I'm trying to run a stepwise ANCOVA, but each level of my 9-level factor is being ...
1
vote
0answers
71 views
How to calculate variable importance for different models? is varImp() the solution?
I'm using caret's train() function for a binary classification outcome with different models (nb, knn, lda, qda, glm, rpart, rf).
I'm using varImp() and plots to determine the importance of every ...
1
vote
1answer
17 views
Importance value (with varImp from carret package) for one of the two numerical predictors has value 100, how do I interpret this?
I'm using two numerical predictors to find an outcome, when using varImp (from the carret package) one of the predictors has 100 importance and the other 0.
How should I interpret this?
1
vote
1answer
58 views
Why can't my ROC reach a low PPV?
I am creating a couple of models (RF, SVM, LR) and I want to evaluate all of them on a certain PPV (0.7).
This question and this question helped me write my code:
...
0
votes
0answers
37 views
Multi-classification: low precision due to imbalanced classes in test data - what to do?
I built a multi-classification model with 3 result classes (XGBoost using R's caret-package): A, B and C. I undersampled my training data - so every class is equally abundant for training. The ...
0
votes
0answers
17 views
Kappa for More than Two Classes in R?
I am building predictive models in R using the oil dataset which has 7 classes. I am wanting to use Kappa for optimization since the classes are extremely ...
0
votes
0answers
7 views
Caret for model performing and tuning paramter
I would like to ask, if I could use cross validation in caret for tuning parametr lambda and also for evaluating model performance. If I use savePredictions="final" and also I will tune parametr, it ...
1
vote
1answer
93 views
Help testing the predictive quality of a binomial GLM (currently attempting using the “caret” package)
Hello world (sorry for the novel; if you read this, I appreciate it!),
I'm running into a question that is probably a mixture of how to approach a problem of modeling and the technical difficulties ...
1
vote
1answer
227 views
LOOCV in Caret package ( randomForest example) - not unique results
I pose you my doubts:
For what I know there is only a single way to perform a LOOCV for a model (i.e. testing each one of the N elements vs the model trained with the other N-1 elements).
Namely, ...
0
votes
1answer
22 views
Caret GBM predict only produces 106 outputs while newdata has 403 rows… What's up?
I am training ML classifiers using caret to predict mortality in a clinical data set. Training with caret's gbm works well, but when I try to use predict, I get very strange results. Here is my code:
...
1
vote
1answer
106 views
Using caret::sbf to apply feature selection where features are selected over different threshold scores
I'm aiming to use caret::sbf to filter a large number of predictors before using different machine learning models to predict a binary outcome. I would like to filter for variables that are identified ...
1
vote
1answer
31 views
Predict different models from K Best models in Caret
I was wondering if there was a way to run predictions on the K best Models and not just the best Model in Caret.
...
1
vote
1answer
34 views
Expanding the Hyper-Parameter Search Grid in Caret
I was wondering if there was a way to expand the hyper-parameter search within caret package or with slight modification. For example, evtree can currently only take alpha in caret but the evtree....
0
votes
2answers
86 views
In a neural network, why can't there be more weights than the number of observations?
After having this exact same issue with caret, I arrived at this thread. However, I do not intuitively understand why this answer is correct.
Why can't there be ...
2
votes
1answer
64 views
Predicted probabilities seem too low with Gradient Boosting Machine on `iris` data
I'm doing a test run of the Gradient Boosting Machine algorithm on the iris data with the caret package.
...
1
vote
1answer
67 views
What method does varImp.gam use in caret package in R?
I see that the caret package has support for gam objects for the varImp function. I was wondering if there was documentation about which method the function uses when gam is the input?
0
votes
0answers
101 views
Result reproducibility using time series cross validation with Caret in R
I'm using Caret package in R to train a Lasso regression on a time series dataset. My problem is that even if I set a seed before the ...
0
votes
1answer
176 views
Using caret::sbf to apply feature selection and classification
I'm aiming to use caret::sbf to filter a large number of predictors before using different machine learning models to predict a binary outcome. I would also like to optimise tuning parameters and do ...
1
vote
2answers
729 views
Multi- Class probabilities of Random Forest inside caret Model
Im facing a problem with the results of a multi-class random forest model.
I want to use a) the predictions of the model and b) the class probabilities of these predictions for further work.
I did a ...
0
votes
0answers
18 views
Techniques to account for differences in misclassification “cost” on variables other than the outcome
Suppose you're in a classic classification context: you want to predict whether a patient has a certain virus. You are working in multiple regions (let's say 2 for simplicity: Region A and Region B) ...
2
votes
1answer
103 views
R: caret (elasticnet): ridge regression: understanding the returned parameters
I wanted to play around with the ridge regression in caret (which apparently uses elasticnet), so I did two experiments:
use the original data
use the modified data where the values of ...
1
vote
1answer
114 views
Machine learning with univariate time series
I am trying to make predictions with daily data with time series in R.
This time series is univariate and contains only data from sales from each 365 days in a four year period.
My intention is to ...
1
vote
0answers
24 views
Can we compare class probabilities between different methods in caret?
I am using the caret package in R for binary classification and I want to compare different methods (e.g., random forest, SVM, ANN, PLS-DA...). I consider the class probabilities as a "certainty score"...
