Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.

Filter by
Sorted by
Tagged with
2
votes
1answer
21 views

Why do the results of LASSO regression differ after removing uninformative variables in glmnet?

I am researching therapy response of melanoma patients based on a number of approximately 80 features with a very small sample size of 60 patients. To eliminate features that do not contribute to the ...
0
votes
0answers
19 views

Classification problem. I have issues with overfitting/leakage, i.e. sensitivity in training is 1.0. How do I fix this?

I have a consumer dataset (size: 10200x180), I am using this to develop a training model to predict consumers who would provide a referral. I have done the following to set the model. Cleaning: I have ...
1
vote
1answer
34 views

Unreasonable bias when using nnet (R package caret) for time series forecasting

I have been trying to forecast a time series in a regression-like setting using neural networks (nnet method in R package caret)....
0
votes
1answer
12 views

ML test accuracy higher than training? Small unbalanced samples were stratified by class

My background is in ecology, it is common to have smaller sample sizes and class imbalances and ML approaches are still increasingly adopted. My specific dataset: training set is 49 sample, my test ...
0
votes
1answer
22 views

Impute missing values of dummy variables, using R's {caret} package: predicted values in between {0;1}?

I'm using {caret} to impute missing data resulting from non-response to survey questions. All of these variables are defined as numeric, though most are dummies. ...
0
votes
0answers
12 views

How to perform Multiple Imputation (MI) for longitudinal survey data, using {caret} in R?

I have a large dataset comprising survey responses throughout a time period of three to four decades (or in other words, slightly above 20 waves). Keep in mind that these are longitudinal data and not ...
1
vote
0answers
38 views

Extreme collinearity for Random Forest?

My data have 450 observations and 2200 predictors that I want to use to train a RF model to classify 4 classes. However, about 2165 of the predictors are very highly correlated to each other. The way ...
0
votes
1answer
45 views

Variable Importance for Caret Random Forest Regression

I have trouble understanding the exact meaning of the feature importance scores in caret for RF regression. As you know there are many potential importance measures for RF. However, there is no clear ...
0
votes
0answers
44 views

Interpretation of AUC - ROC curves with a Binary Predictor

I have data like this: ...
0
votes
0answers
13 views

Machine learning methods for filtering noisy data

I have my data (hear beat) cleaned up with the appropriate filter, however, they are still noisy and need to extract a clean pattern. Which is the best algorithm for this purpose? I have also clean ...
0
votes
1answer
80 views

Model building with 10-fold validation

I have yet to find a sufficient and succinct answer regarding model building with 10-fold cross validation (in this case, using Caret). I've found responses here, for instance: https://stackoverflow....
0
votes
0answers
44 views

How to manually calculate predictions of kernlabs SVM

I am trying to manually replicate the predictions of kernlabs SVM (polynomial & radial kernel) using caret. Here is the code to fit the model: ...
1
vote
1answer
46 views

How many models does caret::trainControl() actually train?

I searched around CV and the caret documentation but couldn't find an answer to this. I'm using caret in R to tune some random forests for classification trained ...
0
votes
1answer
145 views

Cross-validation using Caret in R: Why are coefficients from FinalModel identical to those from lm()?

I think I must be missing some fundamental part of the logic of cross-validation, or machine learning in general. Using the caret package in R, I ran a repeated k-...
2
votes
0answers
47 views

Which R-Squared formula is used in carets' k-fold-cross validation function

I am currently trying to implement a linear model and calculate some out-of-sample accuracy metrics while using k-fold-cross validation. I try to do this using R's caret package, so this question ...
0
votes
0answers
41 views

Interpret varimp() with Caret LDA model

I'm using caret's train() function for a binary classification outcome with an lda models, based on several features ...
1
vote
0answers
41 views

Drop unused parameters from model?

I have trained a model using caret's train function in R (LASSO logistic regression with glmnet, repeatedcv). The goal is both feature selection and getting the model. With the preferred lambda, LASSO ...
1
vote
0answers
22 views

KNN and Classification Factor

I'm studying the KNN algorithm and its application on R. If I correctly understood, KNN is a supervised algorithm able to classify an unlabeled item according to the predominant belonging class in the ...
1
vote
1answer
133 views

What really is 'glmnet' when used in caret in R for binary classification?

like lasso and ridge, elastic net can also be used for classification by using the deviance instead of the residual sum of squares. This essentially happens automatically in caret if the response ...
7
votes
2answers
194 views

How should two cross-validated logistic regression models be compared?

I'm using 100 times 10-fold repeated cross-validation to assess the ROC-AUC performance improvement of adding a biomarker to an existing model: Model_A : pred1 + pred2 Model_B :pred1 + pred2 + pred3 I'...
2
votes
1answer
51 views

Model stacking, Super Learner Algorithm

I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...
0
votes
1answer
76 views

Confusion Matrix for Multi-class Variables using cross-validation

my dataset includes multi-class variables (11 different variables). All of the columns are numeric except the last column which is the label ( Last Column Name = Movement, 11 different types, Type = ...
0
votes
0answers
79 views

Permutation Testing Machine Learning Model Performance With Caret in R

Using the caret package in R, classification machine learning model performance is usually assessed using the confusionmatrix() function, which uses a Binomial Test ...
0
votes
0answers
12 views

What can go wrong if we don't scale and center data in a predictive model? [duplicate]

I have started using R recently, along with the caret package. I note that there is an option to preprocess data so that it is scaled and centered. I am interested in multiple linear regression. What ...
0
votes
0answers
65 views

Random Forest doesn't converge in caret

I'm using some synthetic data to try and understand the performance of different ML algorithms on my real data. I'm finding that RF is consistently overfitting the training data even though its ...
1
vote
1answer
45 views

What's random in Caret if the hold out is fixed for cross validation [closed]

I tried to fix the folds for cross validation in Caret. But there is still some randomness returned by the train function. See code below. Don't they should return the same? ...
2
votes
1answer
52 views

RFE: Pre-define a specificity threshold

I would like to use recursive feature elimination (implemented via caret in R) to perform feature selection for about 40 test results with 2 possible outcomes. Consequently, RFE either models by ...
1
vote
1answer
249 views

How do I calculate confidence intervals on an elastic net regression in R

I am performing an elastic net regression on my data n = 34, p = 46 I first built the model using the "caret" package with the cross validation method to set the optimal alpha and lambda ...
2
votes
0answers
125 views

Elastic net: how to tune for both lambda and alpha? Why is caret better than cv.glmnet?

I'm learning how to use R to fit Elastic Net models using the glmnet package. There appears to be a function, cv.glmnet that ...
0
votes
0answers
27 views

Why is Rsquared smaller in whole sample than k-fold cross-validated?

I want to get a cross-validation $R^2$ for a linear regression model with one predictor to get a measure of how well my model predicts rather than explains data (in R with package caret). I assumed ...
0
votes
1answer
75 views

What does the featurePlot() function from the caret package actually do?

Just a quick example from Max Kuhn's book. ...
0
votes
0answers
30 views

Can I use caret RFE result for subsequent random forest with CV?

I've been googling and reading alot about my issue but couldn't find a clear answer. In order to prevent data leakage, I use caret RFE for feature elimination: ...
1
vote
0answers
27 views

How to apply svm to a dataset with a numerical (not categorical) dependent variable?

Context: I have some band values from a sentinel - 2 derived .tiff I now want to make a prediction regarding areas where i have no actual field data i.e. make a carbon map of sorts Libraries are: <...
1
vote
0answers
50 views

is a final model different from cross validation?

It appears that I have completely misunderstood cross validation for several months so I want your help clarifying the idea using an example rather than only theory from some great SE questions. My ...
1
vote
2answers
53 views

How does repeated cross validation go about 'averaging' model coefficients?

No matter how much I google, I cannot find the answer to this simple question. Say you do 10-fold, repeated (5x) CV logistic regression with elastic net regularization. For alpha you try ...
1
vote
0answers
12 views

Weighted observations for linear model [closed]

I have dataset with sale price of products in years 2015-2020. I would like use it to predict future sale prices. I would like to use sale year as a weight of observation (eg. for 2020-100%, for 2019-...
1
vote
1answer
34 views

How can I implement 5 times repeated 10 fold cross validation using the randomForest package ( instead of caret)?

I would like to eventually use the PIMP-Algorithm (Permutation Variable Importance Measure) in order to get p values for the variables' importance. However, the formula ...
0
votes
1answer
25 views

Difference Between Built-In Cross Validation Functions and Using Caret

I was wondering if someone could provide some insight on the pros/cons of using built-in cross validation functions like cv.glmnet (https://www.rdocumentation.org/packages/glmnet/versions/3.0-2/topics/...
0
votes
0answers
65 views

How to group factor levels for stepwise regression using caret

Using the train() function from caret in R, I'm trying to run a stepwise ANCOVA, but each level of my 9-level factor is being ...
2
votes
0answers
137 views

How to calculate variable importance for different models? is varImp() the solution?

I'm using caret's train() function for a binary classification outcome with different models (nb, knn, lda, qda, glm, rpart, rf). I'm using varImp() and plots to determine the importance of every ...
1
vote
1answer
18 views

Importance value (with varImp from carret package) for one of the two numerical predictors has value 100, how do I interpret this?

I'm using two numerical predictors to find an outcome, when using varImp (from the carret package) one of the predictors has 100 importance and the other 0. How should I interpret this?
1
vote
1answer
59 views

Why can't my ROC reach a low PPV?

I am creating a couple of models (RF, SVM, LR) and I want to evaluate all of them on a certain PPV (0.7). This question and this question helped me write my code: ...
0
votes
0answers
47 views

Multi-classification: low precision due to imbalanced classes in test data - what to do?

I built a multi-classification model with 3 result classes (XGBoost using R's caret-package): A, B and C. I undersampled my training data - so every class is equally abundant for training. The ...
0
votes
0answers
24 views

Kappa for More than Two Classes in R?

I am building predictive models in R using the oil dataset which has 7 classes. I am wanting to use Kappa for optimization since the classes are extremely ...
0
votes
0answers
12 views

Caret for model performing and tuning paramter

I would like to ask, if I could use cross validation in caret for tuning parametr lambda and also for evaluating model performance. If I use savePredictions="final" and also I will tune parametr, it ...
1
vote
1answer
186 views

Help testing the predictive quality of a binomial GLM (currently attempting using the “caret” package)

Hello world (sorry for the novel; if you read this, I appreciate it!), I'm running into a question that is probably a mixture of how to approach a problem of modeling and the technical difficulties ...
1
vote
1answer
706 views

LOOCV in Caret package ( randomForest example) - not unique results

I pose you my doubts: For what I know there is only a single way to perform a LOOCV for a model (i.e. testing each one of the N elements vs the model trained with the other N-1 elements). Namely, ...
0
votes
1answer
38 views

Caret GBM predict only produces 106 outputs while newdata has 403 rows… What's up?

I am training ML classifiers using caret to predict mortality in a clinical data set. Training with caret's gbm works well, but when I try to use predict, I get very strange results. Here is my code: ...

1
2 3 4 5
10