Questions tagged [caret]
Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.
494
questions
2
votes
0
answers
12
views
Variable importance for random forests using caret
Consider the following toy example dataset (in R), including 5 factors:
...
- 109
1
vote
0
answers
21
views
Poorly calibrated probabilities but good classification in confusion matrix
I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/...
- 225
0
votes
0
answers
21
views
Changing Reference Levels of Categorical Variables Changes Confusion Matrix & Prediction Probabilities
I am trying to understand why changing the reference level of a factor changes the results of a model. Consider this example:
...
- 225
1
vote
0
answers
22
views
What is the difference between lm() function and caret::train() function when it comes to creating linear regression models? [duplicate]
When applying the lm function as follows (the assumptions were not considered. The purpose of this example is just to make my question clear) :
...
- 53
1
vote
0
answers
21
views
Formula versus Non-Formula Interface Categorical Variables train() glmnet
I am comparing the confusion matrix between the formula interface and the non-formula interface using caret's train() for elastic net. I am trying to understand why the two interfaces produces ...
- 225
2
votes
2
answers
90
views
What is the interpretation of the "traditional" $R^2$?
Suppose the following data correspond to observed responses and their predictions obtained from some model.
...
- 1,737
0
votes
0
answers
47
views
An aggressive overfitting situation
I gather RNA-seq transcriptomic data from multiple cancer datasets. The datasets are about a treatment of cancer, we check Response vs NoResponse samples.
The RNA-seq data I gather is before the ...
- 161
2
votes
0
answers
55
views
Train-Test Split with nested groups and multiple balancing factors
I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
- 41
0
votes
1
answer
38
views
Why predicted values differ in knn regression when using caret vs FNN?
I was trying to do some manual calculations of knn regression and came across this unusual error. The predicted values done by hand do not match with the ones I got from the 'knnreg' function in the '...
- 31
0
votes
0
answers
17
views
How RFE method make the importance plots?
How does the Caret package make the "importance" plots?
From folds and repeats of CV, or from the full training data?
I am thinking that the RFE works as follows and is not using the CV data ...
- 1
0
votes
0
answers
25
views
Coversion of MSE to percentage in importance plots in the Caret package
When we make an RFE model and we look at variable importance values they are the MSE values like 2.70, 2.31, 1.75, etc. When we run "varImp(model$fit)", we will see 100, 73.27, 68.71 etc., ...
- 1
1
vote
1
answer
43
views
Using ML model and decision tree to create a new risk classification
The idea of this project was to use a Machine Learning model to find the best variables to include in a decision tree algorithm. After evaluating with caret a number of different models I found the ...
- 21
0
votes
0
answers
24
views
Caret: ordinal models with VGAM library
I started to look into the students performance dataset:
https://archive.ics.uci.edu/ml/datasets/Higher+Education+Students+Performance+Evaluation+Dataset
I would like to train two models with caret ...
- 101
0
votes
0
answers
145
views
Bagging with SVM and Neural Networks in R with caret
I am fairly new to the bagging technique and Caret's bagControl() as well as bag() and am currently trying to build an ensemble ...
- 1
0
votes
0
answers
61
views
How does lmStepAIC work in caret when using cross-validation?
In caret package in R , one can train linear models with stepwise selection based on AIC, using this function :
...
- 253
0
votes
0
answers
80
views
mgcv GAM models in R package caret - how to interpret output
I am attempting to evaluate two GAM models I developed in mgcv via leave-one-out cross validation in the caret package. I am a newbie to both GAMs and cross-validation. For the purposes of this ...
- 11
0
votes
0
answers
26
views
How to interpret the output of compare_models in R?
I have been trying caret library, specially the part to compare two or more obtained models. Some of the articles that I read use the compare_models function. As I see on the documentation, it runs a ...
- 101
0
votes
0
answers
35
views
Sample size problems during Random Search Hyperparameter Tuning
I would like to apply the XGBoost algorithm to my data using the xgbTree method in caret.
My ...
- 1
1
vote
1
answer
122
views
Compare survival between two unbalanced groups
Briefly, 184 patients are included in my analysis. I have one variable that seperates 184 patients into two groups. 173 are in group 0 and 11 are in group 1. I need to compare the survival between ...
- 13
0
votes
0
answers
46
views
How should we divide the data into training and testing sets in a specific way instead of probabilities of 70-30%?
The dataset contains 5 stages of the disease. The respective blood concentrations are 12.5, 25, 50, 62, and 75 mg/ml. The intensity is measured at each concentration, which will be used as a response ...
2
votes
1
answer
86
views
How to use R package caret to build a model and get the internal validation result?
I am a learner of R and machine learning. I don't really understand caret's train function. To make it simple, for example, I want to build a model and get the internal validation result.
At the ...
0
votes
0
answers
9
views
Problem predicting caret::train random forest ("rf") model [duplicate]
I´ve been working in a random forest model for credit scoring in R. I've trained a model using caret::train.
My data "df_samples_rf" has the next ...
Mariano Stone
1
vote
1
answer
75
views
Incorporate Weights/Offsets with Nonparametric Models
I am modeling pure premium in R. I have read that pure premiums are usually modeled using a Tweedie distribution (glm). There is generally an offset or weight added to the model, such as an exposure. ...
- 225
0
votes
0
answers
17
views
Predictors for Classification Models using caret in R
I have been reading several different resources on classification and I am finding conflicting information. Some literature says caret's train() function assumes all predictors are numeric and the ...
- 225
0
votes
0
answers
45
views
How do I determine what subset sizes to use for recursive feature elmination (rfe) in the R caret package?
I'm trying to use rfe in the caret package in R, but I'm not sure I understand how to choose the subset sizes. Is it just trial and error or am I overlooking something?
For context: I have high ...
- 59
1
vote
1
answer
310
views
How to interpret the output of caret::findCorrelation function?
The output I received after applying findCorrelation function from the caret package is:
...
- 45
0
votes
1
answer
145
views
How do I report results of an internal validation in Caret?
I have the following question. In a machine learning project I have to solve a regression and a classification task. See also:
Hold-Out VS Cross-Validation - R caret
For this I have about ~650 cases ...
- 41
0
votes
0
answers
197
views
How does caret measure Accuracy and Kappa?
I have trained a logistic elastic net regression using caret package and the method "glmnet", with trainControl set to repeated cross-validation.
I don't know how to interpret the Accuracy ...
1
vote
0
answers
57
views
how do I perform permutation testinging for a prediction model developed within caret package (R)?
I'm fairly new to data science/StackExchange, so please excuse any faux pas
I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
- 21
0
votes
0
answers
124
views
Clarification on r caret package confusionmatrix
I just want to make sure I'm understanding the "confusionmatrix" function in the "caret" package correctly. Looking at the worked example found here:
https://rpubs.com/dtime/...
- 1
0
votes
0
answers
130
views
How to get probabilities from KNN with cross-fold validation using caret::train in R
I am confused about the output of train() with knn models in R. My code below uses the Caravan data set, which comes included in the ISLR2 library:
...
- 11
1
vote
1
answer
127
views
How do I access the p-values of individual predictors using caret::train? [closed]
I can't figure out how to access the p-values for my predictor variables after using k-fold cross-validation with caret::train. Does anyone know? Below is an example using the Boston data set that ...
- 11
0
votes
0
answers
216
views
Recursive Feature elimination for Xgboost and SVM in R
I would like (and need) to use RFE for a dataset with many predictors. Some of them are correlated but I would like the model to pick the best choice.
It is a classification problem and I will ...
- 45
0
votes
0
answers
441
views
R Caret - Random Forest with repeated CV
I am currently struggling to explain the combination of bagging and cross-validation. I am aware of what each does separately but I have difficulties to explain how they are combined (if at all).
For ...
- 1
1
vote
0
answers
127
views
0
votes
0
answers
11
views
Does Breiman's randomForest function in R use Gini score or entropy score or Classification Error rate to decide the splits? [duplicate]
I have read the R documentation on randomForest but I could not find anything about Gini or Entropy in it.
Is there a way to direct the train function of caret to use Gini score?
- 101
2
votes
1
answer
171
views
Highly Correlated Datasets - Why and What Next?
I've got three datasets (of different biological data) that are highly correlated - such that if I use the typical findCorrelation from the ...
- 21
0
votes
1
answer
231
views
How does glmnet in caret choose the values of lambda and how does it compute coefficients of the model?
I have a question that I've been struggling with. My students are asking me, but I can't figure it out myself.
When I train LASSO regression in R caret, I use the method "glmnet" and a grid ...
0
votes
1
answer
138
views
Best model (set of predictors) for my data?
I'm exploring some ML strategies using caret package. My goal is to select best predictors and to obtain optimal model for further predictions. My dataset is:
75 ...
- 55
0
votes
0
answers
69
views
standard deviation of the model R2 in LOOCV in caret
I am performing a LOOCV linear model and I got the parameters R2 and RMSE, but I was wondering if there is a way to calculate the standard deviation of the model R2. I tried to do it in the same way I ...
- 21
0
votes
0
answers
80
views
Caret classifying above chance on randomly generated data
I am attempting to compare a few methods for multi-class classification using caret: 'multinom' (logistic regression), 'nnet' (neural net), and 'svmPoly' and 'svmLinear' (two types of support vector ...
0
votes
0
answers
32
views
How to plot ROC and Recall-precision curves for this obtained data
Good afternoon ,
Assume we have this obtained data : ( I had trained a SOM map from scratch then i used a number of epochs, Each epoch consist of a number of iterations ) :
...
- 113
1
vote
0
answers
156
views
choosing most important predictors for logistic regression
I have a dataset of cars with price label as binary outcome including "affordable" and "costly". I aim to ...
- 11
3
votes
0
answers
174
views
How to determine the correct amount of oversampling (in regression)
For my regression problem I splitted my dataset into a stratified train- and testset (70:30) and I now want to train my models (random forest, gbm, logistic regression) using the trainset. The dataset ...
- 31
1
vote
1
answer
689
views
Caret "Metric RMSE not applicable for classification models" when data is continuous
I am running into an error trying to train a caret model with method='glmnet.' My data is continuous and I am trying to do a LASSO regression, but the existence of a 0 for one of the observations is ...
- 13
0
votes
1
answer
111
views
2
votes
0
answers
59
views
Using cross-validation for model selection and model comparison
Let us suppose that we have two classifiers: SVM and CART. For each one of them, a set of hyperparameters is considered (C=0.001,0.01,... for SVM cp=... for CART).
The question is, can I use k-fold ...
3
votes
3
answers
197
views
Why do the results of LASSO regression differ after removing uninformative variables in glmnet?
I am researching therapy response of melanoma patients based on a number of approximately 80 features with a very small sample size of 60 patients. To eliminate features that do not contribute to the ...
- 33
1
vote
1
answer
107
views
Unreasonable bias when using nnet (R package caret) for time series forecasting
I have been trying to forecast a time series in a regression-like setting using neural networks (nnet method in R package caret)....
- 13
0
votes
1
answer
27
views
ML test accuracy higher than training? Small unbalanced samples were stratified by class
My background is in ecology, it is common to have smaller sample sizes and class imbalances and ML approaches are still increasingly adopted. My specific dataset: training set is 49 sample, my test ...
