Skip to main content

Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models. Use this tag for any on-topic question that (a) involves caret either as a critical part of the question or expected answer, & (b) is not just about how to use caret or R.

Filter by
Sorted by
Tagged with
2 votes
1 answer
72 views

How do I perform a permutation test on a machine learning model to obtain a p-value for its performance?

this is kinda of the same question of this previous post. But since there's no reply, and I'm having a hard time to find some answers, I'd like to ask it again. I'm training a regression model (SVM ...
artvmac's user avatar
  • 73
3 votes
2 answers
173 views

Fitting multiple linear regression models to select molecules for which a feature of interest significantly alters concentration

I'm using data from a proteomics platform called Olink. In this case, the data comes from samples of 16 patients and 16 controls. The patients can be further classified into 2 subgroups. The assay ...
maglorismyspiritanimal's user avatar
1 vote
0 answers
25 views

What is the difference between model\$pred and model\$finalModel\$votes in a random forest model trained by caret?

I trained a random forest model as below: ...
Robin's user avatar
  • 119
0 votes
1 answer
47 views

What can I do about model tuning parameter instability?

I am trying to determine the importance of watershed characteristics on the slope of the concentration-discharge relationship for several rivers. I am using partial least squares regression (plsr) ...
Ryan's user avatar
  • 1
0 votes
0 answers
29 views

Opinion about conversion of factor to numeric variable during model development using caret package

caret package automatically converts factor variables to one-hot encoding. We can also convert the factor variable to a numeric variable before training any model. ...
UseR10085's user avatar
  • 107
0 votes
0 answers
30 views

How to determine cut-off value for binomial glmm?

I have a dataset composed of an urbanization index (continuous variable) for multiple host species (random effect). I want to determine if there is a relationship between urbanization and disease ...
Amanda Goldberg's user avatar
6 votes
4 answers
1k views

Is it really so bad to do SMOTE on the training set before crossvalidation?

I understand that doing this leads to data leakage, but if I get better performance on the test set does it really matter? I tried using caret with ...
maglorismyspiritanimal's user avatar
0 votes
1 answer
45 views

Dealing with Negative Forecasts in Positive-Valued Datasets

I am forecasting a dataset with positive values in R using the Caret package, which applies machine learning algorithms. The model forecasted the good data, but I am encountering an issue with the ...
silent_hunter's user avatar
1 vote
0 answers
42 views

Lasso model by glmnet package in small sample size [closed]

I want to create a lasso model, but I can't have a test set because I have a small sample size. So, I want to use cross-validation to evaluate the model. I have seen that functions like ...
muhammad's user avatar
0 votes
0 answers
37 views

Reporting internal validation from .632 bootstrap in caret

I am developing Machine learning prediction models using the caret package in r (Elastic net, SVM, Random Forest, XGBoost). I have 650 cases with 104 having the event of interest. Instead of splitting ...
Dwayne T's user avatar
2 votes
2 answers
1k views

Too good to be true? Ridge prediction

I have a small data set of 18 persons. I have an outcome variable Y, and 200 predictors. These predictors were chosen based on biology and prior data. I used the caret R package and split the data set ...
user2862862's user avatar
0 votes
0 answers
7 views

best practise for mtry and ntree random forest tuning using the ranger and caret packages [duplicate]

I am implementing a random forest model using the ranger package. I know you cannot specify ntree's in the tune grid for caret. I have seen some suggestions to determine the best mtry values and then ...
Jcarroll's user avatar
0 votes
0 answers
100 views

What means 'scale' hyperparameter in SVM Polinomial (svmPoly method in caret)?

I have searched in many places, and can't find what the hyperparameter 'scale' means in SVM polynomial in caret (method = svmPoly) Caould you tell me, please?
Kaikus's user avatar
  • 219
1 vote
0 answers
341 views

Numeric categorical variables as factors or one hot encoded before using random forest?

I am performing a random forest model in R using caret = rf method. I have 20 explanatory variables and most are continuous but a few are categorical and numeric. For example, there are 6 categories ...
BHope's user avatar
  • 13
0 votes
1 answer
150 views

Is it possible to use variable importance to make inference?

I am dealing with classification predictive models in the context of machine learning; I am using different models (KNN, SVM, Random Forest, Logistic regression) and I am using the function ...
autu_mn's user avatar
  • 33
1 vote
1 answer
413 views

Same variable importance scores for SVM, KNN, and NB classifiers (R CARET )

I am comparing some ML classifiers (RF, KNN, SVM, NB, and XGBOOST). After training these classifiers, I extracted variable importance scores from each of these models using ...
MA0101's user avatar
  • 11
1 vote
2 answers
228 views

How to preprocess my stepwise regression using lasso/ridge?

I am struggling in the preprocessing of some analyses. I have a dataframe with around 100 observations and quite a few possible predictors (categorial and numerical data, about 20 in total). I am ...
umrpedrod's user avatar
0 votes
0 answers
26 views

Decision making algorithm (caret)

I would like to know if the caret package has decision-making algorithms other than the decision tree. I know the decision tree algorithm is intuitive and easy to explain, however, the decision tree ...
Daniel Leme's user avatar
0 votes
1 answer
336 views

Machine learning binary classifcation models with categorical variables. How to approach it?

I am approaching machine learning and I want to ask you some questions, hoping that they do not sound too trivial. I have a pretty small dataset with $n \simeq 500$ observations and about $p \simeq 20$...
autu_mn's user avatar
  • 33
2 votes
0 answers
313 views

Variable importance for random forests using caret

Consider the following toy example dataset (in R), including 5 factors: ...
Leandro T.'s user avatar
5 votes
2 answers
979 views

Poorly calibrated probabilities but good classification in confusion matrix

I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/...
mapleleaf's user avatar
  • 275
0 votes
0 answers
141 views

Changing Reference Levels of Categorical Variables Changes Confusion Matrix & Prediction Probabilities

I am trying to understand why changing the reference level of a factor changes the results of a model. Consider this example: ...
mapleleaf's user avatar
  • 275
1 vote
0 answers
339 views

What is the difference between lm() function and caret::train() function when it comes to creating linear regression models? [duplicate]

When applying the lm function as follows (the assumptions were not considered. The purpose of this example is just to make my question clear) : ...
An116's user avatar
  • 367
1 vote
0 answers
38 views

Formula versus Non-Formula Interface Categorical Variables train() glmnet

I am comparing the confusion matrix between the formula interface and the non-formula interface using caret's train() for elastic net. I am trying to understand why the two interfaces produces ...
mapleleaf's user avatar
  • 275
3 votes
2 answers
417 views

What is the interpretation of the "traditional" $R^2$?

Suppose the following data correspond to observed responses and their predictions obtained from some model. ...
user7064's user avatar
  • 2,207
2 votes
0 answers
70 views

An aggressive overfitting situation

I gather RNA-seq transcriptomic data from multiple cancer datasets. The datasets are about a treatment of cancer, we check Response vs NoResponse samples. The RNA-seq data I gather is before the ...
Programming Noob's user avatar
2 votes
0 answers
291 views

Train-Test Split with nested groups and multiple balancing factors

I have a large (~15,000) sample of data from individuals nested within families (with about half the data points sharing a family). I want to split the sample in to a training and test set so I can ...
mrpeverill's user avatar
1 vote
1 answer
318 views

Why predicted values differ in knn regression when using caret vs FNN?

I was trying to do some manual calculations of knn regression and came across this unusual error. The predicted values done by hand do not match with the ones I got from the 'knnreg' function in the '...
userK's user avatar
  • 51
1 vote
1 answer
74 views

Using ML model and decision tree to create a new risk classification

The idea of this project was to use a Machine Learning model to find the best variables to include in a decision tree algorithm. After evaluating with caret a number of different models I found the ...
fb95's user avatar
  • 31
1 vote
1 answer
682 views

Compare survival between two unbalanced groups

Briefly, 184 patients are included in my analysis. I have one variable that seperates 184 patients into two groups. 173 are in group 0 and 11 are in group 1. I need to compare the survival between ...
tomasz's user avatar
  • 13
2 votes
1 answer
309 views

How to use R package caret to build a model and get the internal validation result?

I am a learner of R and machine learning. I don't really understand caret's train function. To make it simple, for example, I want to build a model and get the internal validation result. At the ...
freemanproust's user avatar
0 votes
0 answers
9 views

Problem predicting caret::train random forest ("rf") model [duplicate]

I´ve been working in a random forest model for credit scoring in R. I've trained a model using caret::train. My data "df_samples_rf" has the next ...
user avatar
1 vote
1 answer
403 views

Incorporate Weights/Offsets with Nonparametric Models

I am modeling pure premium in R. I have read that pure premiums are usually modeled using a Tweedie distribution (glm). There is generally an offset or weight added to the model, such as an exposure. ...
mapleleaf's user avatar
  • 275
1 vote
1 answer
769 views

How to interpret the output of caret::findCorrelation function?

The output I received after applying findCorrelation function from the caret package is: ...
merry123's user avatar
1 vote
1 answer
445 views

How do I report results of an internal validation in Caret?

I have the following question. In a machine learning project I have to solve a regression and a classification task. See also: Hold-Out VS Cross-Validation - R caret For this I have about ~650 cases ...
bckpex's user avatar
  • 61
1 vote
0 answers
153 views

how do I perform permutation testinging for a prediction model developed within caret package (R)? [closed]

I'm fairly new to data science/StackExchange, so please excuse any faux pas I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
Graham's user avatar
  • 29
1 vote
1 answer
517 views

How do I access the p-values of individual predictors using caret::train? [closed]

I can't figure out how to access the p-values for my predictor variables after using k-fold cross-validation with caret::train. Does anyone know? Below is an example using the Boston data set that ...
ben_p_4370's user avatar
0 votes
0 answers
960 views

R Caret - Random Forest with repeated CV

I am currently struggling to explain the combination of bagging and cross-validation. I am aware of what each does separately but I have difficulties to explain how they are combined (if at all). For ...
Ru8zi9's user avatar
  • 1
1 vote
0 answers
322 views

K-nearest neighbor with kernel gives me the exact same accuracy with different initial conditions (R,caret)

...
Orestis's user avatar
  • 43
0 votes
0 answers
11 views

Does Breiman's randomForest function in R use Gini score or entropy score or Classification Error rate to decide the splits? [duplicate]

I have read the R documentation on randomForest but I could not find anything about Gini or Entropy in it. Is there a way to direct the train function of caret to use Gini score?
Shoaib Ashraf's user avatar
2 votes
1 answer
446 views

Highly Correlated Datasets - Why and What Next?

I've got three datasets (of different biological data) that are highly correlated - such that if I use the typical findCorrelation from the ...
smack's user avatar
  • 21
1 vote
1 answer
2k views

How does glmnet in caret choose the values of lambda and how does it compute coefficients of the model?

I have a question that I've been struggling with. My students are asking me, but I can't figure it out myself. When I train LASSO regression in R caret, I use the method "glmnet" and a grid ...
Fedor Duzhin's user avatar
0 votes
1 answer
287 views

Best model (set of predictors) for my data?

I'm exploring some ML strategies using caret package. My goal is to select best predictors and to obtain optimal model for further predictions. My dataset is: 75 ...
Adamm's user avatar
  • 55
0 votes
0 answers
250 views

standard deviation of the model R2 in LOOCV in caret

I am performing a LOOCV linear model and I got the parameters R2 and RMSE, but I was wondering if there is a way to calculate the standard deviation of the model R2. I tried to do it in the same way I ...
Ursula's user avatar
  • 31
1 vote
1 answer
105 views

Caret classifying above chance on randomly generated data

I am attempting to compare a few methods for multi-class classification using caret: 'multinom' (logistic regression), 'nnet' (neural net), and 'svmPoly' and 'svmLinear' (two types of support vector ...
Adam Morgan's user avatar
0 votes
0 answers
59 views

How to plot ROC and Recall-precision curves for this obtained data

Good afternoon , Assume we have this obtained data : ( I had trained a SOM map from scratch then i used a number of epochs, Each epoch consist of a number of iterations ) : ...
Tou Mou's user avatar
  • 113
1 vote
0 answers
513 views

choosing most important predictors for logistic regression

I have a dataset of cars with price label as binary outcome including "affordable" and "costly". I aim to ...
Jasmine N's user avatar
3 votes
0 answers
348 views

How to determine the correct amount of oversampling (in regression)

For my regression problem I splitted my dataset into a stratified train- and testset (70:30) and I now want to train my models (random forest, gbm, logistic regression) using the trainset. The dataset ...
XeeD's user avatar
  • 31
1 vote
1 answer
2k views

Caret "Metric RMSE not applicable for classification models" when data is continuous

I am running into an error trying to train a caret model with method='glmnet.' My data is continuous and I am trying to do a LASSO regression, but the existence of a 0 for one of the observations is ...
Andrew's user avatar
  • 13
0 votes
1 answer
273 views

In trainControl of Caret, how to keep a specific proportion of samples for cross validation?

Sample Data: ...
Mohammad Tanvir Ahamed's user avatar

1
2 3 4 5
10