Questions tagged [caret]
Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.
467
questions
0
votes
0answers
10 views
How to plot ROC and Recall-precision curves for this obtained data
Good afternoon ,
Assume we have this obtained data : ( I had trained a SOM map from scratch then i used a number of epochs, Each epoch consist of a number of iterations ) :
...
1
vote
0answers
24 views
choosing most important predictors for logistic regression
I have a dataset of cars with price label as binary outcome including "affordable" and "costly". I aim to ...
0
votes
0answers
9 views
R2 function in caret is giving an NA output when I compare predictions with actual values in R
I have a linear regression model which regresses house sale_price on a bunch of properties of the house:
lm1 <-
lm(sale_price ~ ., data = train_new)
I've ...
0
votes
0answers
30 views
XGB Early Stopping and Caret - R
I am using caret to train an xgb model. What I'm confused about is the difference between these two scenarios (identical parameters & cv unless otherwise stated):
1. Give caret::train() a tune ...
2
votes
0answers
42 views
How to determine the correct amount of oversampling (in regression)
For my regression problem I splitted my dataset into a stratified train- and testset (70:30) and I now want to train my models (random forest, gbm, logistic regression) using the trainset. The dataset ...
0
votes
0answers
12 views
tunegrid not working with neural networks in caret
I am wanting to build a neural network classifier using the caret package. I have specified a tunegrid with some hyper-parameters that I want to test to get the best accuracy.
After I run the model, ...
0
votes
1answer
68 views
Caret “Metric RMSE not applicable for classification models” when data is continuous
I am running into an error trying to train a caret model with method='glmnet.' My data is continuous and I am trying to do a LASSO regression, but the existence of a 0 for one of the observations is ...
0
votes
0answers
6 views
Weights in Caret train
I would like to know more on the weights parameter in Caret train as I could not find any thing concrete from Caret documentation.
Whether the weight indicates cost sensitive learning by engaging ...
0
votes
0answers
18 views
How variable importance for decision tree classifier in `caret` is estimated?
I trained a decision tree classifier by means the package caret, This is the code:
...
0
votes
1answer
27 views
0
votes
0answers
5 views
How to choose important features in random forest when there is conflict between accuracy and node purity
In random forests, how to select the most important feature when the features bring different levels of node purity and accuracy.
In the graph below, one feature may help to significantly improve the ...
2
votes
0answers
26 views
Using cross-validation for model selection and model comparison
Let us suppose that we have two classifiers: SVM and CART. For each one of them, a set of hyperparameters is considered (C=0.001,0.01,... for SVM cp=... for CART).
The question is, can I use k-fold ...
2
votes
1answer
24 views
Why do the results of LASSO regression differ after removing uninformative variables in glmnet?
I am researching therapy response of melanoma patients based on a number of approximately 80 features with a very small sample size of 60 patients. To eliminate features that do not contribute to the ...
0
votes
0answers
20 views
Classification problem. I have issues with overfitting/leakage, i.e. sensitivity in training is 1.0. How do I fix this?
I have a consumer dataset (size: 10200x180), I am using this to develop a training model to predict consumers who would provide a referral. I have done the following to set the model.
Cleaning:
I have ...
1
vote
1answer
43 views
Unreasonable bias when using nnet (R package caret) for time series forecasting
I have been trying to forecast a time series in a regression-like setting using neural networks (nnet method in R package caret)....
0
votes
1answer
16 views
ML test accuracy higher than training? Small unbalanced samples were stratified by class
My background is in ecology, it is common to have smaller sample sizes and class imbalances and ML approaches are still increasingly adopted. My specific dataset: training set is 49 sample, my test ...
0
votes
1answer
82 views
Impute missing values of dummy variables, using R's {caret} package: predicted values in between {0;1}?
I'm using {caret} to impute missing data resulting from non-response to survey questions. All of these variables are defined as numeric, though most are dummies. ...
0
votes
0answers
28 views
How to perform Multiple Imputation (MI) for longitudinal survey data, using {caret} in R?
I have a large dataset comprising survey responses throughout a time period of three to four decades (or in other words, slightly above 20 waves). Keep in mind that these are longitudinal data and not ...
1
vote
0answers
46 views
Extreme collinearity for Random Forest?
My data have 450 observations and 2200 predictors that I want to use to train a RF model to classify 4 classes. However, about 2165 of the predictors are very highly correlated to each other. The way ...
0
votes
1answer
175 views
Variable Importance for Caret Random Forest Regression
I have trouble understanding the exact meaning of the feature importance scores in caret for RF regression. As you know there are many potential importance measures for RF. However, there is no clear ...
0
votes
0answers
57 views
0
votes
0answers
14 views
Machine learning methods for filtering noisy data
I have my data (hear beat) cleaned up with the appropriate filter, however, they are still noisy and need to extract a clean pattern. Which is the best algorithm for this purpose?
I have also clean ...
0
votes
1answer
251 views
Model building with 10-fold validation
I have yet to find a sufficient and succinct answer regarding model building with 10-fold cross validation (in this case, using Caret). I've found responses here, for instance: https://stackoverflow....
1
vote
1answer
51 views
How to manually calculate predictions of kernlabs SVM
I am trying to manually replicate the predictions of kernlabs SVM (polynomial & radial kernel) using caret.
Here is the code to fit the model:
...
1
vote
1answer
81 views
How many models does caret::trainControl() actually train?
I searched around CV and the caret documentation but couldn't find an answer to this. I'm using caret in R to tune some random forests for classification trained ...
1
vote
1answer
168 views
0
votes
1answer
285 views
Cross-validation using Caret in R: Why are coefficients from FinalModel identical to those from lm()?
I think I must be missing some fundamental part of the logic of cross-validation, or machine learning in general.
Using the caret package in R, I ran a repeated k-...
2
votes
0answers
63 views
Which R-Squared formula is used in carets' k-fold-cross validation function
I am currently trying to implement a linear model and calculate some out-of-sample accuracy metrics while using k-fold-cross validation. I try to do this using R's caret package, so this question ...
0
votes
0answers
80 views
Interpret varimp() with Caret LDA model
I'm using caret's train() function for a binary classification outcome with an lda models, based on several features
...
1
vote
0answers
44 views
Drop unused parameters from model?
I have trained a model using caret's train function in R (LASSO logistic regression with glmnet, repeatedcv). The goal is both feature selection and getting the model. With the preferred lambda, LASSO ...
1
vote
0answers
25 views
KNN and Classification Factor
I'm studying the KNN algorithm and its application on R. If I correctly understood, KNN is a supervised algorithm able to classify an unlabeled item according to the predominant belonging class in the ...
1
vote
1answer
272 views
What really is 'glmnet' when used in caret in R for binary classification?
like lasso and ridge, elastic net can also be used for classification
by using the deviance instead of the residual sum of squares. This
essentially happens automatically in caret if the response ...
7
votes
2answers
266 views
How should two cross-validated logistic regression models be compared?
I'm using 100 times 10-fold repeated cross-validation to assess the ROC-AUC performance improvement of adding a biomarker to an existing model:
Model_A : pred1 + pred2
Model_B :pred1 + pred2 + pred3
I'...
1
vote
0answers
195 views
2
votes
1answer
62 views
Model stacking, Super Learner Algorithm
I've recently started studying ensembles in ML, particularly Super Learner Algorithm. To be honest, although I have read several articles related to this topic, I am a little bit confused. I want to ...
0
votes
1answer
112 views
Confusion Matrix for Multi-class Variables using cross-validation
my dataset includes multi-class variables (11 different variables). All of the columns are numeric except the last column which is the label ( Last Column Name = Movement, 11 different types, Type = ...
0
votes
0answers
100 views
Permutation Testing Machine Learning Model Performance With Caret in R
Using the caret package in R, classification machine learning model performance is usually assessed using the confusionmatrix() function, which uses a Binomial Test ...
0
votes
0answers
12 views
What can go wrong if we don't scale and center data in a predictive model? [duplicate]
I have started using R recently, along with the caret package. I note that there is an option to preprocess data so that it is scaled and centered.
I am interested in multiple linear regression. What ...
0
votes
0answers
85 views
Random Forest doesn't converge in caret
I'm using some synthetic data to try and understand the performance of different ML algorithms on my real data. I'm finding that RF is consistently overfitting the training data even though its ...
1
vote
1answer
69 views
What's random in Caret if the hold out is fixed for cross validation [closed]
I tried to fix the folds for cross validation in Caret. But there is still some randomness returned by the train function. See code below. Don't they should return the same?
...
2
votes
1answer
63 views
RFE: Pre-define a specificity threshold
I would like to use recursive feature elimination (implemented via caret in R) to perform feature selection for about 40 test results with 2 possible outcomes. Consequently, RFE either models by ...
1
vote
1answer
471 views
How do I calculate confidence intervals on an elastic net regression in R
I am performing an elastic net regression on my data n = 34, p = 46
I first built the model using the "caret" package with the cross validation method to set the optimal alpha and lambda ...
2
votes
0answers
153 views
Elastic net: how to tune for both lambda and alpha? Why is caret better than cv.glmnet?
I'm learning how to use R to fit Elastic Net models using the glmnet package. There appears to be a function, cv.glmnet that ...
0
votes
0answers
27 views
Why is Rsquared smaller in whole sample than k-fold cross-validated?
I want to get a cross-validation $R^2$ for a linear regression model with one predictor to get a measure of how well my model predicts rather than explains data (in R with package caret). I assumed ...
0
votes
1answer
121 views
What does the featurePlot() function from the caret package actually do?
Just a quick example from Max Kuhn's book.
...
0
votes
0answers
43 views
Can I use caret RFE result for subsequent random forest with CV?
I've been googling and reading alot about my issue but couldn't find a clear answer.
In order to prevent data leakage, I use caret RFE for feature elimination:
...
1
vote
0answers
27 views
How to apply svm to a dataset with a numerical (not categorical) dependent variable?
Context: I have some band values from a sentinel - 2 derived .tiff
I now want to make a prediction regarding areas where i have no actual field data
i.e. make a carbon map of sorts
Libraries are:
<...
1
vote
0answers
51 views
is a final model different from cross validation?
It appears that I have completely misunderstood cross validation for several months so I want your help clarifying the idea using an example rather than only theory from some great SE questions. My ...
1
vote
2answers
120 views
How does repeated cross validation go about 'averaging' model coefficients?
No matter how much I google, I cannot find the answer to this simple question.
Say you do 10-fold, repeated (5x) CV logistic regression with elastic net regularization.
For alpha you try ...
1
vote
0answers
12 views
Weighted observations for linear model [closed]
I have dataset with sale price of products in years 2015-2020. I would like use it to predict future sale prices. I would like to use sale year as a weight of observation (eg. for 2020-100%, for 2019-...
