Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.

120 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
1answer
2k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
4
votes
0answers
541 views

Reported Coefficients for Glmnet using Caret

I understand GLMnet standardizes the predictor variables by default before fitting the model. After fitting, the computed regression coefficients are then destandardized to allow reporting in their ...
4
votes
0answers
1k views

using caret and glmnet for variable selection

Im using the caret and glmnet package for variable selection. I only want to find the best model and the coefficients and use ...
4
votes
0answers
4k views

Backward stepwise regression with cross validation in R

I would like to do model selection using backward stepwise procedure and cross validation. https://www.otexts.org/fpp/5/3 I have used stepAIC in ...
3
votes
0answers
127 views

Why Does My Model Accuracy Go Down With More Data?

I am using R and the caret package. My code is very straight forward. ...
3
votes
0answers
450 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
3
votes
0answers
416 views

Important question regarding feature selection methodologies in R concerning the randomness of the results

I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features ...
3
votes
0answers
422 views

PLS using a kernel matrix

I would like to use a kernel matrix generated with a custom kernel function to fit a PLS-DA model (I am thinking of caret's PLS-DA at the moment), with only one binary response variable in the Y block....
3
votes
0answers
478 views

Manual vs automated approach for predictive modeling

Many statistics textbooks emphasise a manual modeling design approach, whereby the practictioner performs exploratory analysis by hand to assess several factors including whether there's any ...
2
votes
1answer
37 views

R: caret (elasticnet): ridge regression: understanding the returned parameters

I wanted to play around with the ridge regression in caret (which apparently uses elasticnet), so I did two experiments: use the original data use the modified data where the values of ...
2
votes
1answer
23 views

How do I set reasonable initial gridsearch parameters?

I'm training a number of models with the aim of identifying those which will perform well on my data. As such, I'm using a lot of models that I am unfamiliar with, and they all have their own tuning ...
2
votes
0answers
518 views

Model ensemble with caretStack

I'm building a model ensemble with caretStack (package caretEnsemble). Here is a basic example : ...
2
votes
1answer
644 views

Improve the precision of random forest for count data

I am trying to create a classification model that predicts whether a customer will enquire for a financial product based on some 250 independent variables. 98% of the variables are count variables and ...
2
votes
0answers
946 views

Recursive feature elimination (rfe) performs poorly with binary outcome

Backward elimination with random forests does not work as expected in a simple test case with binary outcome and three continuous predictors. Below, I generate a binary outcome based on a single ...
2
votes
0answers
246 views

Verifying rebalancing of unbalanced binary class Caret R

Still very new to R and have only enough statistical knowledge to be dangerous. But I'm stuck as to whether the following is acheiving what I need it to do. Project is a flight risk prediction using ...
2
votes
0answers
1k views

Learning curve of random forest in caret

I have data with about 1250 rows and 97 features. Using random forest for classification in caret R package, I've trained a model and now, I want to plot the learning curve using the ...
2
votes
0answers
2k views

logic of resamples function in caret R package

The caret R package includes functions to compare models via their resampling distributions. Specifically, it prescribes fitting multiple models using the same resampling profiles (i.e. same versions ...
2
votes
0answers
88 views

How to I model corresponding features in caret

I am trying to build a binary-outcome prediction model wherein the features correspond to each outcome, for example my headers might be: ...
2
votes
0answers
90 views

Which resampling methods are suitable in the presence of dummy variables?

I want to build a machine learning model using the caret package in R. Some of the features in my dataset are dummies taking the value 0 or 1. I would like to know which resampling methods can be used ...
2
votes
0answers
761 views

Difference in execution times between caret and randomForest (even with method = “none”)

I have a dataset with 1205 observations and 285 predictors (all but one categorical). It is a binary classification task. When I run randomForest, it executes in less than 1 second. When I run caret ...
2
votes
0answers
982 views

Why does glmnet in caret give different predictions for different alphas even though lambda is zero?

In R, when using caret to train an elastic net regularization model, I find that different values of alpha give different predictions when the lambda parameter equals zero. This should not be the case,...
2
votes
0answers
538 views

Interpreting varImp using a GLMNET model on a 3-level factor

(tried stackoverflow but told I may have better luck here). I am looking for some help interpreting varImp using GLMNET multinomial model on a 3-level factor variable. The plot and the data don't make ...
2
votes
0answers
127 views

Measuring the effect of sample size on classification

I've got a data frame with 1000 observations and I am playing with classification methods using caret package. I'm mostly using the bootstrap method to check model ...
2
votes
0answers
140 views

Training with very few positives

I have a binary classification problem where the fraction of positives is very low, e.g. 20 positives in 10,000 examples (0.2%) What is an appropriate cross validation scheme for training a ...
1
vote
1answer
9 views

Predict different models from K Best models in Caret

I was wondering if there was a way to run predictions on the K best Models and not just the best Model in Caret. ...
1
vote
1answer
39 views

Machine learning with univariate time series

I am trying to make predictions with daily data with time series in R. This time series is univariate and contains only data from sales from each 365 days in a four year period. My intention is to ...
1
vote
0answers
21 views

Plot Actual vs Predicted SVM Regression

I am building an SVM regression model using caret package, however, I am not sure what is the best approach to plot predicted vs actual values. I have the code below. You can reproduce the output by ...
1
vote
1answer
509 views

R - xgbTree, xgbDart and gbm in Caret predict Small Range

I am currently facing an issue in my regression model. I have tried the models in the title (xgbTree, xgbDART, and gbm in caret), and they tend to predict a very small range for the output variable. ...
1
vote
0answers
80 views

Data splitting with caret: Can we remove the ID Coloumn after folds are created?

so we have a dependent sample (two observations for each participant). To prevent data from one participant being in the training set and in the unseen fold in cross-validation we used the groupKFold ...
1
vote
0answers
26 views

How is this standard error obtained?

I am working through the exercises in Kuhn and Johnson's "Applied Predictive Modelling" and cannot reproduce one of their results in the exercises. Looking at 4.3 we have ... find the number of ...
1
vote
0answers
27 views

How to select the most important features, categorical & numerical data

I need to find out which factors are relevant when predicting low birth weight. My model looks like this: ...
1
vote
1answer
66 views

How do I avoid time leakage in my KNN model?

I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem. Data - ...
1
vote
0answers
281 views

LDA Fit using Caret does not give Standard Deviation

I am following the steps outlined in this tutorial. I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy ...
1
vote
0answers
794 views

Random forest vs Random Forest tuned with caret

Looking for some help please. I have used the randomForest package extensively and have been happy with its performance. I am currently writing a journal paper investigating match outcomes in sport, ...
1
vote
0answers
184 views

Linear discriminant scores using caret

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...
1
vote
0answers
31 views

How to approach memory issues with up/down-sampling problem across millions of rows in database that can't be loaded locally? (class imbalance)

I'm faced with fairly typical class imbalance problem across a dataset with nearly 9MM rows (hard drive failures) that's not stored locally (it's in Postgres table; downloading a .csv of it is not ...
1
vote
0answers
286 views

Using An ROC Curve to Evaluate a model

I have a number of questions on the ROC curves when being used to evaluate a model. My understanding of them is they can be used to determine the probability cutoff when classifying a row in a dataset ...
1
vote
0answers
703 views

Drawing ROC curves from RFE() training results in caret

I want to generate ROC curves using the training data and results from the rfe function in caret. I have managed to do this with the code below but there is some inconsistency between the ROC value ...
1
vote
0answers
357 views

How to use Elastic Net Model to Reduce Collinearity

I am using R to perform a linear regression with a dataset that has clearly correlated independent variables (collinearity). I am using the vif (variance inflation factor) function from the car ...
1
vote
0answers
164 views

R caret calibration function

I'm studying about calibration techniques and I'd like to use the CARET calibration function to examine the quality of my classifiers' probabilities. I read the relevant documentation, however I don'...
1
vote
0answers
310 views

brnn r package (Bayesian Regularization for Feed-Forward Neural Networks)

I'm using the "brnn" package (Bayesian Regularized Neural Networks), in particular I run the train() function from caret package. My data are stored in a data.frame object and I run the caret function ...
1
vote
0answers
84 views

Intuition behind low optimal # variables per split (mtry / max_features) during random forest tuning?

I was wondering if anyone might be able to provide some insight regarding how to interpret the results of some random forest hyperparameter tuning I am performing. The training set consists of: 1000 ...
1
vote
0answers
512 views

random forest imbalanced data-over, under, Smote Sampling

I am using random forest model for an imbalanced dataset. The dependent variable is Yes=73, No=7100. I have 65 independent variables both factor and numeric. I have tried to develop models for ...
1
vote
0answers
155 views

Supervised PCA with caret

I am trying to create my own training model in caret to do prediction using Supervised Principal Components, but I am having difficulty in preprocessing my data. As far as I know, I am not able to ...
1
vote
0answers
14 views

should the model variance from variance for all folds or for all repeats in repeated-n-fold cross validation?

In 10-fold-5-repeat-CV, I have two ways to calculate the model variance. in the first one, I calculate the variance using accuracies of all 50 validation sets. In the second one, I first calculate an ...
1
vote
0answers
196 views

How to check if the imputed value form caret is reliable?(predicting missing values)

I have a dataset which has some missing values and I tried to predict them by using caret. My data set looks like this ...
1
vote
0answers
851 views

KNN imputation for categorical variables

I am using preProcess in caret to knnImpute. As far as I understand, the imputation should include all the variables in the analysis and KNN imputation can only be done effectively if data is on the ...
1
vote
0answers
829 views

Interpreting a PLS-DA model with Caret

I'm train a Partial Least Squares Discrimination Analysis with caret package. My dataset dimensions are 80K rows, 50 continuous explanatory variables, and one ...
1
vote
0answers
299 views

How does caret split into CV folds? Can I avoid splitting randomly?

As I understand when setting trainControl(method="cv") in the caret package, my training set is is split into K folds for cross validation. How is this splitting ...