Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.

126 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
1answer
2k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
4
votes
0answers
5k views

Backward stepwise regression with cross validation in R

I would like to do model selection using backward stepwise procedure and cross validation. https://www.otexts.org/fpp/5/3 I have used stepAIC in ...
3
votes
1answer
1k views

Improve the precision of random forest for count data

I am trying to create a classification model that predicts whether a customer will enquire for a financial product based on some 250 independent variables. 98% of the variables are count variables and ...
3
votes
0answers
988 views

Recursive feature elimination (rfe) performs poorly with binary outcome

Backward elimination with random forests does not work as expected in a simple test case with binary outcome and three continuous predictors. Below, I generate a binary outcome based on a single ...
3
votes
0answers
147 views

Why Does My Model Accuracy Go Down With More Data?

I am using R and the caret package. My code is very straight forward. ...
3
votes
0answers
475 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
3
votes
0answers
471 views

Important question regarding feature selection methodologies in R concerning the randomness of the results

I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features ...
3
votes
0answers
461 views

PLS using a kernel matrix

I would like to use a kernel matrix generated with a custom kernel function to fit a PLS-DA model (I am thinking of caret's PLS-DA at the moment), with only one binary response variable in the Y block....
3
votes
0answers
507 views

Manual vs automated approach for predictive modeling

Many statistics textbooks emphasise a manual modeling design approach, whereby the practictioner performs exploratory analysis by hand to assess several factors including whether there's any ...
3
votes
0answers
130 views

Measuring the effect of sample size on classification

I've got a data frame with 1000 observations and I am playing with classification methods using caret package. I'm mostly using the bootstrap method to check model ...
2
votes
0answers
32 views

Which R-Squared formula is used in carets' k-fold-cross validation function

I am currently trying to implement a linear model and calculate some out-of-sample accuracy metrics while using k-fold-cross validation. I try to do this using R's caret package, so this question ...
2
votes
0answers
94 views

Elastic net: how to tune for both lambda and alpha? Why is caret better than cv.glmnet?

I'm learning how to use R to fit Elastic Net models using the glmnet package. There appears to be a function, cv.glmnet that ...
2
votes
1answer
30 views

How do I set reasonable initial gridsearch parameters?

I'm training a number of models with the aim of identifying those which will perform well on my data. As such, I'm using a lot of models that I am unfamiliar with, and they all have their own tuning ...
2
votes
0answers
285 views

Linear discriminant scores using caret

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...
2
votes
0answers
655 views

Model ensemble with caretStack

I'm building a model ensemble with caretStack (package caretEnsemble). Here is a basic example : ...
2
votes
0answers
277 views

Verifying rebalancing of unbalanced binary class Caret R

Still very new to R and have only enough statistical knowledge to be dangerous. But I'm stuck as to whether the following is acheiving what I need it to do. Project is a flight risk prediction using ...
2
votes
0answers
1k views

Learning curve of random forest in caret

I have data with about 1250 rows and 97 features. Using random forest for classification in caret R package, I've trained a model and now, I want to plot the learning curve using the ...
2
votes
0answers
2k views

logic of resamples function in caret R package

The caret R package includes functions to compare models via their resampling distributions. Specifically, it prescribes fitting multiple models using the same resampling profiles (i.e. same versions ...
2
votes
0answers
91 views

How to I model corresponding features in caret

I am trying to build a binary-outcome prediction model wherein the features correspond to each outcome, for example my headers might be: ...
2
votes
0answers
97 views

Which resampling methods are suitable in the presence of dummy variables?

I want to build a machine learning model using the caret package in R. Some of the features in my dataset are dummies taking the value 0 or 1. I would like to know which resampling methods can be used ...
2
votes
0answers
804 views

Difference in execution times between caret and randomForest (even with method = “none”)

I have a dataset with 1205 observations and 285 predictors (all but one categorical). It is a binary classification task. When I run randomForest, it executes in less than 1 second. When I run caret ...
2
votes
0answers
1k views

Why does glmnet in caret give different predictions for different alphas even though lambda is zero?

In R, when using caret to train an elastic net regularization model, I find that different values of alpha give different predictions when the lambda parameter equals zero. This should not be the case,...
2
votes
0answers
581 views

Interpreting varImp using a GLMNET model on a 3-level factor

(tried stackoverflow but told I may have better luck here). I am looking for some help interpreting varImp using GLMNET multinomial model on a 3-level factor variable. The plot and the data don't make ...
2
votes
0answers
145 views

Training with very few positives

I have a binary classification problem where the fraction of positives is very low, e.g. 20 positives in 10,000 examples (0.2%) What is an appropriate cross validation scheme for training a ...
1
vote
0answers
41 views

Drop unused parameters from model?

I have trained a model using caret's train function in R (LASSO logistic regression with glmnet, repeatedcv). The goal is both feature selection and getting the model. With the preferred lambda, LASSO ...
1
vote
0answers
19 views

KNN and Classification Factor

I'm studying the KNN algorithm and its application on R. If I correctly understood, KNN is a supervised algorithm able to classify an unlabeled item according to the predominant belonging class in the ...
1
vote
0answers
23 views

How to apply svm to a dataset with a numerical (not categorical) dependent variable?

Context: I have some band values from a sentinel - 2 derived .tiff I now want to make a prediction regarding areas where i have no actual field data i.e. make a carbon map of sorts Libraries are: <...
1
vote
0answers
49 views

is a final model different from cross validation?

It appears that I have completely misunderstood cross validation for several months so I want your help clarifying the idea using an example rather than only theory from some great SE questions. My ...
1
vote
0answers
121 views

How to calculate variable importance for different models? is varImp() the solution?

I'm using caret's train() function for a binary classification outcome with different models (nb, knn, lda, qda, glm, rpart, rf). I'm using varImp() and plots to determine the importance of every ...
1
vote
0answers
27 views

Can we compare class probabilities between different methods in caret?

I am using the caret package in R for binary classification and I want to compare different methods (e.g., random forest, SVM, ANN, PLS-DA...). I consider the class probabilities as a "certainty score"...
1
vote
0answers
14 views

Infer estimation accuracy from the percentage of number of trees voting for the majority class?

I am performing a classification with caret in R. With the predict()-function I can also get a percentage-output of the number of trees voting for majority class. In my case I have 4 possible classes ...
1
vote
1answer
71 views

Plot Actual vs Predicted SVM Regression

I am building an SVM regression model using caret package, however, I am not sure what is the best approach to plot predicted vs actual values. I have the code below. You can reproduce the output by ...
1
vote
1answer
1k views

R - xgbTree, xgbDart and gbm in Caret predict Small Range

I am currently facing an issue in my regression model. I have tried the models in the title (xgbTree, xgbDART, and gbm in caret), and they tend to predict a very small range for the output variable. ...
1
vote
0answers
165 views

Data splitting with caret: Can we remove the ID Coloumn after folds are created?

so we have a dependent sample (two observations for each participant). To prevent data from one participant being in the training set and in the unseen fold in cross-validation we used the groupKFold ...
1
vote
0answers
31 views

How is this standard error obtained?

I am working through the exercises in Kuhn and Johnson's "Applied Predictive Modelling" and cannot reproduce one of their results in the exercises. Looking at 4.3 we have ... find the number of ...
1
vote
0answers
29 views

How to select the most important features, categorical & numerical data

I need to find out which factors are relevant when predicting low birth weight. My model looks like this: ...
1
vote
0answers
34 views

When to use which classification model?

This is something that continues to give me trouble. Assuming I am working to extract a classification from a dataset and assuming I have the computing resources to do the necessary calculations (in ...
1
vote
1answer
82 views

How do I avoid time leakage in my KNN model?

I am building a KNN model to predict housing prices. I'll go through my data and my model and then my problem. Data - ...
1
vote
0answers
352 views

LDA Fit using Caret does not give Standard Deviation

I am following the steps outlined in this tutorial. I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy ...
1
vote
0answers
1k views

Random forest vs Random Forest tuned with caret

Looking for some help please. I have used the randomForest package extensively and have been happy with its performance. I am currently writing a journal paper investigating match outcomes in sport, ...
1
vote
0answers
32 views

How to approach memory issues with up/down-sampling problem across millions of rows in database that can't be loaded locally? (class imbalance)

I'm faced with fairly typical class imbalance problem across a dataset with nearly 9MM rows (hard drive failures) that's not stored locally (it's in Postgres table; downloading a .csv of it is not ...
1
vote
0answers
878 views

Drawing ROC curves from RFE() training results in caret

I want to generate ROC curves using the training data and results from the rfe function in caret. I have managed to do this with the code below but there is some inconsistency between the ROC value ...
1
vote
0answers
513 views

How to use Elastic Net Model to Reduce Collinearity

I am using R to perform a linear regression with a dataset that has clearly correlated independent variables (collinearity). I am using the vif (variance inflation factor) function from the car ...
1
vote
0answers
193 views

R caret calibration function

I'm studying about calibration techniques and I'd like to use the CARET calibration function to examine the quality of my classifiers' probabilities. I read the relevant documentation, however I don'...
1
vote
0answers
357 views

brnn r package (Bayesian Regularization for Feed-Forward Neural Networks)

I'm using the "brnn" package (Bayesian Regularized Neural Networks), in particular I run the train() function from caret package. My data are stored in a data.frame object and I run the caret function ...
1
vote
0answers
124 views

Intuition behind low optimal # variables per split (mtry / max_features) during random forest tuning?

I was wondering if anyone might be able to provide some insight regarding how to interpret the results of some random forest hyperparameter tuning I am performing. The training set consists of: 1000 ...
1
vote
0answers
573 views

random forest imbalanced data-over, under, Smote Sampling

I am using random forest model for an imbalanced dataset. The dependent variable is Yes=73, No=7100. I have 65 independent variables both factor and numeric. I have tried to develop models for ...
1
vote
0answers
263 views

Supervised PCA with caret

I am trying to create my own training model in caret to do prediction using Supervised Principal Components, but I am having difficulty in preprocessing my data. As far as I know, I am not able to ...