Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.

Filter by
Sorted by
Tagged with
3
votes
0answers
971 views

Recursive feature elimination (rfe) performs poorly with binary outcome

Backward elimination with random forests does not work as expected in a simple test case with binary outcome and three continuous predictors. Below, I generate a binary outcome based on a single ...
3
votes
0answers
135 views

Why Does My Model Accuracy Go Down With More Data?

I am using R and the caret package. My code is very straight forward. ...
3
votes
0answers
461 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
3
votes
0answers
84 views

Variable Importance results and beyond [closed]

This is something I thought sometime ago, forgot about and now remembered. When using predictive models it is useful to evaluate the relationship between each predictor and the outcome. A very good R-...
3
votes
0answers
431 views

Important question regarding feature selection methodologies in R concerning the randomness of the results

I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features ...
3
votes
0answers
443 views

PLS using a kernel matrix

I would like to use a kernel matrix generated with a custom kernel function to fit a PLS-DA model (I am thinking of caret's PLS-DA at the moment), with only one binary response variable in the Y block....
3
votes
0answers
498 views

Manual vs automated approach for predictive modeling

Many statistics textbooks emphasise a manual modeling design approach, whereby the practictioner performs exploratory analysis by hand to assess several factors including whether there's any ...
3
votes
0answers
129 views

Measuring the effect of sample size on classification

I've got a data frame with 1000 observations and I am playing with classification methods using caret package. I'm mostly using the bootstrap method to check model ...
2
votes
1answer
4k views

How does caret handle factors?

I have been testing conditional trees and random forests with caret, and I've noticed it does something weird with factors. So, for example, a ctree using the base dataset ...
2
votes
1answer
4k views

Poor multiclass classification using Caret in R [closed]

I have 5 continuous variables with 211 rows of data and each row is assigned a region, there are 7 possible regions in total. I wish to build a machine learning model that classifies an unseen row of ...
2
votes
2answers
645 views

AUC of a cross validation model

Let's say i build a 10X cross-validation model, say with Caret. If i want the AUC of this, is it: The average AUC of the 10 validation samples? Something else? Cheers!
2
votes
1answer
6k views

Train / Validate / Test sets in Caret

I want to use caret to compare two different classification algorithms. For example SVM and Elastic net. I want to put aside some samples for test set and then use the rest of the samples for ...
2
votes
1answer
3k views

Is there a way to return the standard error of cross-validation predictions using caret `train`

In the book Applied Predictive Modelling Ch 4., there is the following table: The standard error here is used in the following graph, and to use the "one-standard error method" to find the optimal ...
2
votes
1answer
2k views

Why use dummy variables in GBM using CARET library in R

I have seen a few examples implemting the gbm algorithm on youtube using the titanic dataset. These examples have turned some factor variables into dummy/indicator variables when GBM can handle factor ...
2
votes
1answer
3k views

What is the difference between function createFolds and createDataPartition in caret

I think the two functions are all to split the data, but I really can't get the difference between both of them, even I has read the help manual of them.
2
votes
1answer
805 views

Decision tree does not overfit, why?

The following code trains multiple decision trees on synthetic data with varying complexity: ...
2
votes
2answers
3k views

Should logistic regression models generated with and without cross validation in the caret.train function in R be the same?

I am working with the Titanic dataset and trying to use logistic regression in R to predict survival. The simple approach I tried was to just use the glm function with binomial family and logit link ...
2
votes
1answer
3k views

Explain “validation” process of repeated k-fold cross-validation?

My understanding is currently that the canonical repeated k-fold cross-validation (CV) process might do the following if $n=100$ observations in sample, $k=5$ folds, $i= 10$ iterations (see iteration ...
2
votes
1answer
1k views

Feature selection with Caret for data with more than one target

I am trying to do some feature selection, having around 3500 variables for about 200 samples. To each sample is associated two numerical values (the expected outcome). I can't manage to make the caret ...
2
votes
1answer
50 views

PCA in R: different results for caret and prcomp

Can some tell my why the preProcess function from the caret packages gives a different result than the ...
2
votes
1answer
434 views

R caret classification - why doesn't model accuracy equal accuracy given by predict()?

I have a dataset with 1000 samples, and each sample is 1 of 3 classes. I'm training classifiers on the dataset and predicting classes (5-fold cross-validated) and I'd like to know how well each ...
2
votes
1answer
2k views

r: coefficients from glmnet and caret are different for the same lambda

I've read a few Q&As about this, but am still not sure I understand, why the coefficients from glmnet and caret models based on the same sample and the same hyper-parameters are slightly different....
2
votes
1answer
773 views

Cross-validation tuning

I'm using Caret to perform some analyses, but even after reading some books and the caret manual I still have some theoretical doubts. Below the code I'm using. ...
2
votes
3answers
1k views

Which data split should I use to determine cutoff point for classification?

I'm building a classification model using the caret package. I'm splitting my dataset in train and test (80/20) and training using 10-fold cross-validation repeated ...
2
votes
1answer
6k views

r - choosing correct nnet model

Language: R Background data = 1800 observations (rows) x 5 variables (columns) I am using library(caret) and training regression models using ...
2
votes
1answer
941 views

How to do test set evaluation using a regression model in Caret?

I'm used to using Caret to do classification but now I need to use it for regression. I have successfully trained a model on my training set but I'm not sure what ...
2
votes
1answer
5k views

Leave-one-subject-out cross validation in Caret

Hi Dear Colleagues, I wonder how to correctly setup a leave-one-subject-out cross validation (LOSO) for train() function in caret. Here is my example code: ...
2
votes
2answers
1k views

Find variables selected for each subset using caret feature selection

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables ...
2
votes
1answer
2k views

Help requested with using custom model in caret() package

The caret package (terrific btw) has a lot of models built in but if you want to use a model that is not built in, there is a way as described in outline here http://caret.r-forge.r-project.org/...
2
votes
2answers
61 views

Predict Values based on 16 values of Past 16 years

I have a dataset of total number of Vehicles Registered for 16 years. These are a total of 16 values from 2001-2016. Which Machine Learning Technique would be best for predicting the number of ...
2
votes
1answer
2k views

Always choosing Repeated CV instead of fold CV

To my understanding, Repeated CV gives superior results compared to k fold CV. Is there any case where k fold CV is preferable compared to Repated CV? In particular for Neural Networks, what would ...
2
votes
1answer
2k views

recursive feature elimination: why select subset based on AUC vs sensitivity/specificity

I have a small dataset of 25 observations with a classification variable (factor 0,1) and 82 features scaled to have values between 0 and 1. I used the rfe() ...
2
votes
1answer
2k views

KNN Imputation: Do you use the training preProcess on both training and test sets?

I understand that you want to impute missing values separately for training, testing, and ...
2
votes
1answer
1k views

filterVarImp - Caret Package [closed]

I was trying the filterVarImp function in caret package. While the outcome was categorical we get ROC as the importance measure. ...
2
votes
2answers
37k views

How to make a confusion matrix when testing a model on data with only positive cases?

I have recently encountered a problem with the function confusionMatrix in R from the caret package regarding a machine learning ...
2
votes
1answer
286 views

flexible discriminant analysis computational completixy

I'm using mda package + caret infrastucture to perform a flexible discriminant analysis for a classification problems. I have 26 features of mixed type. I found little if any guidance on the ...
2
votes
1answer
439 views

Does “caret” avoid data snooping due to preprocessing in model tuning? [closed]

Although the obvious answer to my question is yes, since caret is a professional, well-known tool, I tend to be skeptical when using implemented functionality from ...
2
votes
1answer
9k views

How to create a data partition in R using categorical and numerical columns? [closed]

I'm using the createDataPartition method of the caret package as following: ...
2
votes
1answer
7k views

Metrics for multi-class problems in R caret package for various method tags [closed]

The caret package for R provides a variety of error metrics predominantly aimed at 2-class classification models with limited error metrics. Here is a multi-class function to allow caret:::train to ...
2
votes
1answer
6k views

How to implement a hold-out validation in R

Let's say I'm using the Sonar data and I'd like to make a hold-out validation in R. I partitioned the data using the createFolds ...
2
votes
1answer
985 views

R's equivalente of scikit's KFold

I'm new to R and I'm trying to set up a basic k folds CV loop. In Python I'd use scikit's KFold. ...
2
votes
1answer
2k views

R caret package question

...
2
votes
1answer
61 views

R: caret (elasticnet): ridge regression: understanding the returned parameters

I wanted to play around with the ridge regression in caret (which apparently uses elasticnet), so I did two experiments: use the original data use the modified data where the values of ...
2
votes
1answer
582 views

glmStepAIC model is doing better that other models

I am training a model on an imbalanced dataset (about 5-20% of positive class) and trying out different algorithms in R using caret package. I have 57 predictors and around 2000-3000 observations in ...
2
votes
1answer
2k views

Meaning of Splitrule in random forest model

I am running a random forest model with the caret package and get the following as output: ...
2
votes
1answer
588 views

Using Random Forest Variable Importance to train SVM models (R)

I have trained a Random Forest model in R with the caret package but the results are not very promising. I have decided to try with SVM models but I have a great ...
2
votes
1answer
825 views

Comparison of models with transformed dependent variable

I want to check if transforming the dependent variable positively influences the model performance. For example, I have built two models using the caret package. ...
2
votes
1answer
2k views

caret rfe variable selection and test prediction

I have ~800 continuous variables and a categorical response variable (disease/non-disease) and I have been using caret to classify disease based on the continuous variables. I have used caret and ...
2
votes
1answer
2k views

neuralnet error

I am trying to train a neuralnet model by using caret package by this command: ...

1 2
3
4 5
9