Questions tagged [caret]

Caret is an R package containing a set of functions that attempt to streamline the process of creating predictive models.

Filter by
Sorted by
Tagged with
5
votes
3answers
2k views

Use Random Forest model to make predictions from sensor data

say I have a sensor that measures temperature, pressure ++, and want to use this data to predict some quantity "A". If I use multivariate regression, I can simply implement a model of the form A=a0+...
5
votes
1answer
5k views

Using partial AUC as Caret metric for cross-validation?

I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC ...
5
votes
1answer
952 views

Multiple neural networks with single output neuron vs. single NN with multiple output neurons

Main Question Given multiple output parameters that are independent of each other, would multiple ANNs with a single output neuron give better prediction results than a single ANN with multiple ...
5
votes
1answer
2k views

Using Rolling Forecast Origin Resampling in R for Neural Network Time Series

I am new to time series prediction and forecasting with neural networks and am having trouble with cross validation. I am fitting a multivariate time series. I have 236 monthly observations. I am ...
4
votes
1answer
5k views

How to get Sub-Training and Sub-Test from cross validation in Caret

I am using Caret and have divided my data into training(75%) and test (25%) sets. Now I am running 10-Fold CV on training data. I fit the following model: ...
4
votes
4answers
2k views

Why does lambda.min value in glmnet tuning cross-validation change, when repeating test?

I'm using glmnet package to build a linear regression with $\alpha$ = 0.5, to find best $\lambda$. xMatrix [44x15000] ($p>>...
4
votes
1answer
8k views

Tune alpha and lambda parameters of elastic nets in an optimal way

I am trying to tune alpha and lambda parameters for an elastic net based on the glmnet package. I found some sources, which propose different options for that ...
4
votes
2answers
7k views

Repeated CrossValidation, finalModel and ROC curves

I got a problem understanding the meaning of the finalModel when using a repeated CV. ...
4
votes
1answer
2k views

Caret: customizing feature selection, nested inside cross validation

Using caret, I want to train a SVM classifier and estimate its performance using repeated cross validation. My dataset has a very large number of predictors (300K) and I want to reduce this number ...
4
votes
3answers
4k views

Combining randomForests in R, why are the err.rate, mse and rsq components NULL [closed]

From ?combine in randomForest: The confusion, err.rate, mse and rsq components (as well as the corresponding components in the test compnent, if exist) of the ...
4
votes
1answer
623 views

Parameter tuning with vs without nested cross-validation

Disclaimer: This question has been inspired by this one, which is a good question but has unfortunately not attracted an answer that actually answers OPs question. Statistical models often times have ...
4
votes
1answer
17k views

KNN and K-folding in R

I'd like to use KNN to build a classifier in R. I'd like to use various K numbers using 5 fold CV each time - how would I report the accuracy for each value of K (KNN). I'm using the ...
4
votes
1answer
2k views

mtry tuning given by caret higher than the number of predictors

According to this discussion, it seems that the train function of the caret package returns a ...
4
votes
2answers
2k views

Looking for examples or alternatives to R RuleFit ensemble package

Does anyone know of any good example code illustrations for the rulefit Rule Based Learning Ensembles package? The documentation is incredibly lacking. I was guided to the package by this paper. If ...
4
votes
1answer
3k views

caret preProcess knnImpute error more nearest neighbours than there are points

I am trying to impute missing data using preProcess function in caret with kNNImpute method. ...
4
votes
1answer
469 views

Different ROC value for different packages in R, which one is correct?

I noticed that computing ROC with caret package and PROC packege sometimes gives different results. Usually they are the same, but if the predictions are worse than chance, caret will flip them and ...
4
votes
1answer
6k views

LGOCV caret package R

i am learning data mining through book . During classification chapters about Neural Networks the authors have below code. I have below questions: ...
4
votes
1answer
3k views

SMOTE sampling in caret package in R

when using caret packge in the trainControl you can use "smote" sampling. what is the default parameters the train in caret are using for smote?? parameters such as: perc.over = 300, k = 8, perc....
4
votes
1answer
11k views

ROC Area Under Curve (AUC) in SVM - different results between R functions

I have two questions relating to ROC AUC values in SVM training and testing. After training and testing an SVM in caret I've found differences between the AUC ...
4
votes
1answer
1k views

Splitting Longitudinal Data into Training & Test Sets [closed]

I'm trying to find a simple way to split some longitudinal data into a training and test set. I'm familiar with using the Caret package in R to make stratified splits, but only with wide-form data. It ...
4
votes
1answer
2k views

Isn't caret SVM classification wrong when class probabilities are included?

*Please note this question is about the Platt probabilistic output and SVM class assignment, not about the code or the package itself. It just happens to be the code where I stumbled on the issue. In ...
4
votes
0answers
566 views

Reported Coefficients for Glmnet using Caret

I understand GLMnet standardizes the predictor variables by default before fitting the model. After fitting, the computed regression coefficients are then destandardized to allow reporting in their ...
4
votes
0answers
1k views

using caret and glmnet for variable selection

Im using the caret and glmnet package for variable selection. I only want to find the best model and the coefficients and use ...
4
votes
0answers
4k views

Backward stepwise regression with cross validation in R

I would like to do model selection using backward stepwise procedure and cross validation. https://www.otexts.org/fpp/5/3 I have used stepAIC in ...
3
votes
1answer
17k views

Random Forest confusion matrix

I've been creating some random forest models using the caret package in R. I don't have a large amount of data to work with so I'm using 10 x 10-fold CV in lieu of an independent test set. When I ...
3
votes
1answer
7k views

Feature selection using caret + repeatedcv

I am using caret and repeatedcv with repeats for feature selection. That is, ...
3
votes
2answers
2k views

Do I have to preprocess my new data for a prediction, if I have used preprocessing for building the model?

In this example preprocessing is used to construct a NN: ...
3
votes
1answer
1k views

Rpart using Caret changes names of Factors

If I have a factor e.g. sexe with two levels MALE and FEMELLE let's say, using rpart alone I get splits that say for example Sexe = Male and then a yes no split. However using rpart with caret I get a ...
3
votes
2answers
471 views

Can the alpha, lambda values of a glmnet object output determine whether ridge or Lasso?

Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On ...
3
votes
1answer
3k views

Is it necessary to split dataset for cross validation?

I am using caret package in R for training dataset and cross validation process. I am confused about cross validation process. Now, i am splitting the dataset to two subset, training and testing; <...
3
votes
1answer
1k views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa [closed]

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
3
votes
2answers
7k views

What does the varImp function in the caret package actually compute for a glmnet (elastic net) object

I am fitting an elastic net model with glmnet via the caret package with 189 predictors and a binomial criteria (a,b) ...
3
votes
1answer
17k views

Understanding the output of C5.0 classification model using the CARET package

The C5.0 classification model was used in this 4-class problem data with $N_{train}$=165, $P$=11, using caret R-package by ...
3
votes
1answer
3k views

Question about performing k-fold CV with caret

I have read the help manual of caret carefully: see A Short Introduction to the caret Package. In its example, I found it split the data with createDataPartition before a model training. ...
3
votes
1answer
972 views

Other distances than euclidean distance in knn [closed]

Suppose I want to fit a k-nearest-neighbour using caret package in R: ...
3
votes
1answer
1k views

Building random forest and svm in R caret take a very long time

I have searched for that problem but I haven't found a straight forward answer, so I am working with about 1.4 million numeric values and a few train() funtions. Now the problem is with "svmradial" ...
3
votes
1answer
174 views

Diagnosing linear skew of residuals in boosted tree

I have trained a boosted tree regression with the following code (out of the caret and gbm packages: ...
3
votes
1answer
1k views

How do I rank coefficients returned from a ridge regression?

I am running a ridge regression using GLMNET (alpha = 0) and would like to interpret the coefficients returned. I know there isn't really a significance test for this, but can I at least rank the ...
3
votes
1answer
2k views

How to interpret this cross-validated sparse LDA figure using CARET package?

Training data with $p$ =11 predictors and $n$ =165 with 4-class problem was cross-validated (5 times repeated 10-fold CV) using the sparse LDA (aka SDA) using caret ...
3
votes
1answer
3k views

Time-series machine learning methods and R packages

I am trying to determine how to use machine learning models such as for eg., random Forest with (non-financial) time-series data. Using an example, suppose we wanted to find based on monthly scores ...
3
votes
2answers
1k views

How to prepare a dataset for text classification

I would like to compare some algorithms for performing sentiment classification (Naive Bayes, SVM, and ...
3
votes
1answer
4k views

What is the best strategy to train and validate classification using PLS-[classifier] in caret package?

I have 4 clusters (see plot below) extracted from data of medical samples N=218 measured for 11 genes/predictors P=11 by this ...
3
votes
1answer
221 views

Caret: gradient vs gam boosting

What is the difference between a boosted additive model (e.g. caret model: gamboost) and a general stochastic gradient boosting model (caret model: gbm)? A gradient boosting model is additive by ...
3
votes
1answer
2k views

Approximation of a quadratic function with neural networks

I've set a simple function (y = x^2) to generate data to be used to train a neural network. The goal is to do predictions with new data. I've been trying to play around with the parameters of the ...
3
votes
2answers
898 views

gbm with caret when tuning grid has n.trees [closed]

While using caret package to tune a gbm model, suppose the tuning grid has n.trees=c(100,200)...
3
votes
1answer
1k views

Identical variable importance values for different model types

I trained two different caret models on the same multi-class training data using repeated cross validation and computed the variable importance. What strikes me, is that for both models varImp returns ...
3
votes
2answers
3k views

Results from rfe function (caret) to compute average metrics - R

I am computing a SVM-RFE model with the rfe function of the caret package, but I am a bit confused about the results. My code is:...
3
votes
1answer
2k views

Custom resampling method in caret

I need to create a custom resampling method in R package caret where: For each leave-pair-out-cross-validation, from the training set I derive new data using a function I implemented. Then it is used ...
3
votes
0answers
962 views

Recursive feature elimination (rfe) performs poorly with binary outcome

Backward elimination with random forests does not work as expected in a simple test case with binary outcome and three continuous predictors. Below, I generate a binary outcome based on a single ...

1
2
3 4 5
9