New answers tagged


The model used is the one trained on the whole of the training sample using the hyperpameters yielding the optimal average performance based on the CV procedure defined.


Yes, do make sure you are testing unknown patients. (I work with highly multivariate data also with multiple measurements per subject and have met situations where not splitting train patients vs. test patients would underestimate the prediction error by an order of magnitude!)


Your understanding is correct; the data used from the train function essentially act as train and validation sets; they do not reflect "true performance" on unseen data. Yes, we will have to use a separate test set to assess the performance of the particular algorithm on unseen data. To that extent, the use of nested cross-validation might be beneficial for ...


By default, caret will estimate a tuning grid for each method. However, sometimes the defaults are not the most sensible given the nature of the data. The tuneGrid argument allows the user to specify a custom grid of tuning parameters as opposed to simply using what exists implicitly.

Top 50 recent answers are included