2
$\begingroup$

I'm trying to find a simple way to split some longitudinal data into a training and test set. I'm familiar with using the Caret package in R to make stratified splits, but only with wide-form data. It looks like somebody has written a function to do this in Python, but I haven't learned that language yet.

In my case, I'd like to make a stratified split on some outcome classification (that, incidentally, does not change over time), on a data set where each individual has more than one observation, in such a way that if an individual is in one of the training/test sets, then all of their observations are in that same set.

I'd like to avoid having to transpose, then split, then transpose both training and test sets back to longitudinal format.

The only way I can think to do this (so far) is in the following code that I built using various sources on this site (1, 2), but I'm not sure that this is (a) 100% accurate or (b) if there isn't a better solution.

library(dplyr)
set.seed(1)
train <- data %>%
  select(ID, outcome) %>%
  distinct %>%
  group_by(outcome) %>%
  sample_frac(0.8) %>%
  left_join(data)

test <- data[!(data$ID %in% train$ID), ]
$\endgroup$

closed as off-topic by mdewey, Tim, Peter Flom Aug 17 '17 at 12:33

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – mdewey, Tim, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Looks like there's no stratification for outcome. Why do you want to avoid reshaping the data to wide and back to long? $\endgroup$ – lukeA Mar 8 '16 at 12:17
  • $\begingroup$ Fair question! I have a couple of reasons: 1) I honestly think there's got to be a more efficient way! 2) I'm more familiar with transposing in SAS, in which transforming the number of covariates that I have (~40 or so) can be prone to error; I presume the same issue may apply in R. 3) I can reshape data in R to some extent, but not fluently yet. $\endgroup$ – Robert S. Mar 10 '16 at 18:27
  • $\begingroup$ I too would like an answer to this question.... $\endgroup$ – theforestecologist Nov 5 '16 at 4:02
2
$\begingroup$

Just use sample() to choose some number of groups, after converting your id to factor

For example:

smp_size <- floor(0.80 * length (unique (iris$Species)))
iris %>% filter(Species %in% sample(levels(Species),smp_size))
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.