I'm trying to find a simple way to split some longitudinal data into a training and test set. I'm familiar with using the Caret package in R to make stratified splits, but only with wide-form data. It looks like somebody has written a function to do this in Python, but I haven't learned that language yet.

In my case, I'd like to make a stratified split on some outcome classification (that, incidentally, does not change over time), on a data set where each individual has more than one observation, in such a way that if an individual is in one of the training/test sets, then all of their observations are in that same set.

I'd like to avoid having to transpose, then split, then transpose both training and test sets back to longitudinal format.

The only way I can think to do this (so far) is in the following code that I built using various sources on this site (1, 2), but I'm not sure that this is (a) 100% accurate or (b) if there isn't a better solution.

train <- data %>%
  select(ID, outcome) %>%
  distinct %>%
  group_by(outcome) %>%
  sample_frac(0.8) %>%

test <- data[!(data$ID %in% train$ID), ]
  • $\begingroup$ Looks like there's no stratification for outcome. Why do you want to avoid reshaping the data to wide and back to long? $\endgroup$ – lukeA Mar 8 '16 at 12:17
  • $\begingroup$ Fair question! I have a couple of reasons: 1) I honestly think there's got to be a more efficient way! 2) I'm more familiar with transposing in SAS, in which transforming the number of covariates that I have (~40 or so) can be prone to error; I presume the same issue may apply in R. 3) I can reshape data in R to some extent, but not fluently yet. $\endgroup$ – Robert S. Mar 10 '16 at 18:27
  • $\begingroup$ I too would like an answer to this question.... $\endgroup$ – theforestecologist Nov 5 '16 at 4:02

Just use sample() to choose some number of groups, after converting your id to factor

For example:

smp_size <- floor(0.80 * length (unique (iris$Species)))
iris %>% filter(Species %in% sample(levels(Species),smp_size))
  • $\begingroup$ is it rational to split it only per factors in panel data? because it looks we completely ignored time series and act with panel like a cross sectional dataset. $\endgroup$ – ssssoooo Dec 4 '19 at 13:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.