Let's say I have a feature set
X that is of size
m x 2, where
m are samples, and the two features are
timepoint is a variable which ranges from 1 to 100, and represents the time at which the sample was taken, and
panel is a categorical variable that is a unique identifier for each time series. My outcome variable
y is of shape
m x 1 and is a continuous variable. I want to train a regression model.
In other words, a small excerpt with timepoints 1-3 might look like:
y timepoint panel 1.5 1 1 5.4 2 1 6.7 3 1 3.3 1 2 3.4 2 2 4 3 2
How do I properly split this data into train/test, without resulting in leakage between the two sets?
- Train/test split on
X, ignoring the grouping variable (i.e. randomly take 33% of the rows in
Xas my test)?
- Split based on the grouping variable (i.e. randomly take 33% of
the groups in
My hunch is that #2 is the proper way to do this. Suggestions?