Stratified shuffle for normally distributed target variable

Question

When splitting data for a classification problem one is advised to use stratified shuffling in case the target variable is skewed toward a certain class. Indeed, Sklearn has a function for that.

Suppose now that we are splitting the data w.r.t to a target variable T that is normally distributed. Is there any similar tool or technique that could split a the data set into train/val sets so that the mean/variance of T is preserved as much as possible?

I understand that for large enough sets that's already the case but I am interested in practical applications where splitting the data into train/tes/val sets skews the mean and variance by a lot.

Answer 1

Stratified shuffling used in the case target variable takes discrete values. "The folds are made by preserving the percentage of samples for each class," stated in sklearn documentation. So your question is how I split my data with a continuous target variable such that distrb. is preserved.

One way to do it is bin your target variable:

• Use sklearn KBinsDiscretizer to bin your target variable
• Assign discrete labels to each bin
• Use stratified shuffling, with the binned variable as the target to split your data

With this technique, you should be able to preserve mean-variance in each fold given the number of bins determined accordingly.

To determine the number of bins: https://stats.stackexchange.com/a/862/293623