3
$\begingroup$

I have a DataFrame like this:

import pandas as pd
pd.DataFrame({"x": [10, 15, 0, 5, 21, 2, 3, 99],
              "y": [123, 321, 25, 85, 721, 1232, 32, 994],
              "country": ["a", "a", "b", "a", "a", "b", "c", "c"],
              "age_group": ["e", "r", "t", "t", "r", "r", "e", "t"]})

I would like to cross-validate a model, I want to not have the same groups in test and training. E.g. a fold were country "c" and "b" and age_group "e" are in the test would lead to the instance 0, 2 and the last three instances to be in the test set.

I can't find a library for this and I do not have an idea for a simple implementation. Is there something like sklearn.model_selection.GroupKFold for more than one group? Is there an easy way to solve this problem? (I do not want "leave one group out" since in reality, my two group columns have high cardinality.)

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

You can create a new column (country_age) with values that concat the value of country and age_group by _. For example,

df['country_age'] = df['country'].astype(str) + '_' + df['age_group'].astype(str)

a_e, a_r, b_t, ...

Then use the GroupKFold (maybe StratifiedKFold) for this new column.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, I don't know why I didn't come up with this simple solution back then :D $\endgroup$
    – PascalIv
    Dec 10, 2021 at 9:21
-1
$\begingroup$

Here is a kernel containing the a Group Stratified CV function:

https://www.kaggle.com/jakubwasikowski/stratified-group-k-fold-cross-validation

This should allow to separate the folds while keeping constant proportion of the classes specified in some groups.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, but I have two group columns, not one. The point is not the stratification, but having two group columns $\endgroup$
    – PascalIv
    Dec 10, 2019 at 10:04
  • $\begingroup$ The function allows for multiple groups. But then again, I realize you wrote you want NOT to have the same groups in test and training, so it's not really clear what the goal is. Could you please clarify? $\endgroup$
    – Davide ND
    Dec 10, 2019 at 10:32
  • $\begingroup$ sklearn.model_selection.GroupKFold also allows for multiple groups, but I have two columns, each with multiple groups. Imagine you have data for humans, with different sex and eye colors. One fold, for example, should train on "male" and "green eyes" so in the test set there should be no males and no people with green eyes. $\endgroup$
    – PascalIv
    Dec 10, 2019 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.