20
$\begingroup$

I have a dataframe which contains values across 4 columns:

For example:ID,price,click count,rating

What I would like to do is to "split" this dataframe into N different groups where each group will have equal number of rows with same distribution of price, click count and ratings attributes.

Any advice is strongly appreciated, as I don't have the slightest idea on how to tackle this !

$\endgroup$
5
  • $\begingroup$ Are you just looking to create N separate data frames that are disjoint subsets of the original? What do you mean by the "same distribution" of price, click count, and ratings? $\endgroup$
    – Alex A.
    Commented Mar 30, 2015 at 18:58
  • $\begingroup$ Yes, looking for the subsets of the original data frame. On your second question, suppose I have values of visit counts from 1 to 10 and decided to create 3 different subsets, so will select some rows in each group from 1 to 4 visit count bucket, some rows from 4 to 7 visit count bucket and some from 7 to 10 visit count bucket and this should be satisfied with respect to all attributes (price, click count and rating). It's like sampling the data into different groups with equal probability of attributes. Hope this helps. $\endgroup$
    – rajpal
    Commented Mar 30, 2015 at 19:18
  • $\begingroup$ possible duplicate of R language: how to split a data frame $\endgroup$
    – Alex A.
    Commented Mar 30, 2015 at 19:40
  • $\begingroup$ The question requests a split that preserves the distributions of the variables. Without further information, it's not possible to determine the correct method with which to approach this problem. I vote to migrate this to CV.com $\endgroup$
    – DWin
    Commented Mar 30, 2015 at 20:14
  • $\begingroup$ Do you mean to preserve only the marginal distributions or the joint distribution? $\endgroup$ Commented Aug 27, 2018 at 16:22

4 Answers 4

19
$\begingroup$

If I understand the question correctly, this will get you what you want. Assuming your data frame is called df and you have N defined, you can do this:

split(df, sample(1:N, nrow(df), replace=T))

This will return a list of data frames where each data frame is consists of randomly selected rows from df. By default sample() will assign equal probability to each group.

$\endgroup$
0
9
$\begingroup$

Edit: The minDiff package has been superceded by the anticlust package.


This is a very late answer, but I found this page while googling whether the problem as stated has ever been discussed anywhere. Maybe my answer will help if someone finds this page from now on.

I wrote an R package, which does exactly what the question asked for: it takes a data.frame and creates N different groups while trying to minimize the differences between groups in one or several criteria. It uses a simple method based on repeated random assignment, which is also the suggested method in the approved response.

This is the link to the package minDiff:

To tackle the stated problem, you could use:

library(minDiff)
assigment <- create_groups(dataframe, criteria_scale = c("price", "click count", "rating"), sets_n = N, repetitions = 1000)

The repetitions argument will determine how often you randomly create different groups. The best assignment - the one that has minimal differences between groups - will be returned.

$\endgroup$
9
$\begingroup$

This can be solved with nesting using tidyr/dplyr

require(dplyr) 
require(tidyr)

num_groups = 10

iris %>% 
   group_by((row_number()-1) %/% (n()/num_groups)) %>%
   nest %>% pull(data)
$\endgroup$
2
  • 1
    $\begingroup$ Knew there had to be something simple but couldn't think what - love this solution @Holger Brandl! If like me you care about group size not number of groups, you can replace n()/num_groups() with my_group_size $\endgroup$ Commented Oct 1, 2020 at 17:46
  • $\begingroup$ This should be the accepted answer! So easy. $\endgroup$
    – Tyler R.
    Commented Mar 3, 2022 at 16:00
7
$\begingroup$

Although Alex A's answer gives an equal probability for each group, it does not meet the question's request for the groups to have an equal number of rows. In R:

stopifnot(nrow(df) %% N == 0)
df    <- df[order(runif(nrow(df))), ]
bins  <- rep(1:N, nrow(df) / N)
split(df, bins)
$\endgroup$
2
  • $\begingroup$ @whuber Can you propose an answer to that here? $\endgroup$ Commented Nov 6, 2016 at 17:37
  • $\begingroup$ The answer ought to depend on what "same distribution" means. It appears the question is asking to cluster observations based on four variables, with each cluster having the same number of observations. There are myriad ways to do this. $\endgroup$
    – whuber
    Commented Nov 6, 2016 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.