11
$\begingroup$

I have a dataframe which contains values across 4 columns:

For example:ID,price,click count,rating

What I would like to do is to "split" this dataframe into N different groups where each group will have equal number of rows with same distribution of price, click count and ratings attributes.

Any advice is strongly appreciated, as I don't have the slightest idea on how to tackle this !

$\endgroup$
  • $\begingroup$ Are you just looking to create N separate data frames that are disjoint subsets of the original? What do you mean by the "same distribution" of price, click count, and ratings? $\endgroup$ – Alex A. Mar 30 '15 at 18:58
  • $\begingroup$ Yes, looking for the subsets of the original data frame. On your second question, suppose I have values of visit counts from 1 to 10 and decided to create 3 different subsets, so will select some rows in each group from 1 to 4 visit count bucket, some rows from 4 to 7 visit count bucket and some from 7 to 10 visit count bucket and this should be satisfied with respect to all attributes (price, click count and rating). It's like sampling the data into different groups with equal probability of attributes. Hope this helps. $\endgroup$ – rajpal Mar 30 '15 at 19:18
  • $\begingroup$ possible duplicate of R language: how to split a data frame $\endgroup$ – Alex A. Mar 30 '15 at 19:40
  • $\begingroup$ The question requests a split that preserves the distributions of the variables. Without further information, it's not possible to determine the correct method with which to approach this problem. I vote to migrate this to CV.com $\endgroup$ – DWin Mar 30 '15 at 20:14
  • $\begingroup$ Do you mean to preserve only the marginal distributions or the joint distribution? $\endgroup$ – kjetil b halvorsen Aug 27 '18 at 16:22
12
$\begingroup$

If I understand the question correctly, this will get you what you want. Assuming your data frame is called df and you have N defined, you can do this:

split(df, sample(1:N, nrow(df), replace=T))

This will return a list of data frames where each data frame is consists of randomly selected rows from df. By default sample() will assign equal probability to each group.

$\endgroup$
5
$\begingroup$

Although Alex A's answer gives an equal probability for each group, it does not meet the question's request for the groups to have an equal number of rows. In R:

stopifnot(nrow(df) %% N == 0)
df    <- df[order(runif(nrow(df))), ]
bins  <- rep(1:N, nrow(df) / N)
split(df, bins)
$\endgroup$
  • 3
    $\begingroup$ Your observation about the deficiencies of the accepted answer is a good one. However, your answer still does not address the part of the question that is of interest (and is the only reason it was not closed here): how do you achieve the "same distribution of price, click count and ratings attributes" in each group? $\endgroup$ – whuber Apr 15 '16 at 20:41
  • $\begingroup$ @whuber Can you propose an answer to that here? $\endgroup$ – Léo Léopold Hertz 준영 Nov 6 '16 at 17:37
  • $\begingroup$ The answer ought to depend on what "same distribution" means. It appears the question is asking to cluster observations based on four variables, with each cluster having the same number of observations. There are myriad ways to do this. $\endgroup$ – whuber Nov 6 '16 at 20:40
5
$\begingroup$

This is a very late answer, but I found this page while googling whether the problem as stated has ever been discussed anywhere. Maybe my answer will help if someone finds this page from now on.

I wrote an R package, which does exactly what the question asked for: it takes a data.frame and creates N different groups while trying to minimize the differences between groups in one or several criteria. It uses a simple method based on repeated random assignment, which is also the suggested method in the approved response.

This is the link to the package minDiff:

To tackle the stated problem, you could use:

library(minDiff)
assigment <- create_groups(dataframe, criteria_scale = c("price", "click count", "rating"), sets_n = N, repetitions = 1000)

The repetitions argument will determine how often you randomly create different groups. The best assignment - the one that has minimal differences between groups - will be returned.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.