1
$\begingroup$

I'm trying to downsample from a matrix of numbers. Each number is the number of times we saw a particular event. By downsample I mean I want to pseudo-randomly select values from each row so they equal approximately 1/30th of what they currently are.

Attached is roughly what I want to get out. It's a little more exact than I actually want but this is what I meant.

An example before and after table
        Col 1   Col 2
Frank   930     900
David   10000   12000
Rosa    7500    6900
Simone  500     460

        Col 1   Col 2
Frank   30      28
David   300     250
Rosa    210     310
Simone  18      40

The goal is to simulate if we ran an experiment fewer times. All values in the matrix are between 0 and about 200k. Currently each column has about 45k rows and sum to between 30 and 40 million. The goal is to downsample by about 30 fold so I end up with each column summing 1-1.5 million. It could go up to about 2 million if necessary but can't really go below 1 million. I don't want to just divide each cell by 30 or sample randomly.

My initial plan was to populate a list with each row name equal to the number of times it appeared than randomly select from that list a certain number of times. Someone suggested I try the function rbinom but I couldn't work out how to use it properly. I've included what I did for each variant of rbinom and roughly what I got out of each one below. If anyone can point me in the right direction with it that would be great. This was attempting to reduce by about 10.

I want to get out a matrix with the same number of rows as I started with where each column sums to between 1 and 1.5 million. The order of the rows doesn't matter as long as they keep their row names.

I'm working in R.

#Produces a list that's all 0s.  
dbinom(foo$col1,1500000, 0.1)

#Produces a list that's all 0s    
pbinom(foo$col1,1500000, 0.1)

#Produces a list that's mostly NAs
qbinom(foo$col1,1500000, 1/10)

#Gives insanely high numbers (~150000) on every row
rbinom(foo$col1,1500000, 0.1)
$\endgroup$

closed as off-topic by mkt, Peter Flom May 16 at 12:07

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – mkt, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ It's unclear what you mean by "each column to go from." Could you perhaps show a tiny example of what this operation should do? $\endgroup$ – whuber May 9 at 14:40
  • $\begingroup$ Sorry, each column currently has about 40k rows, the sum of the column is about 30 million, I want to rbinom to reduce the total of that column to about 1.5 million rather than either randomly sampling it or just reducing each row about 30 fold. Does that help? $\endgroup$ – Sethzard May 9 at 15:05
  • $\begingroup$ It helps a little, but there remains much more to be specified. Do you need to select the same rows in each column? Or maybe just the same number of rows in each column? Or doesn't it matter? What will the output be--a set of samples of columns or a matrix? Must the rows appear in the same order they originally did? What can you say about the elements of the matrix? For instance, are they non-negative? Do they have a known bound? What does "~" really mean? That is, if we cannot get the column totals to exactly equal their targets, how do we determine what is close enough? $\endgroup$ – whuber May 9 at 15:53
  • $\begingroup$ We're almost there: but could you explain what "downsampling" might possibly mean when the output matrix is supposed to have the same number of rows as the input matrix? Usually, "sampling" is a form of selecting or subsetting things. $\endgroup$ – whuber May 9 at 17:12
  • $\begingroup$ Because the description is still ambiguous, why not display an example of what you want using a tiny matrix? $\endgroup$ – whuber May 10 at 13:27

Browse other questions tagged or ask your own question.