Sheesh, I'm really confused.

So, I have a dataset of individuals, and a dataset of their friendships. I want to test whether a particular (numeric) variable is correlated among friends. To do this, I am bootstrap resampling individuals.

My original data looks like:

Joe   1
Sam   3
Pete  2
...   ...

with a friendship network like:

Joe       Pete
Joe       Sam
Sam       Jill
...       ...

My basic statistic is created by correlating friend 1's variable with friend 2's variable. That is, I join my tables to create:

1         Joe     Pete     2
1         Joe     Sam      3
3         Sam     Jill     4

Standard errors will be wrong, of course, because each friend may appear many times in the data.

So, I resample individuals with replacement. When I do this, I might get Joe's name 2 times and Pete's name 3 times. I then want to recreate the friendship network and rerun my correlation. By doing this many times, I will get the sampling distribution of my statistic.

But, how many times should the Joe-Pete friendship appear in the new data? I guess 2*3 = 6 times... is that right?

Examples in R would be welcome, but mostly I want to know how to think about this. (I'm even confused... is it legitimate to resample at individual level, given that by assumption friends' data is correlated... argh!!!)

  • 1
    $\begingroup$ It is not clear for me what and why do you want to resample..? Your second question: "But, how many times should the Joe-Pete friendship appear in the new data?" is even more unclear -- bootstrap will give you random number of pairs so what are you counting here..? $\endgroup$
    – Tim
    Jun 23, 2016 at 13:43
  • $\begingroup$ To avoid any ambiguity, can you precise if your variable is numerical, categorical or ordinal ? $\endgroup$
    – brumar
    Jun 23, 2016 at 13:55
  • 1
    $\begingroup$ Could you be more specific about what you mean by "correlated among friends"? One possible interpretation--which assumes you didn't really mean "correlated," but meant "dependent"--is that the distribution of unordered pairs of the variables among the friends somehow differs from the distribution of such pairs among non-friends. I suspect you implicitly have in mind some metric to compare values of those variables and that you might mean that the variables tend to be closer between friends than between non-friends. But I'm just guessing... . If I'm correct, what is the metric? $\endgroup$
    – whuber
    Jun 23, 2016 at 15:13
  • 1
    $\begingroup$ @dash2 "correlation" has a precise meaning in statistics, it implies that a mathematical relation (by default linear) describes well their dependence. Is it really what you look for ? Is the reformulation suggested by whuber acceptable to you ? If not, why? $\endgroup$
    – brumar
    Jun 23, 2016 at 17:35
  • 1
    $\begingroup$ Although you might get some useful answers here, what you are trying to do is probably very difficult. I see that "Bootstrap a network, non-parametrically" is one of the items on Cosma Shalizi's list of "Things I wish I knew how to do" in his notebook pages (and he is one of the world's leading statisticians at the moment.) $\endgroup$
    – Flounderer
    Jun 23, 2016 at 22:06

1 Answer 1


I don't feel ultra confident on my answer, but I would be interested to be reviewed if necessary. If I were you, I would go for a permutation test because it does not suffer from the non-independence you are pointing. Pick a good statistic that represents the effect you are studying. Something like $$S=\sum_{i,j} |V(i)-V(j)|*I_{i,j}$$ with $I_{i,j}=1$ if $i$ and $j$ are connected (0 otherwise). After computing this statistic for your data-set, do it for each permutation of your data by shuffling the VARIABLE column.

NAME  VAR             NAME  VAR    NAME  VAR    NAME  VAR
Joe   1               Joe   2      Joe   5      Joe   5
Sam   3               Sam   3      Sam   3      Sam   3       ....
Pete  2               Pete  1      Pete  2      Pete  1
John  5               John  5      John  1      John  2
...   ...             ...   ...    ...   ...    ...   ...
Original data-set     data-set1    data-set2    data-set3      ....

In r this can be done with the built-in function sample on the column you wish to shuffle. As we do a permutation test (not a bootstrap procedure) there is no replacement.

For each new data-set, compute your $S_k$ score. If your hypothesis is true, your $S_k$ scores must be most of the time higher than $S$.

Your p-value is the number of times your permuted set get better or equal $S_k$ score than your original S.

  • 1
    $\begingroup$ This looks like a good idea--but could you be more explicit about what you mean by "breaking and redistributing"? The validity of the result appears to rely on exactly how that is carried out. $\endgroup$
    – whuber
    Jun 23, 2016 at 15:15
  • 1
    $\begingroup$ This is a good idea and I might well do it. But, I think the original question (how to resample a network?) still deserves a specific answer, because I've seen it done a fair bit. $\endgroup$
    – dash2
    Jun 23, 2016 at 21:20
  • $\begingroup$ I think you are right. But I personally think that there is no general answer. It depends on your goal. You can chose to preserve the structure and only resampling the values (as I did in my answer). This is in line with the sole interpretation of your goal I can come up with. Another goal being given, I guess you can come up with procedures could maybe involve resampling the network itself. $\endgroup$
    – brumar
    Jun 23, 2016 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.