4
$\begingroup$

I need to incorporate distance buffers into the selection of treatment and control units in a randomized-controlled trial in order to minimize spillovers between arms. Cluster in this study is a village.

Specifically, I have 2 rural provinces, each with 200+ villages / clusters. For simplicity, let's say I have 1 treatment arm and a control arm. I need an appropriate distance (e.g. 2 - 3 km) between villages selected into the study.

$\endgroup$
  • $\begingroup$ Is the intervention being applied at the village level? Or will you select individuals within selected villages? $\endgroup$ – Placidia Jan 13 '15 at 17:41
  • $\begingroup$ intervention is at the village level $\endgroup$ – cece Jan 13 '15 at 17:55
3
$\begingroup$

Option 1: Modify the sampling frame

This is tricky because you can be greedy. For example villages A, B and C lie on a straight line 2km apart from each other, then you might want to drop B from the sample, rather than A and C.

One approach is to redefine the sampling frame:

  1. List all villages in the sampling frame and construct a distance matrix.
  2. Draw a network graph where each node represent a village connected to its nearest neighbor villages. E.g. A ---- B ---- C ---- D.
  3. If A and C are <3km from B give B a score of 2. A gets score of 1, ditto for C unless D is <3km away. The score measures the number of neighboring villages that are too close.
  4. Delete the village with the highest score and recompute the graphs and scores. E.g. after deleting B, A and C have score of 0.
  5. Continue until all scores are 0.

If several villages have the same highest score, pick one at random. If you are really greedy, pick one and run the routine see what happens, then start again but pick another, and so on, until your sampling frame ensures no two villages can be closer than 3km whilst dropping the fewest number of villages from the original sampling frame.

In this case some villages in the original sampling frame have zero probability of ever being chosen since they drop out from the modified frame (your population changes).

Option 2: Modify the sample

If you use randomization inference, then simply decide on an exact procedure to drop villages from the sample such that none are too close. This defines your sampling procedure. Use this sampling procedure to compute your test statistics.

In this case most villages will have some positive probability of being chosen (your population does not change).

Option 3: Model the spillover effects.

Don't try to avoid the spillover, instead used distance information to model the spillover. Spillover is an interesting question.

My preference is this last option, unless you have very low power.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

At a practical level, @Fred's suggestions would possibly work. But strictly speaking, there are problems with that approach. If you are using finite population (design based) methods of inference, then it is important that the probability that any two individuals both appear in the sample be greater than 0. But if Bob from Little Whingeing is in the sample, then Joe from Greater Whingeing, 1 km away, is excluded. There is 0 probability that we get both Bob and Joe. @Fred's option 2 doesn't get around this. Option 1 redefines the population, so your inference is no longer about the real world population you have. You will end up culling a lot of villages in high density areas, while less populated areas have all their villages remaining in the frame.

You didn't mention how many villages you are going to select, but if it is a small number, you could try a matched-pairs design. Explicitly select representative pairs of similar villages, and randomize intervention and control to the pair. You would analyse the results as a randomized block design, in which each pair would be a block. If all you care about is the comparison between case and control, it's OK to turn the villages into fixed, block effects.

I don't know what kind of geography you have, but perhaps you could combine nearby villages and apply the intervention to the whole group. If they are so close that they function as a unit, perhaps they should be randomized as a unit. This also redefines the frame, but everyone is still included, and you don't have the Bob/Joe problem. This assumes that you have clusters of villages, then green space, then another cluster. If it's just villages everywhere, this won't work.

Modelling the spillover, Fred's option 3, sounds interesting, although challenging.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ 1. "if Bob from Little Whingeing is in the sample, then Joe from Greater Whingeing, 1 km away, is excluded. There is 0 probability that we get both Bob and Joe." Yes but when A and B are <3km away you have to drop one. If you do so by tossing a coin, then both have a chance of being in the sample. 2. The point about "the real world population" is a little forced. I am not sure that the original sampling frame represent any specific population of interest. More likely a convenience sample. Just making it more convenient. 3. Spillovers: See Gerber & Green, Field Experiments. $\endgroup$ – Fred Jan 13 '15 at 18:52
  • $\begingroup$ Note that matching does not overcome the issue that A and B can never both be in the study group. The problem is one of lack of independent sampling, not so much zero probability of sampling (as per previous comment on tossing a coin) $\endgroup$ – Fred Jan 13 '15 at 19:01
  • $\begingroup$ Bob and Joe each have a non-zero probability of being in the sample, but Prob(Joe|Bob) = 0. To calculate variances in finite population samples, you need the joint probabilities as well, and they need to be > 0. $\endgroup$ – Placidia Jan 13 '15 at 19:04
  • $\begingroup$ Yes that P(Joe|Bob)=0. But my point is that matching does not overcome this. As for variance, it depends what you are trying to estimate. If the goal is the ATE with zero spillover then you want P(Joe|Bob)=0 or P(Bob|Joe)=0, or you will get the wrong estimate and variance. Thus, the event space changes -- only a subset of all possible samples from the original population are allowed. This is why I suggested randomization inference. Note that if you do not randomize between dropping Bob or Joe, but always drop Bob say, then you are restricting the space of possible samples even more. $\endgroup$ – Fred Jan 13 '15 at 19:17
  • $\begingroup$ The purpose of matching would be avoid random selection of clusters altogether, and simply randomize the treatment arms, adjusting for "village effects". This assumes you can match sensibly and that the number of villages is not too large. It also limits inference to the comparison between treatments. In practice, however, I think either of your options would probably yield similar results to unconstrained randomization in a world without spillover. $\endgroup$ – Placidia Jan 13 '15 at 19:48
1
$\begingroup$

My previous answer (and comments) suggested a matched pairs design. If you want to keep the original idea: a random selection of villages subject to the constraint of "no close neighbours", you can simply make a random selection until you find a bunch of villages that are suitably distant. Chuck any randomization that does not meet the criteria. Presumably, in a second round of randomization, you will allocate treatment and control arms.

The randomization scheme described above actually does deliver a random selection of villages from the set of "villages that are suitably distant". You don't have to create a frame of distant villages. This randomization effectively does the work for you.

The downside: potential bias is introduced since the sample no longer represents the true population. For example, if this is the UK, you will be undersampling the Home Counties and oversampling Scotland and Ulster. However, if the treatment effect is not affected by whatever governs the distribution of villages, your comparison will not be biased.

And furthermore: you can account for the sampling bias mentioned above by weighting the responses in terms of village density within (say) counties. If village distribution is homogeneous across the rural provinces you are studying, you don't need to worry about this.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What you are suggesting here is the same as my Option 2, I think. Also, the estimate is not biased for what you are trying to estimate, which is the effect absent any spillover in villages >3km apart, say. That would exclude the home countries. If we wanted to compute this same effect in the home counties then we need a double counterfactual: What would have happened had they not received the treatment in the hypothetical world where these villages are not close together. Changing the sampling frame makes this more explicit. $\endgroup$ – Fred Jan 14 '15 at 14:38
0
$\begingroup$

I wrote a STATA command called clustermax which selects a maximum number of point clusters for user-selected distance parameters. It is based on the principle of resolving spatial conflicts as outlined by Fred in Option 1.

Download:
github.com/FMBarba/clustermax

Example use from the helpfile:

Generate 100 random points
    . set obs 100
    . set seed 1
    . gen lat = 40  + runiform()
    . gen lon = -90 + runiform()*4

Set cluster size to exactly one; minimum between-cluster distance 20km:
    . clustermax lat lon, seed(1) gen(cluster) b(20) n(1)

Set minimum cluster size to one; minimum between-cluster distance 20km; maximum within-cluster distance 20km:
    . clustermax lat lon, seed(1) gen(cluster) w(20) b(20) n(1)

Set minimum cluster size to three; minimum between-cluster distance 25km; maximum within-cluster distance 15km:
    . clustermax lat lon, seed(1) gen(cluster) w(15) b(25) n(3)

Note: clustermax requires the packages geodist, tuples, distinct and ftools.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.