5
$\begingroup$

Background

I was asked to perform the randomization for a small study with 3 conditions $A$, $B$ and $C$. The three conditions are three different smartphone types. The participants enter the study sequentially. Because the number of participants is rather small (i.e. around $70$), I opted to use the randomization algorithm proposed by Zhao & Weng (2011) that provides balanced treatment allocation.

After I performed the randomization, I got a list that looks something like this:

$$ \begin{array}{l|l} \text{Participant no.} & \text{Condition} \\ \hline 1 & A \\ 2 & C \\ 3 & C \\ \color{red}{4} & \color{red}{B} \\ \color{red}{5} & \color{red}{B} \\ \color{red}{6} & \color{red}{B} \\ \color{red}{7} & \color{red}{B} \\ 8 & C \\ \ldots & \ldots \\ \end{array} $$

The problem

After I provided the randomization list, the client said that they don't want to randomize more than 3 subsequent participants to the same condition. This is because they have a limitted number of smartphones of each type. So the red part in the list above is problematic.

Questions

  1. Would it be harmful to repeat the randomization with different RNG seeds until the condition of the client is satisfied? Or would this defeat the purpose of the randomization?
  2. Is there an algorithm that guarantees a balanced allocation as well as the constraint outlined above?

References

Zhao W, Weng Y (2011): Block urn design — A new randomization algorithm for sequential trials with two or more treatments and balanced or unbalanced allocation. Contemp Clin Trials. 32(6): 953-961 (link)

$\endgroup$
8
  • 1
    $\begingroup$ @FransRodenburg Yeah, block randomization is a good idea that would balance the phones per block/batch. Unfortunately, the participants don't enter in batches but continuously and keep the phone for a certain amount of time (e.g. a week) before returning it. I'm not sure if the participants enter at a steady rate (I suspect they don't). $\endgroup$ Commented Jul 7, 2019 at 10:50
  • 1
    $\begingroup$ @IsabellaGhement You're right, that's an error. I updated the example to include 4 "B"s. $\endgroup$ Commented Jul 7, 2019 at 17:48
  • 2
    $\begingroup$ Following up on the comment from @FransRodenburg is there some reason why defining "virtual batches" each of size 3, representing each sequential set of 3 participants, couldn't be used even if participants show up continuously? Then there would never be more than 2 phones of the same type distributed in sequence. (Batches of size 6 pose a risk of runs of 4.) Maybe not strict randomization but might be good enough for this application. Not knowing the times of participant entry would seem to pose the same problems (if any) in any randomization scheme. $\endgroup$
    – EdM
    Commented Jul 7, 2019 at 21:52
  • 1
    $\begingroup$ My suggestion might have helped solved the issue at hand but didn't answer the interesting question in the title. I'll hold off for a while on posting an answer, to try to enhance the chances of someone finding this as unanswered and then addressing in an informed way the issues involved in making choices among RNG seeds during the design phase of a project. $\endgroup$
    – EdM
    Commented Jul 8, 2019 at 14:20
  • 1
    $\begingroup$ Thinking about this for a week has made me appreciate some additional issues here, now addressed in an answer. $\endgroup$
    – EdM
    Commented Jul 15, 2019 at 17:26

1 Answer 1

3
$\begingroup$

There is no problem in general with throwing away a recommendation made by a randomization process if it doesn't fit some constraint. That's just rejection sampling. For example, you can get a random sequence of integers from 1 through 5 by rolling a die repeatedly, accepting any value in that range, and throwing away any values of 6 that appear.

The problem here is the constraint: no more than 3 sequential participants with the same condition. That poses a risk of selection bias regardless of how you proceed.

The deterministic aspect of a permuted-block design can lead to selection bias. For example, in the size-3 blocks that I initially suggested in a comment (without thinking enough about selection bias), once the first 2 assignments are made within a block the third is determined. A clinician's recommendation of a patient for inclusion in an unblinded trial can be unconsciously influenced by any sense that the trial's choice for that patient might be "right" or "wrong" based on clinical judgement. Any partial information about the next "random" treatment in line poses this problem.

Matts and Lachin discuss two types of selection bias in permuted-block designs: a potential bias from an "excess of correct guesses of treatment assignments beyond that expected by chance" and knowing the next treatment condition with certainty. This smartphone trial could face both problems if it's known that no more than 3 of the same model can ever be assigned sequentially during the trial. For example, if the last 3 phones distributed in the trial were SamdroidX models but a salesperson thinks that a SamdroidX is "right" for the next customer, that could lead to a push for a sale rather than proposing the trial.

The Zhao and Weng paper that you link (manuscript freely available here) both provides a nice overview of the problem and demonstrates a way to minimize selection bias while providing balanced allocation with their block urn design. But once you impose the constraint of no more than 3 in a row by choosing among multiple random seeds, the risk of selection bias increases.

If you were to use a permuted-block design, a larger block size than 3 or using different block sizes might be preferable to fixed size-3 blocks.* For example, alternating size-3 and size-6 blocks would prevent runs of more than 3 while providing harder patterns to recognize. Or you could design larger-size blocks, then discard blocks or rearrange the order among blocks to meet the constraint.

However you proceed, the constraint of sequences of no more than 3 of the same condition will risk selection bias. The magnitude of the risk will depend both on the assignment scheme and on details of how the trial would be implemented. It might help to use the types of analyses described by Matts and Lachlin and by Zhao and Weng to compare proposed schemes with respect to their risks of that bias.


*With size-3 permuted blocks there would never be more than 2 of the same model assigned in a row. I suppose that if those implementing the trial were expecting a limit of 3 and didn't know about this inherent limit of 2 then selection bias could be minimized. The less that those implementing the trial know about the design, the less the chance of selection bias. But people can be pretty good at recognizing patterns and might unconsciously internalize the design.

$\endgroup$
2
  • $\begingroup$ (+1) These are excellent points, thanks for following up on this question! It seems to me that you are saying that the potential for selection bias is an "innate" problem for randomized block designs because one could guess or know the treatment allocation, is that correct? Is it fair to say then, that, if we would assume that if the trialists are completely blinded concerning the next patient, selection bias would be unlikely for both scenarios (the block randomization and the repeated-seed-scenario)? One could also randomize the block size, say either 3 or 6, as you suggest. $\endgroup$ Commented Jul 15, 2019 at 17:46
  • 1
    $\begingroup$ @COOLSerdash The less the implementers know or can guess about the design, the less the risk of selection bias. Matts and Lachlin say "the potential for selection bias is completely eliminated if patients are randomized as a block, rather than as they arrive for entry," but that seems unlikely in your scenario. They also say: "if the treatment assignments are unmasked as they occur, then the permuted-block design is subject to selection bias." Knowledge of past assignments can provide (perhaps unconsciously) information about the next assignment. $\endgroup$
    – EdM
    Commented Jul 15, 2019 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.