What's wrong with (some) pseudo-randomization

Question

I came across a study in which patients, who were all over 50 years of age, were pseudo-randomized by birth year. If the birth year were an even number, usual care, if an odd number, intervention.

It's easier to implement, it's harder to subvert (it's easy to check what treatment a patient should have received), it's easy to remember (the assignment went on for several years). But still, I don't like it, I feel like proper randomization would have been better. But I can't explain why.

Am I wrong for feeling that, or is there a good reason to prefer 'real' randomization?

Answer 1

You are right to be skeptical. In general, one should use 'real' randomization, because typically one doesn't have all knowledge about relevant factors (unobservables). If one of those unobservables is correlated with the age being odd or even, then it is also correlated with whether or not they received treatment. If this is the case, we cannot identify the treatment effect: effects we observe could be due to treatment, or due to the unobserved factor(s).

This is not a problem with real randomization, where we don't expect any dependence between treatment and unobservables (though, of course, for small samples it may be there).

To construct a story why this randomization procedure might be a problem, suppose the study only included subjects that were at age 17/18 when, say, the Vietnam war started. With 17 there was no chance to be drafted (correct me if I am wrong on that), while there was that chance at 18. Assuming the chance was nonnegligible and that war experience changes people, it implies that, years later, these two groups are different, even though they are just 1 year apart. So perhaps the treatment (drug) looks like it doesn't work, but because only the group with Vietnam veterans received it, this may actually be due to the fact that it doesn't work on people with PTSD (or other factors related to being a veteran). In other words, you need both groups (treatment and control) to be identical, except for the treatment, to identify the treatment effect. With assignment by age, this is not the case.

So unless you can rule out that there is no unobserved differences between the groups (but how do you do that if it isn't observed?), real randomization is preferable.

Answer 2

It is a good exercise to uphold contrarian views from time to time, so let me begin by offering a few reasons in favor of this form of pseudo-randomization. They are, principally, that it is little different than any other form of systematic sampling, such as obtaining samples of environmental media at points of a grid in the field or sampling every other tree in an orchard, and therefore this sampling might enjoy comparable advantages.

The analogy here is perfect: age was "gridded" by year starting at an origin of zero and assignment to the groups alternated along this (one-dimensional) grid. Some advantages of this approach are to guarantee wide, even dispersion of the sample across the field or orchard (or ages, in this case), which helps even out influences related to location (or time). This can be especially useful when theory suggests that location is the predominant factor in variation of response. Moreover, except for really tiny samples, analyzing the data as if they were a simple random sample introduces relatively little error. Furthermore, some randomization is possible: in the field we can randomly choose the origin and orientation of the grid. In the present case, we can at least randomize whether the even years are controls or treatment subjects.

Another advantage of gridded sampling is to detect localized variation. In the field, this would be "pockets" of unusual responses. Statistically, we may think of them as manifestations of spatial correlation. In the present situation, if there is some chance that a relatively narrow age range experiences unusual responses, then the gridded design is an excellent choice, because a purely randomized design can by chance contain large gaps in ages within one of the groups. (But a better design might be to stratify: use parity of age to form two analytical strata and then, independently within each stratum, randomize patients into control and treatment groups.)

Unfortunately, this defense falls apart once we come to terms with how ages are actually reported. US Census data show that (1) self-reported ages tend to be rounded to multiples of five (I have seen this in analyses of rural block group data) and (2) this tendency is associated with indicators of lower education or socioeconomic status. (It is also well known, although difficult to test, that the final digit in many self-reported ages is $9$, that people in certain fields of work, such as acting, tend to reduce their reported ages, and others will exaggerate their ages for various purposes.) Thus, at least to a slight degree in at least some areas of the US (and even more so elsewhere in the world), the parity of one's reported age is likely to be associated with factors important for the experiment. This renders the concern in the question less than hypothetical: it is real. At this point, the previous answers in this thread capably present the additional thoughts I would care to make, so I will stop and invite you to re-read them.

Answer 3

I agree the example you give is pretty innocuous but...

If the agents involved (either the person dealing out the intervention or the people getting the intervention) become aware of the assignment scheme they can take advantage of it. Such self selection should be fairly obvious why it is problematic in most experimental designs.

One example I am aware of in criminology goes like this; The experiment was meant to test the deterrent effect of a night in jail after a domestic dispute vs. just asking the perpetrator to leave for the night. Officers were given a booklet of sheets, and the color of the current sheet on top was meant to identify what treatment the perp. in the particular incident was supposed to receive.

What ended up happening was officers intentionally disobeyed the study design, and chose a sheet based on personal preferences for what should be done to the perp. It isn't out of the extreme to suspect similar fudging of years is at least possible in your example.

Answer 4

1. What you are proposing is NOT pseudo-randomization. Pseudo-randomization uses a seed to reproducibly generate a pseudo-random sequence based on the internal clock of a computer. The randomization assignment does NOT depend on patient level characteristics.

2. The point of randomization is to balance the distribution of predictive covariates, not just the means (and it's not guaranteed you would even have that). In other words, for any given treated patient the closest available matched control will always differ by one year of age. While it's true little is published about the characteristics of people born in even years versus odd years, you introduce a sensitivity that's otherwise moot when using traditional randomization.

3. If you use a deterministic criterion to randomize patients, how will you ensure an approximate 50:50 (or other) allocation of treatment vs. control? The situation gets exponentially worse if you try to stratify by, say, site.

4. What if you need to randomize to one of three different treatments? Or worse, what about an adaptive randomization where you begin randomizing to one of three treatment arms, and then based on safety or efficacy, you decide to drop an arm and randomize 1:1?