23
$\begingroup$

I came across a study in which patients, who were all over 50, were pseudo-randomized by birth year. If birth year were an even number, usual care, if an odd number, intervention.

It's easier to implement, it's harder to subvert (it's easy to check what treatment a patient should have received), it's easy to remember (the assignment went on for several years). But still, I don't like it, I feel like proper randomization would have been better. But I can't explain why.

Am I wrong for feeling that, or is there a good reason to prefer 'real' randomization?

$\endgroup$
  • 1
    $\begingroup$ Welcome to the site! Glad to see your posting here. $\endgroup$ – Andy W Mar 27 '13 at 17:13
  • $\begingroup$ By the answers above, I feel better to "randomize" by DAY of birth! Odd day to treatment, even day to control... Adalberto $\endgroup$ – AADF Apr 3 '13 at 18:03
  • 6
    $\begingroup$ @Adalberto That misses the main point, which is that any definite, non-randomized procedure of assigning subjects to groups cannot be assured of having the desirable properties that a randomized procedure has. Suppose you spend years of time on such a study only afterwards to have a reviewer point out an unexpected but strong confounder between treatment and parity of birth day? Because we cannot anticipate all such confounding, we sidestep the problem by means of random assignment. $\endgroup$ – whuber Apr 3 '13 at 18:08
28
$\begingroup$

You are right to be skeptical. In general, one should use 'real' randomization, because typically one doesn't have all knowledge about relevant factors (unobservables). If one of those unobservables is correlated with the age being odd or even, then it is also correlated with whether or not they received treatment. If this is the case, we cannot identify the treatment effect: effects we observe could be due to treatment, or due to the unobserved factor(s).

This is not a problem with real randomization, where we don't expect any dependence between treatment and unobservables (though, of course, for small samples it may be there).

To construct a story why this randomization procedure might be a problem, suppose the study only included subjects that were at age 17/18 when, say, the Vietnam war started. With 17 there was no chance to be drafted (correct me if I am wrong on that), while there was that chance at 18. Assuming the chance was nonnegligible and that war experience changes people, it implies that, years later, these two groups are different, even though they are just 1 year apart. So perhaps the treatment (drug) looks like it doesn't work, but because only the group with Vietnam veterans received it, this may actually be due to the fact that it doesn't work on people with PTSD (or other factors related to being a veteran). In other words, you need both groups (treatment and control) to be identical, except for the treatment, to identify the treatment effect. With assignment by age, this is not the case.

So unless you can rule out that there is no unobserved differences between the groups (but how do you do that if it isn't observed?), real randomization is preferable.

$\endgroup$
  • $\begingroup$ Thanks. Nice example. (I forgot to call it pseudo-randomization, I've edited that in the question ). $\endgroup$ – Jeremy Miles Mar 27 '13 at 17:34
  • 2
    $\begingroup$ (+1) As I was reading the question, Vietnam was the first example that immediately sprang to mind. It was amusing to see you had taken the very same tack. I suppose it's the most obvious choice given the stated ages of the subjects, though ages in the early-to-mid 60s are a bit closer. $\endgroup$ – cardinal Mar 27 '13 at 22:16
  • $\begingroup$ Apologies for off-topic ping: there is a suggestion on Meta to make [randomized-experiment] a synonym of [random-allocation] tag (stats.meta.stackexchange.com/a/4651). You have enough reputation in this tag in order to vote for this suggestion here: stats.stackexchange.com/tags/random-allocation/synonyms - it now needs 4 upvotes to go through. If you disagree with the proposal, consider commenting on Meta to explain why. I will delete this comment soon. Cheers. $\endgroup$ – amoeba says Reinstate Monica May 2 '17 at 12:03
18
$\begingroup$

It is a good exercise to uphold contrarian views from time to time, so let me begin by offering a few reasons in favor of this form of pseudo-randomization. They are, principally, that it is little different than any other form of systematic sampling, such as obtaining samples of environmental media at points of a grid in the field or sampling every other tree in an orchard, and therefore this sampling might enjoy comparable advantages.

The analogy here is perfect: age was "gridded" by year starting at an origin of zero and assignment to the groups alternated along this (one-dimensional) grid. Some advantages of this approach are to guarantee wide, even dispersion of the sample across the field or orchard (or ages, in this case), which helps even out influences related to location (or time). This can be especially useful when theory suggests that location is the predominant factor in variation of response. Moreover, except for really tiny samples, analyzing the data as if they were a simple random sample introduces relatively little error. Furthermore, some randomization is possible: in the field we can randomly choose the origin and orientation of the grid. In the present case, we can at least randomize whether the even years are controls or treatment subjects.

Another advantage of gridded sampling is to detect localized variation. In the field, this would be "pockets" of unusual responses. Statistically, we may think of them as manifestations of spatial correlation. In the present situation, if there is some chance that a relatively narrow age range experiences unusual responses, then the gridded design is an excellent choice, because a purely randomized design can by chance contain large gaps in ages within one of the groups. (But a better design might be to stratify: use parity of age to form two analytical strata and then, independently within each stratum, randomize patients into control and treatment groups.)

Unfortunately, this defense falls apart once we come to terms with how ages are actually reported. US Census data show that (1) self-reported ages tend to be rounded to multiples of five (I have seen this in analyses of rural block group data) and (2) this tendency is associated with indicators of lower education or socioeconomic status. (It is also well known, although difficult to test, that the final digit in many self-reported ages is $9$, that people in certain fields of work, such as acting, tend to reduce their reported ages, and others will exaggerate their ages for various purposes.) Thus, at least to a slight degree in at least some areas of the US (and even more so elsewhere in the world), the parity of one's reported age is likely to be associated with factors important for the experiment. This renders the concern in the question less than hypothetical: it is real. At this point, the previous answers in this thread capably present the additional thoughts I would care to make, so I will stop and invite you to re-read them.

$\endgroup$
  • $\begingroup$ (+1) Particularly, for the counterargument set up. $\endgroup$ – cardinal Mar 27 '13 at 22:18
13
$\begingroup$

I agree the example you give is pretty innocuous but...

If the agents involved (either the person dealing out the intervention or the people getting the intervention) become aware of the assignment scheme they can take advantage of it. Such self selection should be fairly obvious why it is problematic in most experimental designs.

One example I am aware of in criminology goes like this; The experiment was meant to test the deterrent effect of a night in jail after a domestic dispute vs. just asking the perpetrator to leave for the night. Officers were given a booklet of sheets, and the color of the current sheet on top was meant to identify what treatment the perp. in the particular incident was supposed to receive.

What ended up happening was officers intentionally disobeyed the study design, and chose a sheet based on personal preferences for what should be done to the perp. It isn't out of the extreme to suspect similar fudging of years is at least possible in your example.

$\endgroup$
  • $\begingroup$ Nice example, thanks, but part of the reasoning was that fudging was much harder - they couldn't argue that the sheet was (say) yellow, because I can go and check the date of birth and see if they were correctly assigned. $\endgroup$ – Jeremy Miles Mar 27 '13 at 17:32
  • 3
    $\begingroup$ I agree @JeremyMiles, it is just another reason though for double-blind randomized studies. It is merely an intended argument against pseudo-randomization - that it is easier to circumvent the intended treatment than actual randomization. (My example actually isn't an example of pseudo-randomization, but it illustrates the point succinctly.) $\endgroup$ – Andy W Mar 27 '13 at 17:39
  • $\begingroup$ Well, that depends on how the (true) randomization was done - the people involved in the study did it partly to avoid problems with subversion. If you use actual randomization, you need good record keeping to ensure that the person who determines the randomzation communicates with the person delivering the treatment, and the person delivering does the right thing. With your example, if they'd used house number (say), the officers might have had a harder time subverting, even though it wasn't random. $\endgroup$ – Jeremy Miles Mar 27 '13 at 17:43
  • 1
    $\begingroup$ Apologies for off-topic ping: there is a suggestion on Meta to make [randomized-experiment] a synonym of [random-allocation] tag (stats.meta.stackexchange.com/a/4651). You have enough reputation in this tag in order to vote for this suggestion here: stats.stackexchange.com/tags/random-allocation/synonyms - it now needs 4 upvotes to go through. If you disagree with the proposal, consider commenting on Meta to explain why. I will delete this comment soon. Cheers. $\endgroup$ – amoeba says Reinstate Monica May 2 '17 at 12:04
0
$\begingroup$

Complete randomisation based on the random distribution is not predictable, in your case it is known that whether a case would be assigned to Intervention or Control before eligibility confirmation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.