Clustering standard errors for respondents who move during treatment period

Question

I am conducting an impact evaluation, but the question applies generally to any situation where observations move between cluster-units during the course of treatment.

I am using robust standard errors clustered at the town level. Treatment was a program that lasted six months, assigned based on household- and town-level characteristics. Outcome data was collected at the end of the six-month program. During those six months, some respondents moved from one town to another.

Should SEs be clustered based on where people lived at the beginning of the program (at the time of treatment assignment) or where they lived when we collected data (six months later, after some respondents moved)? I'm torn. On the one hand, it could be where they lived at the time of treatment assignment, in keeping with an ITT analysis. On the other hand, any questions in the survey about town-level outcomes (prices, community engagement, crime) are measured in the town where people lived at data collection, not in the original village.

Answer 1

The most crucial thing we need to know is how treatment was assigned. Individually random or at the town level. Often this is not properly thought about and then if people actually do cluster their standard errors, they are doing it at the wrong level.

There is a very good paper by Cameron and Miller (2014) who summarize:

[If] either the regressors or the errors are likely to be uncorrelated within a potential group, then there is no need to cluster within that group [...] If a key regressor is randomly assigned within clusters [...] then the within-cluster correlation of the regressor is likely to be zero. Thus there is no need to cluster standard errors, even if the model’s errors are clustered.

Your question remains relevant if treatment was assigned at the village level. What you would have to do then depends a little on the context and timing really, but it seems to me that clustering alone can not solve the issue fully or unimpeachably. However, who can ever claim this in any way? And if your data is good enough, it could be rectified to neglect this and rely on clustering.

The aspects of your context you have to consider for deciding how to cluster are:

1. Intra-cluster correlation of treatment (and effective treatment, i.e. has someone who moved after the first month from a T to a C village to be considered treated? If yes how about someone moved after a week?). You will not get around taking some decisions there, of course not without cross-comparing.

2. Intra-cluster correlation of the outcome. Eventhough, in the context of survey based RCTs it's unlikely that you can not argue that outcomes do not correlate across individuals in the new and the old surrounding.

My suggestion is to (i) read the paper I linked above and (ii) run some simulations with fake (re-randomized) treatment to figure out which tests give you the right rejection rates when you know that there is no treatment effect. This should be done by each study before talking about any of their results. I think most people would be surprised how far off the tests are which they apply blindly/