How does one define what measure to use for a Regression Discontinuity design? For example I am looking at the effectiveness of a reading learning program for 3rd graders.

*

*How do I define an acceptable cutoff measure?  Do I look at accepted reading assessment tests, such as DIBELS, which is used as a standard assessment measure in my state, to use as a pretest?  Or can I use any other test that seems to measure a specific form of reading achivement?


*How would I determine a cutoff score? Again, is there I standard cutoff which I would find by researching previous studies using a specific test assessment?  Or, If I already have test scores for students who have already gone thru the program, can I use a historic mean or median score, or other statistic to set as the cutoff number once I implement the treatment program?
 A: Regarding Q1. Yes, do not use ad-hoc measures. If the current measures does not adequately capture what we want them to represent than the definition of new measurement type is a separate task. Assuming this is not supposed to be methodologically novel work, using existing literature to justify our choices is the safe choice to avoid spurious findings as well as have solid theoretical foundations in our work.
Regarding Q2. What we want is the effect size to allow us the interpretation of the practical significance of observed difference as well as the $p$-values to assess the statistical significance. (Please note that saying that an effect size is large because of a very small $p$-value is totally wrong.) That said, yes, assuming that we use a specific measurement instrument it makes sense to anchor our cut-off in similar manner as previous studies. Do note though that this might have serious implications, see Identifying the Effect of Changing the Policy Threshold in Regression Discontinuity Models (2015) by Dong and Lewbel for an in-depth discussion. That said, if we want to lend further generalisability to our study it makes sense to try to different cut-off to specifically show how sensitive the findings are to cut-off choice.
