2
$\begingroup$

I am not a statistician or econometrician, so please bear with me. For a term paper, I am estimating local treatment effects using a regression discontinuity design, and I want to test whether the effect of crossing the threshold is different for men and women. For the full sample, I keep observations only within the desired bandwidth and then run the following Stata code:

reg y rv above rv_above, r

where y = outcome; rv = running variable; above = indicator if the running variable is above the threshold; and rv_above is an interaction term between rv and above, allowing the slope of the control function to change above and below the threshold.

I believe there should be two (equivalent?) ways to obtain the effects for the subsample of men and the subsample of women:

1) I can create a dummy variable, for example female = 1 for women and = 0 for men, then run the code reg y rv above rv_above if female == 1, r to get the treatment effect for women and the code reg y rv above rv_above if female == 0, r to get the treatment effect for men. Once I do this, how do I test that the coefficient on "above" is different between the two samples? Is it as simple as showing that the 95% confidence intervals do not overlap, and then you can conclude that they are statistically different at the 5% level? Or is there another formal statistical test of significance that I should be using if I choose to do this split sample analysis?

2) Instead of splitting the sample, I can include a dummy variable for gender in the regression and interact it with all of the variables previously included:

reg y rv above rv_above gender gender_rv gender_above gender_rv_above, r

Is this also a correct way to do things? And in this case, how do I test that the effect for men and women is statistically different?

Thanks for any guidance you can share.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Here's how you would do both in Stata. I don't think theory suggests that one would be better than the other, but you should do both to see if it matters in your setting. I think one potential situation is if you have very different numbers of men and women near the cutoff, but are not adjusting the bandwidth accordingly.

Below is a voting example where incumbent variable (i) plays the role of gender. I am using the dataset that comes with the rd command. Here the treatment is having a Democratic representative in the US Congress, and the assignment variable $Z$ is the vote share garnered by the Democratic candidate. At $Z=50\%$, the probability of $treatment=1$ jumps from zero to one because democracy. Suppose we are interested in the effect a Democratic representative has on the log of federal spending within that Congressional district. Note that you can create interactions on the fly using factor variable notation.

The results here are not sensitive to split versus single model choice. First we will fit two separate RD model and make sure the regression results match them:

. capture ssc install rd

. use votex, clear
(102nd Congress)

. 
. /* RD/local linear regression model with subsamples */
. rd lne d if i==0, mbw(100) bw(0.2) ker(rec)
Two variables specified; treatment is 
assumed to jump from zero to one at Z=0. 

 Assignment variable Z is d
 Treatment variable X_T unspecified
 Outcome variable y is lne

(306 missing values generated)
(306 missing values generated)
(306 missing values generated)
Estimating for bandwidth .2
------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       lwald |   -.109116   .2329603    -0.47   0.640    -.5657098    .3474778
------------------------------------------------------------------------------

. rd lne d if i==1, mbw(100) bw(0.2) ker(rec)
Two variables specified; treatment is 
assumed to jump from zero to one at Z=0. 

 Assignment variable Z is d
 Treatment variable X_T unspecified
 Outcome variable y is lne

(43 missing values generated)
(43 missing values generated)
(43 missing values generated)
Estimating for bandwidth .2
------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       lwald |  -.1357077    .129839    -1.05   0.296    -.3901874     .118772
------------------------------------------------------------------------------

These give a roughly 13.5% reduction in spending for incumbent Democrats and 11% for non-incumbents. Priors updated!

Here we fit the models separately and combine them with suest, which is a post-estimation command that allows you to test cross-equation hypotheses: 

. /* OLS version with subsamples */
. reg lne c.d##i.win if d > -.2 & d < .2 & i==0



      Source |       SS           df       MS      Number of obs   =        40
-------------+----------------------------------   F(3, 36)        =      0.35
       Model |  .247093725         3  .082364575   Prob > F        =    0.7875
    Residual |  8.40992166        36  .233608935   R-squared       =    0.0285
-------------+----------------------------------   Adj R-squared   =   -0.0524
       Total |  8.65701539        39  .221974754   Root MSE        =    .48333

------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           d |   .8024805   1.924507     0.42   0.679    -3.100601    4.705562
       1.win |   -.109116    .233947    -0.47   0.644    -.5835826    .3653505
             |
     win#c.d |
          1  |   1.013878   2.835951     0.36   0.723    -4.737698    6.765454
             |
       _cons |   21.42176   .1649851   129.84   0.000     21.08716    21.75637
------------------------------------------------------------------------------

. estimates store noninc

. reg lne c.d##i.win if d > -.2 & d < .2 & i==1

      Source |       SS           df       MS      Number of obs   =       227
-------------+----------------------------------   F(3, 223)       =      0.41
       Model |  .260048768         3  .086682923   Prob > F        =    0.7449
    Residual |  46.9787176       223  .210666895   R-squared       =    0.0055
-------------+----------------------------------   Adj R-squared   =   -0.0079
       Total |  47.2387663       226   .20902109   Root MSE        =    .45898

------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           d |   .9206187   .9301593     0.99   0.323    -.9124081    2.753646
       1.win |  -.1357077   .1541286    -0.88   0.380    -.4394425    .1680272
             |
     win#c.d |
          1  |  -.9084667   1.229594    -0.74   0.461    -3.331578    1.514645
             |
       _cons |   21.45386   .1145629   187.27   0.000      21.2281    21.67962
------------------------------------------------------------------------------

. estimates store inc

. suest noninc inc, vce(robust)

Simultaneous results for noninc, inc

                                                Number of obs     =        267

------------------------------------------------------------------------------
             |               Robust
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
noninc_mean  |
           d |   .8024805   1.330993     0.60   0.547    -1.806219    3.411179
       1.win |   -.109116   .2214206    -0.49   0.622    -.5430924    .3248603
             |
     win#c.d |
          1  |   1.013878   2.131018     0.48   0.634    -3.162841    5.190596
             |
       _cons |   21.42176   .1299416   164.86   0.000     21.16708    21.67645
-------------+----------------------------------------------------------------
noninc_lnvar |
       _cons |  -1.454107    .277406    -5.24   0.000    -1.997812   -.9104011
-------------+----------------------------------------------------------------
inc_mean     |
           d |   .9206187   .7730626     1.19   0.234    -.5945561    2.435793
       1.win |  -.1357077   .1289316    -1.05   0.293     -.388409    .1169936
             |
     win#c.d |
          1  |  -.9084667   1.083725    -0.84   0.402    -3.032528    1.215595
             |
       _cons |   21.45386   .0950153   225.79   0.000     21.26763    21.64009
-------------+----------------------------------------------------------------
inc_lnvar    |
       _cons |  -1.557477   .1351502   -11.52   0.000    -1.822367   -1.292588
------------------------------------------------------------------------------

. test [noninc_mean]1.win = [inc_mean]1.win

 ( 1)  [noninc_mean]1.win - [inc_mean]1.win = 0

           chi2(  1) =    0.01
         Prob > chi2 =    0.9173

This says that the data is consistent with the effects being the same, so no heterogeneity.

Now we fit the interactive version:

. /* OLS version with interactions */
. reg lne c.d##i.win##i.i if d > -.2 & d < .2, vce(robust)

Linear regression                               Number of obs     =        267
                                                F(7, 259)         =       0.69
                                                Prob > F          =     0.6829
                                                R-squared         =     0.0120
                                                Root MSE          =     .46245

------------------------------------------------------------------------------
             |               Robust
         lne |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           d |   .8024805    1.34886     0.59   0.552    -1.853648    3.458609
       1.win |   -.109116   .2243928    -0.49   0.627    -.5509826    .3327505
             |
     win#c.d |
          1  |   1.013878   2.159624     0.47   0.639    -3.238779    5.266534
             |
         1.i |   .0320956   .1631351     0.20   0.844    -.2891444    .3533356
             |
       i#c.d |
          1  |   .1181382   1.559872     0.08   0.940    -2.953508    3.189784
             |
       win#i |
        1 1  |  -.0265916   .2596628    -0.10   0.919    -.5379107    .4847274
             |
   win#i#c.d |
        1 1  |  -1.922344   2.422844    -0.79   0.428    -6.693326    2.848637
             |
       _cons |   21.42176   .1316858   162.67   0.000     21.16245    21.68108
------------------------------------------------------------------------------

. nlcom noninc:_b[1.win]

      noninc:  _b[1.win]

------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      noninc |   -.109116   .2243928    -0.49   0.627    -.5489178    .3306858
------------------------------------------------------------------------------

. nlcom inc:_b[1.win] + _b[1.win#1.i]

         inc:  _b[1.win] + _b[1.win#1.i]

------------------------------------------------------------------------------
         lne |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         inc |  -.1357077   .1306623    -1.04   0.299    -.3918011    .1203857
------------------------------------------------------------------------------

. test (_b[1.win]) = (_b[1.win] + _b[1.win#1.i])

 ( 1)  - 1.win#1.i = 0

       F(  1,   259) =    0.01
            Prob > F =    0.9185

. test 1.win#1.i

 ( 1)  1.win#1.i = 0

       F(  1,   259) =    0.01
            Prob > F =    0.9185

Again, we cannot reject the null that the effects are same.

To sum up, both approaches suggest that the effects on spending are very similar for incumbents and non-incumbents.

Stata Code:

cls
capture ssc install rd
use votex, clear

/* RD/local linear regression model with subsamples */
rd lne d if i==0, mbw(100) bw(0.2) ker(rec)
rd lne d if i==1, mbw(100) bw(0.2) ker(rec)

/* OLS version with subsamples */
reg lne c.d##i.win if d > -.2 & d < .2 & i==0
estimates store noninc
reg lne c.d##i.win if d > -.2 & d < .2 & i==1
estimates store inc
suest noninc inc, vce(robust)
test [noninc_mean]1.win = [inc_mean]1.win

/* OLS version with interactions */
reg lne c.d##i.win##i.i if d > -.2 & d < .2, vce(robust)
nlcom noninc:_b[1.win]
nlcom inc:_b[1.win] + _b[1.win#1.i]
test (_b[1.win]) = (_b[1.win] + _b[1.win#1.i])
test 1.win#1.i
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks Dimitriy for working through this helpful example! The code seems to make sense to me. Will try adapting to my own setting tomorrow and will get back in case anything is unclear. $\endgroup$
    – Jackson
    Commented Jun 4, 2018 at 18:57
  • $\begingroup$ @Jackson Glad it makes sense. I've updated the answer with some more commentary on the results. $\endgroup$
    – dimitriy
    Commented Jun 5, 2018 at 0:37
  • $\begingroup$ @Jackson Did this help? $\endgroup$
    – dimitriy
    Commented Jun 8, 2018 at 22:58
  • $\begingroup$ @Jackson Then you can select it as the answer by clicking on the ✔. $\endgroup$
    – dimitriy
    Commented Jun 10, 2018 at 19:30
  • $\begingroup$ Whoops, thought I had already done so. Now it's taken care of! $\endgroup$
    – Jackson
    Commented Jun 12, 2018 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.