1
$\begingroup$

I'm designing a randomized encouragement trial in which half of the sample will be randomly assigned to receive a special invitation to try a new intervention. This design will likely result in two-sided non-compliance with respect to random assignment:

  1. Everyone randomized to the encouragement arm will receive a special invitation to try the intervention, but only a subset of people in this group will take up this offer.

  2. People randomized to the control arm will NOT receive a special invitation to try the intervention, but some will learn about it through other channels and try it out on their own.

Encouragement designs account for this non-compliance by estimating the local average treatment effect (LATE). LATE is the effect of the intervention on 'compliers'—those who tried the intervention because they were randomly encouraged to do so but would not have tried if not encouraged.

I simulated a basic dataset that mimics the planned study, and have a question about interpreting the treatment effect.

# setup
  library(dplyr)
  library(arm)
  library(AER)
  library(ivpack)
  library(stargazer)
  n <- 134

# https://rpubs.com/wsundstrom/t_ivreg
# function to calculate corrected SEs for OLS regression 
  cse = function(reg) {
      rob = sqrt(diag(vcovHC(reg, type = "HC1")))
      return(rob)
  }
# corrected SEs for IV regressions... slight difference from S&W method
  ivse = function(reg) {
      rob = robust.se(reg)[,2]
      return(rob)
  }

# create dataframe
  dat <- data.frame(partID=seq(1, n, 1),
                    trt=c(rep(0, n/2), 
                          rep(1, n/2)))

# set proportion use
  useT <- .8  # treatment group (encouraged)
  useC <- .2  # control group (not encouraged)

# create use variable
  set.seed(493)
  dat$use <- c(rbinom(n/2, 1, useC),
               rbinom(n/2, 1, useT))

# create covariate
# http://stackoverflow.com/questions/42147053/simulate-continuous-variable-that-is-correlated-to-existing-binary-variable

  x1    <- dat$use               # fixed given data
  rho   <- 0.1                   # desired correlation = cos(angle)
  theta <- acos(rho)             # corresponding angle
  x2    <- rnorm(n, 2, 0.5)      # new random data
  X     <- cbind(x1, x2)         # matrix
  Xctr  <- scale(X, center=TRUE, 
                 scale=FALSE)    # centered columns (mean 0)

  Id   <- diag(n)                           # identity matrix
  Q    <- qr.Q(qr(Xctr[ , 1, drop=FALSE]))  # QR-decomposition, just matrix Q
  P    <- tcrossprod(Q)          # = Q Q'   # projection onto space defined by x1
  x2o  <- (Id-P) %*% Xctr[ , 2]                 # x2ctr made orthogonal to x1ctr
  Xc2  <- cbind(Xctr[ , 1], x2o)                # bind to matrix
  Y    <- Xc2 %*% diag(1/sqrt(colSums(Xc2^2)))  # scale columns to length 1

  x <- Y[ , 2] + (1 / tan(theta)) * Y[ , 1]     # final new vector

  dat$age <- (1 + x) * 25 
  cor(dat$use, dat$age)
  dat$age <- round(dat$age, 0)

# outcome
  outT <- .35
  outC <- .05
  dat$y <- c(rbinom(n/2, 1, outC),
             rbinom(n/2, 1, outT))

# IV Regression
  ivR = ivreg(y ~ use + rescale(age) | rescale(age) + trt , data = dat)
  stargazer(ivR, 
            se=list(ivse(ivR)),
            title="IV Regression", 
            type="text", 
            df=FALSE, 
            digits=5,
            ci=TRUE) 

The coefficient on use is 0.43. This is the effect on the 'compliers'. The outcome y is binary. I want to be able to say that the intervention increased y by 43% points from a to b.

How do I get a? The proportion among the control group is 0.03, but this includes compliers and never-takers.

$\endgroup$
  • $\begingroup$ (1) I tried to follow your case but I get stuck on your linear regression. Without getting the details of the iv stuff, it seems that you are using an incorrect regression model. At least this is what it appears to me since your model predicts negative values. A suitable model should restrict the possible model values between 0 and 1. For instance logistic regression. $\endgroup$ – Martijn Weterings Aug 16 '17 at 16:38
  • $\begingroup$ (2a) What kind of relation do 'use' and 'y' have? I currently imagine that 'use' relates to 'following the intervention', and 'y' relates to some measure of health. (2b) Why does 'y' not correlate with 'use'? What is the treatment expected to be doing to increase 'y', if not via 'use'? (2c) I don't get the question "How do I get a? The proportion among the control group is 0.03, but this includes compliers and never-takers". What are compliers? What are never-takers? How do these relate to 'use' and 'treatment'? Why does it matter that you combine them (what is the structure)? $\endgroup$ – Martijn Weterings Aug 16 '17 at 16:48
  • $\begingroup$ @MartijnWeterings IV is the causal identification strategy here. The book "Mostly Harmless Econometrics" does a nice job explaining. $\endgroup$ – Eric Green Aug 16 '17 at 20:59
  • $\begingroup$ a reference to an article might be a more approachable introduction to IV than a book. On the other side, my comment about IV was just a side note. Could you clarify the other issues such that your design, and problem, becomes more clear..... $\endgroup$ – Martijn Weterings Aug 16 '17 at 23:27
  • 1
    $\begingroup$ @MartijnWeterings You could check this article for an into to IV and an explanation of why the 2SLS estimator is consistent for the LATE given binary data. ncbi.nlm.nih.gov/pmc/articles/PMC4201653 $\endgroup$ – eric_kernfeld Aug 16 '17 at 23:49
1
$\begingroup$

I'm gonna need some help here, but here's one approach to an answer.

The parameter you want is not necessarily identifiable from your dataset.

Consider the following dataset, where out is the outcome, trt is the encouragement, use is the actual intervention, num says how many people fell into that pattern of encouragement/intervention, and comp is how we might categorize the patients in that group using the compliance classes outlined in this review.

id  out use trt comp
1   1   0   0   never_taker OR complier
2   1   1   1   complier
3   0   0   0   complier
4   1   1   0   always_taker

You'll observe the same data no matter whether participant 1 is a never-taker or a complier. But, participant 1's status affects the parameter you seek. If they are a complier, that pushes the untreated complier outcome probability from 0 to 1/2.

$\endgroup$
  • $\begingroup$ Thanks. I think that is the core problem. I thought I might be missing something, but you illustrate the issue pretty clearly. $\endgroup$ – Eric Green Aug 16 '17 at 23:52
  • $\begingroup$ I am afraid I've missed something, though, because this answer seems to suggest that the LATE is also not identifiable. There is extra probabilistic information that I am not accounting for, especially that trt is randomly assigned. That's kind of the whole point... $\endgroup$ – eric_kernfeld Aug 16 '17 at 23:57
1
$\begingroup$

You've got

E(y $\vert$ encouraged) = $\tfrac{18}{67}$

E(y $\vert$ !encouraged) = $\tfrac{2}{67}$

E(use $\vert$ encouraged) = $\tfrac{53}{67}$

E(use $\vert$ !encouraged) = $\tfrac{16}{67}$

Equation (2) from the source from other Eric in intuitive words:

  • the total effect is the 53 that 'use' resulting in 18 'y'.
  • substract from that the 16 that normally (without encouragement) 'use' and result in 2 'y'
  • Then the causal effect of 'use', the 37 extra 'use' that stems from the encouragement, is 16 extra 'y', and "therefore" the causal effect is

$\frac{E(y\vert e)-E(y\vert !e)}{E(use\vert e)-E(use\vert !e)} = \frac{16}{37} = 0.43...$

This analysis is without the age parameter since it does not correlate anyway (then we do not need the regression to obtain the estimated values). For the model with the covariates (if you have a serious effect of age in the real data) you will need some regression model to obtain those estimated values. I suggest that you use a reasonable model, restricted to values between 0 and 1, such that the predicted values make at least some sense.


If you wish something like: "I want to be able to say that..."

... - the intervention - increased - y - by 43% points - from a to b.

What the double regression does is more something like (using the same structure):

... - the change in intervention use that is due to the encouragement (from 16 to 53) - increased - y - with a coefficient of 0.43 - from 2 to 18"

The encouragement increases the use and the use increases y. The 0.43 is not related to the relative increase of y (from a to b) but to to the coefficient that relates the change of 'use' with the change of 'y'.

$\endgroup$
  • 1
    $\begingroup$ This is a valid point regarding how the effect should be described, but I believe that Eric Green was asking about the baseline rate among compliers only. The number you give (2) may include some never-takers as well. $\endgroup$ – eric_kernfeld Aug 17 '17 at 2:22
  • $\begingroup$ Introduction of a 'tendency' to be influenced by the treatment (dividing the group not-takers into compliers, and never-takers while giving these groups different baseline 'y') introduces an extra confounding variable and makes the instrumental variable invalid. Namely, if you introduce that new confounder then this rises the question: "does the IV change the use and indirectly the y, or does it merely select a group with different baseline 'y'?" $\endgroup$ – Martijn Weterings Aug 17 '17 at 3:19
  • $\begingroup$ @MartijnWeterings: "The intervention increases the use and the use increases y." In a randomized encouragement trial, I believe this would be "encouragement increases use/uptake of the intervention, and use increases y". $\endgroup$ – Eric Green Aug 17 '17 at 8:58
  • $\begingroup$ I have changed it. It gets confusing when you are speaking about the 'treatment that encourages the use of the intervention'. Those terms treatment and intervention rhyme. $\endgroup$ – Martijn Weterings Aug 17 '17 at 9:29
  • $\begingroup$ I think this is still inaccurate: "intervention increases the use and the use increases y". The encouragement is not the intervention. People are randomly assigned to receive an encouragement to try an intervention. This encouragement increases use/uptake of the intervention, and in turn, use of the intervention increases some outcome. $\endgroup$ – Eric Green Aug 17 '17 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.