Is the treatment affecting both response variables by the same multiplying factor?

Question

Experimental setup

We have N individuals that underwent treatment A and N individuals that underwent treatment B. In each individual, we measure after treatment two variables X and Y. X and Y are both unitless.

There is no meaningful before treatment measurement and the data from the two treatment groups cannot be paired in any logical way.

Question

The treatment has an effect on both X and Y. I would like to compare these effects. I would like to test the null hypothesis that the treatment affect both variables X and Y by the same multiplying factor.

How should I go about performing such test?

Example

Consider this made-up data set

set.seed(12)
N = 1e3

d = data.frame(
    Treatment = rep(LETTERS[1:2], each=2*N),
    variable  = rep(rep(LETTERS[24:25], each=N),2),
    value     = c
    (
        runif(N,0,100),         # Treatment A, variable X
        runif(N,50,120),        # Treatment A, variable Y
        runif(N,0,100) * 1.2,   # Treatment B, variable X
        runif(N,50,120) * 1.5   # Treatment B, variable Y
    )
)


require(ggplot2)

ggplot(d, aes(x=variable, y=value, color=Treatment, group=Treatment)) + stat_summary(fun.y=mean, geom="point") + stat_summary(
    fun.y=mean,
    fun.ymin = function(x) mean(x) - sd(x) / sqrt(length(x)), 
    fun.ymax = function(x) mean(x) + sd(x) / sqrt(length(x)), 
    geom="errorbar", width=0.3) + theme_classic() 

This data set was created so that the treatment affected the variable X by a multiplying factor of 1.2 and affected the variable Y by a multiplying factor of 1.5. The null should therefore be rejected as 1.2 ≠ 1.5.

My thoughts

I thought I could run some linear model of the kind

Value ~ Treatment + variable + Treatment:variable

with a type I sum of squares and investigate the significance of the interaction term. I might have to transform the data beforehand to ensure both variables have the same mean and variance but I am not sure. This regression does not quite seem to be what I am after though and conceptually, it hurts my soul to organize my data under such format for a test (although it is the format I chose for my made-up data)!

Answer 1

Here's how I might handle this using Seemingly Unrelated Regression (SUR). The regressions are related because the errors associated with the dependent variables may be correlated for the two outcomes. This gives you the ability to conduct hypothesis tests across equations.

I did it two ways. The first involves fitting the SUR model on a original scale, calculating the elasticities (percentage change in the outcome when x goes from zero to one), and then doing the hypothesis test that the elasticities are the same.

The second is also SUR, but with logged outcomes so you can approximate elasticities that way. That would probably be easier to do.

Fitting the regressions this way allows you test the coefficients (or functions of the coefficients) in the two equations are the same.

Here we will model length and miles per gallon of 74 cars with a binary variable foreign. Both methods yield foreign origin elasticities of -15% and +22%, so you would reject the null that the multiplicative effects are the same.

. sysuse auto, clear
(1978 Automobile Data)

. replace length = length/10
variable length was int now float
(74 real changes made)

. sum length mpg foreign 

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
      length |         74    18.79324    2.226634       14.2       23.3
         mpg |         74     21.2973    5.785503         12         41
     foreign |         74    .2972973    .4601885          0          1

. 
. /* (1) Using SUR */
. sureg (length i.foreign) (mpg i.foreign), small dfk

Seemingly unrelated regression
--------------------------------------------------------------------------
Equation             Obs   Parms        RMSE    "R-sq"     F-Stat        P
--------------------------------------------------------------------------
length                74       1    1.841856    0.3251      34.69   0.0000
mpg                   74       1     5.35582    0.1548      13.18   0.0004
--------------------------------------------------------------------------

------------------------------------------------------------------------------
             |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
length       |
     foreign |
    Foreign  |  -2.758916   .4684448    -5.89   0.000    -3.684832      -1.833
       _cons |   19.61346   .2554194    76.79   0.000     19.10861    20.11832
-------------+----------------------------------------------------------------
mpg          |
     foreign |
    Foreign  |   4.945804   1.362162     3.63   0.000     2.253389     7.63822
       _cons |   19.82692   .7427186    26.70   0.000     18.35888    21.29496
------------------------------------------------------------------------------

. 
. /* calculate the elasticities first */
. margins, eydx(foreign) post

Conditional marginal effects                    Number of obs     =         74

ey/dx w.r.t. : 1.foreign
1._predict   : Linear prediction, predict(xb equation(length))
2._predict   : Linear prediction, predict(xb equation(mpg))

------------------------------------------------------------------------------
             |            Delta-method
             |      ey/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.foreign    |
    _predict |
          1  |  -.1515958    .026691    -5.68   0.000    -.2043525   -.0988391
          2  |   .2227026    .059396     3.75   0.000      .105302    .3401032
------------------------------------------------------------------------------
Note: ey/dx for factor levels is the discrete change from the base level.

. 
. /* conduct F-test that the efefcts are equal */
. test _b[1.foreign:1._predict]=_b[1.foreign:2._predict]

 ( 1)  [1.foreign]1bn._predict - [1.foreign]2._predict = 0

       F(  1,   144) =   21.22
            Prob > F =    0.0000

. 
. /* (2) Use natural logs of outcomes to get elasticities */
. gen ln_length = ln(length)

. gen ln_mpg = ln(mpg)

. sureg (ln_length i.foreign) (ln_mpg i.foreign), small dfk

Seemingly unrelated regression
--------------------------------------------------------------------------
Equation             Obs   Parms        RMSE    "R-sq"     F-Stat        P
--------------------------------------------------------------------------
ln_length             74       1    .0992019    0.3275      35.06   0.0000
ln_mpg                74       1    .2420209    0.1465      12.36   0.0006
--------------------------------------------------------------------------

------------------------------------------------------------------------------
             |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
ln_length    |
     foreign |
    Foreign  |  -.1494025   .0252303    -5.92   0.000    -.1992721   -.0995329
       _cons |   2.970867   .0137568   215.96   0.000     2.943676    2.998058
-------------+----------------------------------------------------------------
ln_mpg       |
     foreign |
    Foreign  |   .2163907   .0615539     3.52   0.001     .0947248    .3380567
       _cons |   2.960203   .0335623    88.20   0.000     2.893865    3.026542
------------------------------------------------------------------------------

. test [ln_length]1.foreign=[ln_mpg]1.foreign

 ( 1)  [ln_length]1.foreign - [ln_mpg]1.foreign = 0

       F(  1,   144) =   19.42
            Prob > F =    0.0000