1
$\begingroup$

I would like to know if a variable X3 contributes more to explain outcome Y1 or outcome Y2. In model formula terms, I want to know whether the model improvement between

  1. Y1 ~ X1 + X2 + X3 vs Y1 ~ X1 + X2 or
  2. Y2 ~ X1 + X2 + X3 vs Y2 ~ X1 + X2

is greater. Is there any way to compare a variable X3 across outcomes?

$\endgroup$
3
  • $\begingroup$ A couple of questions: 1) Are Y1 and Y2 reasonably comparable? 2) Are both prediction equations (i.e., the ones predicting Y1 and Y2) using the same predictive model? $\endgroup$
    – jluchman
    Jan 19 at 15:49
  • $\begingroup$ Y1 and Y2 are different variables, for example height and income. The equations are relatively similar, i.e. the exact same predictor variables and the same model but different model parameters (and likely a different set of predictor variables that were selected by the model). $\endgroup$
    – Brigitte
    Jan 20 at 21:23
  • $\begingroup$ A few more questions: 3) are the models here (generalized) linear models? 4) Is it important to the analysis question that the two models compared have different numbers of predictors/is it reasonable to force both to have the same set? $\endgroup$
    – jluchman
    Jan 21 at 17:40

1 Answer 1

2
$\begingroup$

Comparing predictive utility of an independent variable across predictive equations is tricky and is a topic of discussion in this article. The article here suggests one way to disentangle the utility of a predictor on two different models is to use a dominance analysis (for example using this Stata implementation).

Consider for instance a model like this:

mpg ~ trunk + price + foreign

and

length ~ trunk + weight

Where our interest is in comparing 'trunk' across outcomes.

This model could be fit using sureg in Stata or a systemfit model in R.

The general idea is to fit a model where you can get a single fit statistic reflecting the entirety of the model in a single value. In the below, the fit statistic is a McFadden pseudo-R2 ($1 - \frac{loglikelihood_{full}}{loglikelihood_{null}}$).

The implementation fitted below uses the domme module in Stata as it is easier to fit to models like these.

. sysuse auto
(1978 automobile data)

. domme (mpg = trunk price foreign) (length = trunk weight), reg(sureg (mpg = trunk price foreign) (length = trunk weight)) fitstat(e(), mcf) noconditional nocomplete

Total of 31 models/regressions

Progress in running all regression subsets
0%------50%------100%
....................
General dominance statistics: sureg
Number of obs             =                      74
Overall Fit Statistic     =                  0.1496

            |      Dominance      Standardized      Ranking
            |      Stat.          Domin. Stat.
------------+------------------------------------------------------------------------
mpg         |
 trunk      |         0.0111      0.0744            4 
 price      |         0.0112      0.0746            3 
 foreign    |         0.0034      0.0230            5 
length      |
 trunk      |         0.0238      0.1588            2 
 weight     |         0.1001      0.6692            1 
-------------------------------------------------------------------------------------

The idea here is that it appears that 'trunk' predicts 'length' better than it predicts 'mpg' when considering both in the context of the same model (i.e., fitted to the same loglikelihood) which makes the effects across equations more comparable (see the articles above for additional caveats about multi-equation models).

Also possible do implement a model like this using R's {domir} package but it is not yet optimized to do so easily (i.e., takes a lot more programming know-how).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.