3
$\begingroup$

This is a complicated question, and I'd very much appreciate any help I can get. I've been running on it nonstop for two or three days, and am running out of time. To be clear: This isn't so much a syntax or code question, its more of a concept/theory question that takes the form of a code difference. Or I might be asking you to diagram a kafka-esque monster for me.

Please let me know if I left anything out.

Question 1: What is the difference between predicted probabilities and Marginal Effects? More specifically: Is there a way I can judge when the Stata margins command is moving from one to the other?

Question 2: What is the conceptual/statistical difference between the atMeans and asObserved options in Stata14's Margins commands?

Question 3 (Bonus): The nearest method I've developed, and am using due to a time crunch, is to set dichotomous (Binary) variables to their modal values, and then set the rest to asObserved. Shoot holes in this method.

Just for background, I am running a statistical analysis using a binary logistic regression on a pair of datasets in Stata14. My dependent variable is Homeownership, and the independent variable is whether or not a respondent has student loans, both of which are binary variables. I'm using 8 other control variables. In one dataset (NFCS2012) all 8 are categorical variables. The other dataset has 2 continuous variables.

Q1: One of my professors suggested that, in addition to odds-ratios, it would be a good idea to use predicted probabilities. The command he suggested was prvalue, which is part of Spost9. Prvalue does not allow for factor variables (The i. notation in stata). However, the authors of prvalue (Long and Freese) have created the mtable command have released Spost13, which extends Stata's stock margins command. I have seen some commentary online about how you can get predicted probabilities from the margins (And thus Mtable) using the at(spec) option. The help files do not mention this.

Q2 The problem comes when attempting to hold the rest of my variables constant at a value. Specifically, whether I should hold them at their means using atMeans or at their observed values, using asObserved. I'm struggling with the difference between the two. I can see the difference in the values rendered, which is major, but I can't quite get at the theory of what as observed is doing, and whether that still qualifies as holding variables at a constant value.

When I use asObserved, I get values that fall within an expected range .630ish for Yes, and .57 for no. This makes me suspicious, because they resemble the initial cross tabs I did of my DV and IV. Put plainly, its spooky.

The code for this would be: mtable, at(RecodedG21 = (0, 1) A3 = 2 A4A_new_w = 1) asobserved statistics(all)

When I used atMeans, the predicted probabilities jump outside the range of what I would expect, at .676 for yes and .596 for no. I can write about this, however I'm worried that I'm not getting the most accurate description of my model from this option.

The example code for this would be: mtable, at(RecodedG21 = (0, 1) A3 = 2 A4A_new_w = 1) atmeans statistics(all)

I initially attempted to set all of my categorical variables to their modal values (Which in hind set, was silly and wasted time), and realized that it would skew the probabilities into the .8 range, and wasn't theoretically sound, as I couldn't justify picking one ideal type amongst 8*8*6*6*6*3*2*2*2 levels across 2 possible outcomes for my variables.

I've been reading papers however where people set their categorical variables to means. Am I missing something that Stata is doing? Or am I misconceptualizing this?

Compounding this is that when I go online I see people discuss predicted probabilities and marginal effects interchangeably, so I'm worried that I'm straying off course and will be explaining something to my thesis committee that is totally different than from what I'm actually doing. They're all at a conference, and I have a draft due, so 'm askin yall.

Q3(Bonus): I'm under a heavy time crunch, so I'm running with a method of using Mtable, setting binary variables to their modal values, and then using atMeans for the rest, simply because right now I have a better grasp and can write about that. If I were to use asObserved, what potential criticisms would you have of that?

$\endgroup$
0

2 Answers 2

8
$\begingroup$

In a logit model,

$$Pr[y = 1 \vert x, d] = p = \frac{\exp (\alpha + \beta \cdot x + \gamma \cdot d)}{1+\exp (\alpha + \beta \cdot x + \gamma \cdot d)}. $$

After some tedious calculus and simplification, the partial derivative of that with respect to a continuous variable $x$ becomes:

$$ \frac{\partial Pr[y=1 \vert x]}{\partial x} = \beta \cdot p \cdot (1-p). $$

This tells you the change in probability from a 1 unit increase in $x$.

Marginal effects of categorical/factor variables like $d$ (those prefixed with i.) are calculated by using finite differences rather than derivatives. That is, Stata calculates $$Pr[y = 1 \vert x , d=1] - Pr[y = 1 \vert x , d=0].$$ Note that $p$ and its derivative/finite difference are both function of $x$ and $d$, so you can evaluate either of them at various possible values. You can use the mean of $x$, $\bar x$. That is the atmeans: marginal effect at the mean. This corresponds to the predicted probability and its derivative for the average person. You can also use each observation's own value of $x$ and then take the average over the estimation sample. That's the asobserved, the default, also called the average marginal effect. You can also use the median or any other representative value. You can even do combinations of all these. There is no consensus which one is better and frequently the choice does not matter very much. You just have to be explicit about which one you are using.

However, there are some values that arguably don't make sense. For example, no one is half female and half male. For categorical or binary variables, it often makes sense to to use the base or modal value instead of the mean if you go the MEM/MER route.

Here's some code that hopefully makes the differences clear:

. sysuse auto, clear
(1978 Automobile Data)

. gen price2 = price/10000

. logit foreign price2

Iteration 0:   log likelihood =  -45.03321  
Iteration 1:   log likelihood = -44.947363  
Iteration 2:   log likelihood =  -44.94724  
Iteration 3:   log likelihood =  -44.94724  

Logistic regression                             Number of obs     =         74
                                                LR chi2(1)        =       0.17
                                                Prob > chi2       =     0.6784
Log likelihood =  -44.94724                     Pseudo R2         =     0.0019

------------------------------------------------------------------------------
     foreign |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      price2 |   .3528042   .8436183     0.42   0.676    -1.300657    2.006266
       _cons |  -1.079792   .5878344    -1.84   0.066    -2.231927    .0723418
------------------------------------------------------------------------------

Let's calculate average predictions at own values of price2 in 4 ways:

. /* (1) compare predicted probabilities versus margins */
. predict phat_by_stata, pr

. gen double phat_by_hand=exp(-1.079792+ .3528042*price2)/(1+exp(-1.079792+.3528042*price2)) 

. sum phat*

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
phat_by_st~a |         74    .2972973    .0224245   .2761411   .3731745
phat_by_hand |         74    .2972974    .0224245   .2761412   .3731746

. margins

Predictive margins                              Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   .2972973     .05307     5.60   0.000      .193282    .4013126
------------------------------------------------------------------------------

. margins, asobserved

Predictive margins                              Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   .2972973     .05307     5.60   0.000      .193282    .4013126
------------------------------------------------------------------------------

As you can see, they all agree. Now let's calculate the additive marginal effects, which you can get by adding , dydx(x) to margins:

. /* (2) marginal effects: the change in the probability from 10K increase in price */  
. margins, dydx(price2)

Average marginal effects                        Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()
dy/dx w.r.t. : price2

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      price2 |   .0735299   .1751266     0.42   0.675    -.2697119    .4167716
------------------------------------------------------------------------------

. sum price2

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
      price2 |         74    .6165257    .2949496      .3291     1.5906

. 
. di "phat with price2 of .6165257: " exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.3528042*.6165257))
phat with price2 of .6165257: .29686339

. margins, at(price2==.6165257)

Adjusted predictions                            Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()
at           : price2          =    .6165257

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _cons |   .2968633   .0531901     5.58   0.000     .1926126     .401114
------------------------------------------------------------------------------

. 
. di "marginal effect with price2 of .6165257: " 0.3528042*(exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.352
> 8042*.6165257)))*(1-exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.3528042*.6165257)))
marginal effect with price2 of .6165257: .07364277

. margins, dydx(price2) at(price2==.6165257) 

Conditional marginal effects                    Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()
dy/dx w.r.t. : price2
at           : price2          =    .6165257

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      price2 |   .0736428   .1759676     0.42   0.676    -.2712474    .4185329
------------------------------------------------------------------------------

. margins, dydx(price2) atmeans

Conditional marginal effects                    Number of obs     =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()
dy/dx w.r.t. : price2
at           : price2          =    .6165257 (mean)

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      price2 |   .0736428   .1759676     0.42   0.676    -.2712474    .4185329
------------------------------------------------------------------------------

This means that a car is 7 percentage points more likely to be foreign when the price jumps by 10K from the average price. Further up, the average increase when the price jumps by 10K from own value is pretty similar.

You can even mix the two (after fitting a model with more variables):

margins, dydx(price2) at((means) _continuous (base) _factor (asobserved) weight mpg==22)

Code For Above:

sysuse auto, clear
gen price2 = price/10000
logit foreign price2
/* (1) compare predicted probabilities versus margins */
predict phat_by_stata, pr
gen double phat_by_hand=exp(-1.079792+ .3528042*price2)/(1+exp(-1.079792+.3528042*price2)) 
sum phat*
margins
margins, asobserved
/* (2) marginal effects: the change in the probability from 1K increase in price */ 
margins, dydx(price2)
margins, dydx(price2) asobserved
sum price2
di "phat with price2 of .6165257: " exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.3528042*.6165257))
margins, at(price2==.6165257)
di "marginal effect with price2 of .6165257: " 0.3528042*(exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.3528042*.6165257)))*(1-exp(-1.079792+ .3528042*.6165257)/(1+exp(-1.079792+.3528042*.6165257)))
margins, dydx(price2) at(price2==.6165257) 
margins, dydx(price2) atmeans
margins, dydx(price2) 
$\endgroup$
1
  • $\begingroup$ @EricClick Did this help, or does it need some clarification? $\endgroup$
    – dimitriy
    Dec 21, 2016 at 23:32
3
$\begingroup$

I'll answer 1. The marginal effect is the predicted increment of the response variable associated with a unit increase in one of the covariates keeping the others constant. In linear regression, it is just the beta parameter. In logistic regression, it depends on the value of the covariate.

The predicted probability is just the predicted probability for the outcome to be 1 (the label associated with the value 1)

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.