0
$\begingroup$

I understand the output of the margins command gives the weighted average predicted probability at each value of a factor (averaged over the other confounders), but I am not clear what the p-value is referring to, that is, what is the null hypothesis being tested? If it reports 0.06 and 0.09 respectively for a two level factor with a significant p-value for both levels, what does that actually mean? How do these p-values relate to the p-values for the factor coefficients in the actual model?

This paper describes what the method used by the margins: http://ije.oxfordjournals.org/content/43/3/962.full

Specifically, the section that begins "To apply method 1 in practice after performing a logistic regression..."

I don't have specific example output, but an example output of this command is on page 12 here: http://www.stata.com/manuals13/rmargins.pdf

Improve this question
$\endgroup$
3
  • $\begingroup$ could you include some output from the margins command? Please use the code tag around your output so it's formatted correctly $\endgroup$
    – Marquis de Carabas
    Commented Dec 21, 2016 at 17:28
  • $\begingroup$ And please include a link that explains exactly what Stata is doing here. $\endgroup$
    – Frank Harrell
    Commented Dec 21, 2016 at 17:38
  • $\begingroup$ Thanks for the suggestions, I've added a few links. Another description of this method can be found in Gelman and Hill, section 5.6, but p-values are not explored. $\endgroup$
    – jeffmax
    Commented Dec 21, 2016 at 18:02

1 Answer 1

Reset to default
2
$\begingroup$

You did not provide any code, so it's hard to tell what margins is doing, since it can do a great number of things. It is good practice to provide code, or--even better--a replicable example. Stata makes this very easy.

My best guess from reading your description is that your margins is conducting a test that the average predicted probability (with own covariates and setting the factor variable to each value sequentially) is individually zero. This is usually not very meaningful. These predictions use the index coefficients that logit spits out, multiplied by the covariates and wrapped up inside a logistic() function.

Here's an example where the outcome is giving birth to low birthweight child and we will predict that as if all mothers are white, then black, and then other, holding age at own values. Then we will repeat what margins does by hand using a loop for each race:

. webuse lbw, clear
(Hosmer & Lemeshow data)

. qui logit low age i.race

. margins race, post

Predictive margins                              Number of obs     =        189
Model VCE    : OIM

Expression   : Pr(low), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        race |
      white  |   .2469149   .0446876     5.53   0.000     .1593288    .3345011
      black  |   .4069807   .0961398     4.23   0.000     .2185502    .5954113
      other  |   .3657971   .0585134     6.25   0.000      .251113    .4804812
------------------------------------------------------------------------------

. 
. /* Replicate each row of margins */
. forvalues v=1/3 {
  2.         local race: label race `v'
  3.         di _newline(1) "`race'"
  4.         di "z-stat: "    %9.2f (_b[`v'.race]-0)/_se[`v'.race]
  5.         di "p-value: "   %9.2f normal(-(_b[`v'.race]-0)/_se[`v'.race])
  6.         di "95%CI LB: "  %9.7f _b[`v'.race]-abs(invnormal(0.025))*_se[`v'.race]
  7.         di "95%CI UB: "  %9.7f _b[`v'.race]+abs(invnormal(0.025))*_se[`v'.race]
  8. }

white
z-stat:      5.53
p-value:      0.00
95%CI LB: 0.1593288
95%CI UB: 0.3345011

black
z-stat:      4.23
p-value:      0.00
95%CI LB: 0.2185502
95%CI UB: 0.5954113

other
z-stat:      6.25
p-value:      0.00
95%CI LB: 0.2511130
95%CI UB: 0.4804812

Also, here's what margins is doing under the hood to get the white prediction of .2469149:

qui logit low age i.race
replace race = 1
predict phat, pr
sum phat
Improve this answer
$\endgroup$
4
  • $\begingroup$ Got it, that makes sense- as you point out, the test of whether each marginal average predicted probability is 0 doesn't seem likely to be interesting within the context of a model with covariates. I wonder why it would be shown by default. $\endgroup$
    – jeffmax
    Commented Dec 21, 2016 at 21:25
  • 1
    $\begingroup$ @jeffmax329 My guess is that most people look at marginal effects or index coefficients, in which case the null of zero is a more or less meaningful, and margins inherits this default behavior. It is also easy enough to suppress the p-values. You can also look at contrasts of these margins or do tests on them after posting, which will accomplish the same goal. $\endgroup$
    – dimitriy
    Commented Dec 21, 2016 at 21:32
  • $\begingroup$ Another area where this bites is when you ask for exponentiated logit coefficients with the or option, in which case the baseline is 1 rather than 0, since the effect is multiplicative on the odds-ratio scale. Stata gives you the p-value for the null that $\exp(\beta)=0$, which is strange. $\endgroup$
    – dimitriy
    Commented Dec 21, 2016 at 21:35
  • $\begingroup$ @DimitriyV.Masterov Can you please explain why you calculate the z-score of the margins the same way you would calculate the z-score of the coefficients? I've written this question about it, thanks! $\endgroup$
    – cd98
    Commented Apr 7, 2018 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.